{"id":3942,"library":"correctionlib","title":"CorrectionLib","description":"CorrectionLib is a generic Python library providing a well-structured JSON data format for various ad-hoc correction factors commonly found in high-energy physics (HEP) analysis. It offers a companion evaluation tool suitable for use in C++ and Python programs, supporting multi-dimensional binned lookups, categorical maps, and formulas. The library is actively maintained, with its current version being 2.8.0, and releases occur regularly to introduce new features, improve compatibility, and address bug fixes.","status":"active","version":"2.8.0","language":"python","source_language":"en","source_url":"https://github.com/cms-nanoAOD/correctionlib","tags":["physics","HEP","data analysis","correction factors","awkward-array","numpy","json-schema","pydantic"],"install":[{"cmd":"pip install correctionlib","lang":"bash","label":"PyPI"}],"dependencies":[{"reason":"Used for vectorized evaluation and array manipulation.","package":"numpy","optional":false},{"reason":"Used for vectorized evaluation, especially with jagged array inputs/outputs.","package":"awkward","optional":false},{"reason":"Used for schema validation and parsing of correction definitions.","package":"pydantic","optional":false},{"reason":"Core component for Python bindings to the C++ evaluator.","package":"pybind11","optional":false}],"imports":[{"symbol":"CorrectionSet","correct":"from correctionlib import CorrectionSet"},{"symbol":"schemav2","correct":"from correctionlib import schemav2"}],"quickstart":{"code":"import correctionlib\nimport os\nimport json\n\n# Create a dummy JSON correction file content\ncorrection_json_content = '''\n{\n  \"schema_version\": 2,\n  \"description\": \"Simple example corrections\",\n  \"corrections\": [\n    {\n      \"version\": 1,\n      \"name\": \"my_sf\",\n      \"inputs\": [\n        {\"name\": \"x\", \"type\": \"real\"},\n        {\"name\": \"y\", \"type\": \"real\"}\n      ],\n      \"output\": {\"name\": \"scale_factor\", \"type\": \"real\"},\n      \"data\": {\n        \"nodetype\": \"binning\",\n        \"input\": \"x\",\n        \"edges\": [0.0, 5.0, 10.0],\n        \"content\": [\n          {\n            \"nodetype\": \"binning\",\n            \"input\": \"y\",\n            \"edges\": [0.0, 5.0, 10.0],\n            \"content\": [\n              0.9,\n              1.0\n            ]\n          },\n          {\n            \"nodetype\": \"binning\",\n            \"input\": \"y\",\n            \"edges\": [0.0, 5.0, 10.0],\n            \"content\": [\n              1.1,\n              1.2\n            ]\n          }\n        ],\n        \"flow\": \"clamp\"\n      }\n    }\n  ]\n}\n'''\n\n# Save the content to a temporary file\nfile_path = \"example_correction.json\"\nwith open(file_path, \"w\") as f:\n    f.write(correction_json_content)\n\n# Load the corrections from the file\ncorrections = correctionlib.CorrectionSet.from_file(file_path)\n\n# Access a specific correction\nmy_correction = corrections[\"my_sf\"]\n\n# Evaluate the correction\n# Example 1: x=3.0, y=2.0 (falls in first x-bin, first y-bin)\nresult1 = my_correction.evaluate(3.0, 2.0)\nprint(f\"Correction for x=3.0, y=2.0: {result1}\")\n\n# Example 2: x=7.0, y=6.0 (falls in second x-bin, second y-bin)\nresult2 = my_correction.evaluate(7.0, 6.0)\nprint(f\"Correction for x=7.0, y=6.0: {result2}\")\n\n# Clean up the temporary file\nos.remove(file_path)\n","lang":"python","description":"This quickstart demonstrates how to define a simple 2D binned correction using CorrectionLib's JSON schema, save it to a file, and then load and evaluate it in Python. It shows how to retrieve a specific correction from a `CorrectionSet` and calculate the corrected value for given inputs."},"warnings":[{"fix":"If `numpy` array output is required, explicitly call `.to_numpy()` on the result, e.g., `my_correction.evaluate(...).to_numpy()`.","message":"Starting with v2.8.0, the `evaluate()` method will return flat `awkward` arrays when flat `awkward` arrays are passed as input. Previously, it would return `numpy` arrays in this scenario.","severity":"breaking","affected_versions":">=2.8.0"},{"fix":"Upgrade CorrectionLib to version 2.6.0 or higher to ensure correct vectorized evaluation with `numpy` 2.0+.","message":"Versions of CorrectionLib prior to v2.6.0 exhibit silent wrong results for vectorized evaluation when used with `numpy` 2.0. This is a critical compatibility issue.","severity":"breaking","affected_versions":"<2.6.0 with numpy >=2.0"},{"fix":"Update your correction JSON files to use `\"inf\"` or `\"-inf\"` strings for infinite bin edges as per the updated schema. Version 2.6.1 allows reading older JSONs for backwards compatibility, but authoring new ones should use strings.","message":"Since v2.6.0, the representation of infinities in bin edges for correction definitions has changed from floating-point values to string literals (`\"inf\"` or `\"-inf\"`). Older JSON files might not be forward-compatible without conversion.","severity":"breaking","affected_versions":">=2.6.0 (for JSON definitions)"},{"fix":"Ensure your environment is running Python 3.9 or a more recent compatible version.","message":"Python 3.6 support has been officially dropped starting from v2.5.0. The library now requires Python 3.9 or newer.","severity":"breaking","affected_versions":">=2.5.0"},{"fix":"Consult the official `pydantic` v1 to v2 migration guide. `bump-pydantic` is a tool available to assist with automated code migration.","message":"CorrectionLib migrated to `pydantic` v2 in v2.5.0. Users interacting directly with CorrectionLib's schema models (e.g., `correctionlib.schemav2.CorrectionSet`) should be aware of `pydantic` v1 vs v2 breaking changes, particularly method renames (e.g., `dict()` to `model_dump()`) and configuration changes.","severity":"gotcha","affected_versions":">=2.5.0"},{"fix":"To avoid ABI issues, install `correctionlib` by forcing a source build within your environment: `pip install --no-binary=correctionlib correctionlib`. This ensures C++ components are compiled with compatible settings.","message":"When installing `correctionlib` and planning to use its C++ evaluator (e.g., from ROOT or a standalone executable), C++ ABI compatibility between the installed library and your environment is crucial. Mismatches can lead to linker errors.","severity":"gotcha","affected_versions":"All versions (when compiling from source/using C++ bindings)"}],"env_vars":null,"search_vec":"'2.8.0':66 'activ':59 'ad':19 'ad-hoc':18 'address':78 'analysi':31,84 'array':89 'awkward':88 'awkward-array':87 'bin':50 'bug':79 'c':42 'categor':52 'common':23 'companion':35 'compat':76 'correct':21,85 'correctionlib':1,2 'current':63 'data':14,83 'dimension':49 'energi':28 'evalu':36 'factor':22,86 'featur':74 'fix':80 'format':15 'formula':55 'found':24 'generic':5 'hep':30,82 'high':27 'high-energi':26 'hoc':20 'improv':75 'introduc':72 'json':13,92 'json-schema':91 'librari':7,57 'lookup':51 'maintain':60 'map':53 'multi':48 'multi-dimension':47 'new':73 'numpi':90 'occur':69 'offer':33 'physic':29,81 'program':45 'provid':8 'pydant':94 'python':6,44 'regular':70 'releas':68 'schema':93 'structur':12 'suitabl':38 'support':46 'tool':37 'use':40 'various':17 'version':64 'well':11 'well-structur':10","created_at":"2026-04-12T03:34:14.520945+00:00","updated_at":"2026-04-16T03:52:59.619935+00:00","problems":[{"fix":"Install `correctionlib` from source to ensure its C++ components are built with the compatible ABI of your environment. Use `pip install --no-binary=correctionlib correctionlib`. For CMSSW, specific installation recipes might be required.","cause":"This linker error typically occurs in C++ environments (like ROOT or CMSSW) when the C++ components of `correctionlib` (e.g., installed via Python wheels) have an ABI incompatibility with the C++ compiler/environment being used.","error":"undefined symbol: _ZN10correction13CorrectionSet9from_file"},{"fix":"Explicitly construct `std::string` objects for string literals when passing them to `evaluate`, for example, `corr->evaluate({std::string(\"central\"), std::string(\"M\"), 5, 1.2, 240.})`. Ensure the `correctionlib` C++ library is built with a compiler and C++ standard compatible with the calling environment.","cause":"This C++ compilation error, often seen when calling `Correction::evaluate` within ROOT or similar environments, arises from type mismatches or ABI incompatibilities when converting arguments to `std::vector<std::variant<int, double, std::string>>`. String literals might not be correctly implicitly converted to `std::string` for the `std::variant` type.","error":"error: cannot cast private base class 'std::__detail::__variant::_Variant_storage<false, int, double, std::__cxx11::basic_string<char> >' to 'std::variant<int, double, std::__cxx11::basic_string<char> >'"},{"fix":"Review your JSON file against the `correctionlib` schema documentation (e.g., `schemav2.html`) and ensure all corrections, binnings, categories, and other elements are correctly structured and typed. Tools or IDEs might offer validation against the published JSON schema URL.","cause":"The input JSON file provided to `correctionlib` does not conform to its defined schema (Schema v1 or v2). This can be due to incorrect field names, missing required fields, invalid types for values, or structural inconsistencies in the correction definitions.","error":"pydantic.ValidationError"},{"fix":"Install the package using `pip install correctionlib`. If you are working within a virtual environment, ensure it is activated. For specific environments like CMSSW, follow the recommended installation procedures.","cause":"The `correctionlib` Python package is not installed in your current Python environment, or the Python interpreter you are using does not have access to the installed package.","error":"ModuleNotFoundError: No module named 'correctionlib'"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"2.9.0","cli_name":"correction","cli_version":"usage: correction [-h] [--width WIDTH] [--html HTML]","type":"library","homepage":"https://cms-nanoaod.github.io/correctionlib","github":"https://github.com/cms-nanoAOD/correctionlib","docs":null,"changelog":null,"pypi":"https://pypi.org/project/correctionlib/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","serialization"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-29","next_check":"2026-07-28","install_tag":null}}