{"id":5775,"library":"ir-datasets","title":"IR Datasets","description":"ir_datasets provides a common interface to many Information Retrieval (IR) ad-hoc ranking benchmarks, training datasets, and more. It handles downloading, extracting, and providing a unified iterator format for various IR datasets. The library is actively maintained with frequent updates, currently at version 0.5.11, with new datasets and bug fixes released regularly.","status":"active","version":"0.5.11","language":"python","source_language":"en","source_url":"https://github.com/allenai/ir_datasets","tags":["information retrieval","datasets","benchmarks","NLP"],"install":[{"cmd":"pip install ir-datasets","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Commonly used for evaluating IR experiments, integrates directly with ir_datasets qrels.","package":"ir-measures","optional":true},{"reason":"Popular IR experimentation toolkit that integrates with ir_datasets.","package":"PyTerrier","optional":true}],"imports":[{"symbol":"ir_datasets","correct":"import ir_datasets"}],"quickstart":{"code":"import ir_datasets\n\n# Load a dataset, e.g., MS-MARCO passage ranking training set\ndataset = ir_datasets.load('msmarco-passage/train')\n\n# Iterate through documents\nprint(\"First 3 documents:\")\nfor i, doc in enumerate(dataset.docs_iter()):\n    print(f\"  Doc ID: {doc.doc_id}, Text: {doc.text[:70]}...\")\n    if i >= 2: break\n\n# Iterate through queries\nprint(\"\\nFirst 3 queries:\")\nfor i, query in enumerate(dataset.queries_iter()):\n    print(f\"  Query ID: {query.query_id}, Text: {query.text}\")\n    if i >= 2: break\n\n# Access relevance judgments (qrels)\nprint(\"\\nFirst 3 qrels:\")\nfor i, qrel in enumerate(dataset.qrels_iter()):\n    print(f\"  Query ID: {qrel.query_id}, Doc ID: {qrel.doc_id}, Relevance: {qrel.relevance}\")\n    if i >= 2: break","lang":"python","description":"This quickstart demonstrates how to load a dataset, such as MS MARCO, and iterate through its documents, queries, and relevance judgments using the `ir_datasets.load()` function and its iterator methods."},"warnings":[{"fix":"Prefer `dataset.docs_iter()`, `dataset.queries_iter()`, and `dataset.qrels_iter()` for memory-efficient processing of large datasets. Only use dictionary-based access like `dataset.qrels_dict()` when you are certain the data will fit in memory.","message":"Converting iterators to dictionaries (e.g., `dataset.qrels_dict()`) will load the entire dataset's relevance judgments into memory. For very large datasets, this can lead to high memory consumption and potential crashes. Use iterators (`dataset.qrels_iter()`) for memory efficiency with large collections.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For stable code, rely on the official Python API methods like `dataset.docs_iter()` as described in the main documentation. If using the Beta API, be aware of its experimental nature and potential for changes.","message":"ir_datasets includes a 'Beta Python API' which offers alternative access patterns (e.g., `dataset.docs` as an iterable object with slicing). This API is experimental, may contain bugs, and is subject to breaking changes in future versions.","severity":"gotcha","affected_versions":"Versions 0.5.0 and later (since beta API introduction)"},{"fix":"Always check the specific dataset's documentation or the output from `ir_datasets` when attempting to load a new collection. Be prepared to follow manual instructions for non-public datasets.","message":"Some datasets are not publicly available and require manual steps (e.g., data usage agreements, local file paths) to access. `ir_datasets` will provide instructions on how to obtain these datasets, but it cannot automate their download in all cases.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'0.5.11':48 'activ':40 'ad':15 'ad-hoc':14 'benchmark':18,60 'bug':53 'common':7 'current':45 'dataset':2,4,20,36,51,59 'download':25 'extract':26 'fix':54 'format':32 'frequent':43 'handl':24 'hoc':16 'inform':11,57 'interfac':8 'ir':1,3,13,35 'iter':31 'librari':38 'maintain':41 'mani':10 'new':50 'nlp':61 'provid':5,28 'rank':17 'regular':56 'releas':55 'retriev':12,58 'train':19 'unifi':30 'updat':44 'various':34 'version':47","created_at":"2026-04-14T05:06:35.467932+00:00","updated_at":"2026-04-16T15:48:12.208826+00:00","problems":[{"fix":"Install the library using `pip install ir-datasets` or activate the correct virtual environment where it is installed.","cause":"The ir-datasets library is not installed in the current Python environment or the environment is not activated correctly.","error":"ModuleNotFoundError: No module named 'ir_datasets'"},{"fix":"Verify the correct dataset ID from the `ir_datasets` documentation (e.g., `ir_datasets.list_datasets()`) and ensure it is spelled correctly.","cause":"The specified dataset ID or name does not exist or is misspelled in the ir-datasets registry.","error":"KeyError: 'dataset_id not found' (or similar, e.g., 'KeyError: 'dataset_name'')'"},{"fix":"Clear the `ir_datasets` cache (e.g., by deleting the `~/.ir_datasets/` directory) and retry the dataset access. Ensure sufficient disk space and stable network connectivity, and verify the `IR_DATASETS_HOME` environment variable if using a custom data directory.","cause":"A required dataset file was not found, likely due to a failed or incomplete download, corrupted cache, or incorrect path configuration for the dataset storage.","error":"FileNotFoundError: [Errno 2] No such file or directory: '...' (pointing to a dataset file path)"},{"fix":"Install the necessary optional dependency. For example, if 'warc' is missing, run `pip install ir-datasets[warc]`. Consult the `ir-datasets` documentation for the correct optional dependencies for your dataset.","cause":"The dataset you are trying to access requires an optional dependency for a specific data format (e.g., WARC, Z-compressed files) which has not been installed.","error":"ModuleNotFoundError: No module named 'warc' (or 'unlzw3', 'lz4', etc.)"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.6.3","cli_name":"","cli_version":null,"type":"library","homepage":"https://ir-datasets.com/","github":"https://github.com/allenai/ir_datasets","docs":"https://project.readthedocs.io/","changelog":null,"pypi":"https://pypi.org/project/ir-datasets/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","ai-ml"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-31","next_check":"2026-07-28","install_tag":null}}