{"id":5141,"library":"cached-path","title":"cached-path","description":"cached-path is a Python file utility library that provides a unified, simple interface for accessing both local and remote files. It automatically downloads and caches remote resources, making them available as local file paths. Currently at version 1.8.10, the library maintains an active development pace with frequent patch and minor releases to address compatibility and add new features.","status":"active","version":"1.8.10","language":"python","source_language":"en","source_url":"https://github.com/allenai/cached_path","tags":["file-management","caching","remote-files","cloud-storage"],"install":[{"cmd":"pip install cached-path","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for accessing resources from AWS S3 (s3:// scheme).","package":"boto3","optional":true},{"reason":"Required for accessing resources from Google Cloud Storage (gs:// scheme).","package":"google-cloud-storage","optional":true},{"reason":"Required for accessing resources from HuggingFace Hub (hf:// scheme).","package":"huggingface-hub","optional":true},{"reason":"Required for accessing resources from Beaker (beaker:// scheme).","package":"beaker-py","optional":true}],"imports":[{"symbol":"cached_path","correct":"from cached_path import cached_path"},{"symbol":"get_cache_dir","correct":"from cached_path import get_cache_dir"}],"quickstart":{"code":"import os\nfrom cached_path import cached_path\n\n# Download and cache a remote file\nremote_url = \"https://raw.githubusercontent.com/allenai/cached_path/main/README.md\"\nlocal_path = cached_path(remote_url)\nprint(f\"Cached file path: {local_path}\")\nassert os.path.exists(local_path)\n\n# Example with an archive, extracting it\narchive_url = \"https://github.com/allenai/cached_path/releases/download/v0.1.0/cached_path-0.1.0.tar.gz\"\nextracted_dir = cached_path(archive_url, extract_archive=True)\nprint(f\"Extracted archive directory: {extracted_dir}\")\nassert os.path.isdir(extracted_dir)\n\n# Clean up (optional, for demonstration)\n# import shutil\n# shutil.rmtree(os.path.dirname(local_path))\n# shutil.rmtree(extracted_dir)","lang":"python","description":"This quickstart demonstrates how to use `cached_path()` to download and cache a remote file, and how to extract an archive. It verifies that the returned paths exist locally."},"warnings":[{"fix":"Upgrade cached-path to version 1.7.2 or later to ensure compatibility with recent `boto3` and `botocore` releases.","message":"Older versions of cached-path (pre-1.7.2) had incompatibility issues with `boto3/botocore >=1.37.34`. Upgrading `boto3` or `botocore` without updating `cached-path` could lead to errors when accessing S3 resources.","severity":"breaking","affected_versions":"<1.7.2"},{"fix":"If you rely on capturing download progress, redirect `stderr` instead of `stdout` or configure a custom progress display.","message":"As of v1.8.10, the default progress bar output for downloads was changed to write to `stderr` instead of `stdout`. Scripts redirecting `stdout` might no longer capture progress information.","severity":"gotcha","affected_versions":">=1.8.10"},{"fix":"Upgrade to version 1.8.9 or later to benefit from improved data integrity during file caching operations.","message":"Version 1.8.9 introduced a fix to ensure filesystem sync (`os.fdatasync`) when replacing temporary files with permanent ones. While a data integrity improvement, it implies that prior versions could be susceptible to data loss or corruption in case of power failure or system crash during a file download/replacement operation.","severity":"gotcha","affected_versions":"<1.8.9"},{"fix":"Keep `cached-path` updated to the latest version, especially when updating `huggingface-hub`, to ensure full compatibility with the `hf://` scheme.","message":"Multiple versions (1.6.7, 1.7.3, 1.7.4) added support for newer `HuggingFace-Hub` versions. Using an older `cached-path` with a very new `huggingface-hub` might lead to unexpected behavior or errors when using the `hf://` scheme. Always ensure `cached-path` is kept up-to-date when working with `huggingface-hub`.","severity":"gotcha","affected_versions":"<1.7.4"},{"fix":"Understand the distinction: `extract_archive=True` extracts the full archive, while `url!path/to/file` extracts and points to a specific entry within the archive.","message":"When using `extract_archive=True`, `cached_path` extracts the entire archive and returns the path to the extracted directory. To access a specific file or sub-directory within an archive without extracting everything, append `!/path/to/file` to the URL (e.g., `hf://model!config.json`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be explicit about your desired cache directory and understand the precedence: argument > `set_cache_dir()` > environment variable.","message":"The default cache directory is `~/.cache/cached_path/`. This can be overridden globally via the `CACHED_PATH_CACHE_ROOT` environment variable, programmatically with `set_cache_dir()`, or per-call using the `cache_dir` argument to `cached_path()`. Conflicting settings might lead to unexpected cache locations.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'1.8.10':43 'access':20 'activ':48 'add':61 'address':58 'automat':27 'avail':35 'cach':2,5,30,67 'cached-path':1,4 'cloud':72 'cloud-storag':71 'compat':59 'current':40 'develop':49 'download':28 'featur':63 'file':10,25,38,65,70 'file-manag':64 'frequent':52 'interfac':18 'librari':12,45 'local':22,37 'maintain':46 'make':33 'manag':66 'minor':55 'new':62 'pace':50 'patch':53 'path':3,6,39 'provid':14 'python':9 'releas':56 'remot':24,31,69 'remote-fil':68 'resourc':32 'simpl':17 'storag':73 'unifi':16 'util':11 'version':42","created_at":"2026-04-14T01:20:58.539604+00:00","updated_at":"2026-04-16T01:13:29.531072+00:00","problems":[{"fix":"Install the 'cached-path' library using pip: `pip install cached-path`.","cause":"The 'cached-path' library is not installed in the current Python environment, or there's a naming conflict or an incorrect import from another library that previously exposed a 'cached_path' utility.","error":"ModuleNotFoundError: No module named 'cached_path'"},{"fix":"Verify the correctness of the file path or URL. Ensure the file exists and is accessible. For long paths on Windows, consider enabling long path support in the OS or setting a shorter `cache_dir`.","cause":"The local file path or remote URL provided to `cached_path()` does not exist, is inaccessible due to permissions, or is too long for the operating system (e.g., Windows).","error":"FileNotFoundError: [Errno 2] No such file or directory"},{"fix":"Correct the URL string to follow a valid format and ensure it uses a supported scheme as outlined in the `cached-path` documentation.","cause":"The string provided to `cached_path()` is not a valid URL format or does not use one of the supported schemes (e.g., 'http', 'https', 's3', 'gs', 'hf').","error":"ValueError: When the URL is invalid"},{"fix":"Check the `cached-path` library's changelog or documentation for the specific version you are using. Update your code to be compatible, or upgrade/downgrade `cached-path` to a compatible version (e.g., `pip install cached-path==X.Y.Z`).","cause":"The installed version of `cached-path` is incompatible with the code attempting to use it, or a specific attribute or function has been removed, renamed, or is not available in that version.","error":"AttributeError: module 'cached_path' has no attribute 'some_attribute'"},{"fix":"Enable long path support in your Windows OS settings. Alternatively, specify a shorter custom cache directory using `cached_path(url_or_filename, cache_dir='/short/path')` or `cached_path.set_cache_dir('/short/path')`.","cause":"On Windows, the default maximum path length can be exceeded by cached files, leading to silent failures or `FileNotFoundError` when `cached-path` tries to create or access very long paths.","error":"cached-path windows path length limitation"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"1.8.10","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/allenai/cached_path","docs":"https://cached-path.readthedocs.io/","changelog":"https://github.com/allenai/cached_path/blob/main/CHANGELOG.md","pypi":"https://pypi.org/project/cached-path/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["http-networking","serialization"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-30","next_check":"2026-07-28","install_tag":null}}