{"id":2621,"library":"obstore","title":"obstore: Python Object Storage Interface","description":"obstore is a Python library providing a simple, high-throughput interface for various object storage services like Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compliant APIs. It features both synchronous and asynchronous APIs, streaming downloads/uploads, and automatic multipart uploads for large files, powered by a Rust backend for performance. The current version is 0.9.2, with a frequent release cadence, often introducing minor updates and fixes.","status":"active","version":"0.9.2","language":"python","source_language":"en","source_url":"https://github.com/developmentseed/obstore","tags":["object-storage","s3","gcs","azure-blob-storage","cloud-storage","async","rust-powered","high-performance"],"install":[{"cmd":"pip install obstore","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for `S3Store.from_session()` to integrate with AWS sessions.","package":"boto3","optional":true},{"reason":"Required for `obstore.list(return_arrow=True)` to enable zero-copy Arrow integration for listing results.","package":"arro3-core","optional":true}],"imports":[{"symbol":"obstore","correct":"import obstore as obs"},{"note":"Store classes like MemoryStore, S3Store, GCSStore are located in the `obstore.store` submodule, not directly under `obstore`.","wrong":"import obstore.MemoryStore","symbol":"MemoryStore","correct":"from obstore.store import MemoryStore"},{"note":"All operations (put, get, list, delete, copy, etc.) are top-level functions exported by the `obstore` module, not methods of the store object itself.","wrong":"store.put('path', b'data')","symbol":"put","correct":"obs.put(store, 'path', b'data')"}],"quickstart":{"code":"import obstore as obs\nfrom obstore.store import MemoryStore\n\n# Initialize an in-memory store for demonstration\nstore = MemoryStore()\n\n# Define a file path and content\nfile_path = \"my_document.txt\"\nfile_content = b\"Hello, obstore world!\"\n\n# Put the object into the store\nobs.put(store, file_path, file_content)\nprint(f\"Object '{file_path}' put into store.\")\n\n# Get the object from the store\nresponse = obs.get(store, file_path)\nretrieved_content = response.bytes()\nprint(f\"Retrieved content: {retrieved_content.decode()}\")\n\nassert retrieved_content == file_content\nprint(\"Content matches!\")\n\n# Asynchronous operations are also available (requires an async context)\n# async def main():\n#    await obs.put_async(store, 'async_file.txt', b'async data')\n#    res_async = await obs.get_async(store, 'async_file.txt')\n#    print(f\"Async retrieved: {res_async.bytes().decode()}\")\n# import asyncio\n# asyncio.run(main())","lang":"python","description":"This quickstart demonstrates how to initialize an in-memory object store and perform basic synchronous `put` and `get` operations. Note that operations like `put` and `get` are top-level functions within the `obstore` module, taking the store instance as their first argument. Asynchronous counterparts (e.g., `put_async`, `get_async`) are also available."},"warnings":[{"fix":"Upgrade Python to 3.10 or newer.","message":"Support for Python 3.9 was deprecated in `obstore` version 0.9.0. Users on Python 3.9 should upgrade their Python environment.","severity":"breaking","affected_versions":">=0.9.0"},{"fix":"Refactor S3 store construction to use explicit credential providers or environment variable configuration instead of `from_session`.","message":"In older versions (prior to 0.7.0), `S3Store.from_session()` and `S3Store._from_native()` were removed. Users should transition to using credential providers for S3 authentication.","severity":"breaking","affected_versions":"<0.7.0"},{"fix":"Review all path inputs to `obstore` functions and ensure they are properly formed (e.g., not already percent-encoded if they contain special characters).","message":"Path encoding behavior changed in version 0.8.2 to prevent unintentional double-encoding. Users must now ensure that paths provided to `obstore` are valid and correctly encoded.","severity":"breaking","affected_versions":">=0.8.2"},{"fix":"Update `AzureStore` constructor calls to use `container_name='my-container'` instead of `container='my-container'`.","message":"In version 0.5.0, the `container` parameter for `AzureStore`'s constructor was renamed to `container_name` and became a keyword-only argument. Using `container` will raise an error.","severity":"breaking","affected_versions":">=0.5.0"},{"fix":"Always call object storage operations as `obstore.<function_name>(store_instance, ...)`.","message":"Unlike many object storage libraries, `obstore`'s core operations (`put`, `get`, `list`, `delete`, `copy`, etc.) are top-level functions (e.g., `obs.put(store, ...)`) rather than methods on the store object (e.g., `store.put(...)`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"If using `return_arrow=True`, ensure `pip install obstore[arrow]` or `pip install arro3-core` is run.","message":"When listing objects with `obstore.list(return_arrow=True)`, the optional `arro3-core` dependency is required. Without it, attempting to use this feature will fail.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'0.9.2':64 'amazon':24 'api':36,43 'async':88 'asynchron':42 'automat':47 'azur':29,82 'azure-blob-storag':81 'backend':57 'blob':30,83 'cadenc':69 'cloud':27,86 'cloud-storag':85 'compliant':35 'current':61 'downloads/uploads':45 'featur':38 'file':52 'fix':75 'frequent':67 'gcs':80 'googl':26 'high':15,93 'high-perform':92 'high-throughput':14 'interfac':5,17 'introduc':71 'larg':51 'librari':10 'like':23 'minor':72 'multipart':48 'object':3,20,77 'object-storag':76 'obstor':1,6 'often':70 'perform':59,94 'power':53,91 'provid':11 'python':2,9 'releas':68 'rust':56,90 'rust-pow':89 's3':25,34,79 's3-compliant':33 'servic':22 'simpl':13 'storag':4,21,28,31,78,84,87 'stream':44 'synchron':40 'throughput':16 'updat':73 'upload':49 'various':19 'version':62","created_at":"2026-04-11T01:35:48.826224+00:00","updated_at":"2026-04-16T17:30:50.817203+00:00","problems":[{"fix":"Install the 'obstore' library using pip: `pip install obstore`","cause":"The 'obstore' library is not installed in the Python environment where the code is being executed.","error":"ModuleNotFoundError: No module named 'obstore'"},{"fix":"Explicitly pass the correct `region` parameter when initializing `S3Store`, or ensure the `AWS_REGION` environment variable is correctly set. For example: `store = S3Store(\"your-bucket-name\", region=\"us-west-2\")`","cause":"When interacting with an S3 bucket, the AWS region was not specified or was configured incorrectly during `S3Store` initialization.","error":"GenericError: Generic S3 error: Error performing list request: Received redirect without LOCATION, this normally indicates an incorrectly configured region."},{"fix":"Use the correct capitalization for `PutMode` values (e.g., `mode=\"Create\"` or `mode=\"Overwrite\"`). Refer to the `PutMode` documentation for supported values. For example: `await obs.put_async(store, key, buf, mode=\"Create\")`","cause":"The 'mode' parameter for `put_async` (and `put`) requires specific capitalization (e.g., 'Create' instead of 'create'), or the requested put mode is not yet fully supported by the underlying S3 implementation for the given store configuration.","error":"NotImplementedError (when using mode=\"create\" with put_async)"},{"fix":"Provide valid authentication credentials via environment variables (e.g., `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), configuration parameters during store initialization, or by setting up a custom credential provider.","cause":"The credentials used to access the object storage service are invalid, expired, or missing, leading to an authentication failure.","error":"obstore.exceptions.UnauthenticatedError"},{"fix":"Verify that the `path` argument provided to operations like `get`, `head`, or `delete` correctly points to an existing object within the storage bucket.","cause":"The specified object or path does not exist in the configured object storage location.","error":"obstore.exceptions.NotFoundError"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.11.1","cli_name":"","cli_version":null,"type":"library","homepage":"https://developmentseed.org/obstore","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/obstore/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["aws","gcp","azure","data","http-networking"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-28","next_check":"2026-07-28","install_tag":null}}