{"id":7968,"library":"azureml-fsspec","title":"AzureML fsspec Protocol Handler","description":"The `azureml-fsspec` library enables the `fsspec` (Filesystem Spec) library to interact with Azure Machine Learning datastores. It registers 'azureml://' and 'adl://' protocols, allowing users to access, read, and write files within Azure ML datastores using the familiar `fsspec` API. The current version is 1.3.1, and it's released as part of the broader Azure ML SDK ecosystem, typically following its release cadence.","status":"active","version":"1.3.1","language":"python","source_language":"en","source_url":"https://github.com/Azure/azureml-open-source-libraries/tree/main/azureml-fsspec","tags":["azure","azureml","fsspec","filesystem","cloud storage"],"install":[{"cmd":"pip install azureml-fsspec","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core dependency for filesystem abstraction, requires version >=2023.10.0.","package":"fsspec"},{"reason":"Underlying Azure Machine Learning SDK for datastore authentication and management, requires version ~=1.51.0.","package":"azureml-core"},{"reason":"Microsoft Authentication Library for Python, used for Azure AD authentication, requires >=1.20.0,<2.0.0.","package":"msal"}],"imports":[{"wrong":"from azureml.fsspec import AzureMLFileSystem","symbol":"AzureMLFileSystem","correct":"from azureml.fsspec import AzureMLFileSystem"}],"quickstart":{"code":"import fsspec\nimport azureml.fsspec # This import registers the \"azureml\" and \"adl\" protocols\nimport os\n\nprint(\"azureml-fsspec imported, protocols registered.\")\n\n# Environment variables for AzureML workspace details. \n# In a real scenario, these must be set or passed via the fsspec.filesystem() call or URI query params.\nsubscription_id = os.environ.get(\"AZUREML_SUBSCRIPTION_ID\", \"\")\nresource_group = os.environ.get(\"AZUREML_RESOURCE_GROUP\", \"\")\nworkspace_name = os.environ.get(\"AZUREML_WORKSPACE_NAME\", \"\")\ndatastore_name = os.environ.get(\"AZUREML_DATASTORE_NAME\", \"workspaceblobstore\") # Common default datastore name\n\nif all([subscription_id, resource_group, workspace_name]):\n    print(f\"\\nAttempting to get AzureML filesystem instance for datastore: {datastore_name}...\")\n    try:\n        # Get a filesystem instance using specific workspace details and datastore.\n        # This approach explicitly provides connection details.\n        fs = fsspec.filesystem(\"azureml\",\n                                subscription_id=subscription_id,\n                                resource_group=resource_group,\n                                workspace_name=workspace_name,\n                                datastore_name=datastore_name)\n        print(f\"Successfully initialized AzureML filesystem object: {type(fs)}\")\n\n        # Example: Try to list the root of the specified datastore. \n        # This will only succeed with valid credentials and permissions.\n        # path_to_list = f\"azureml://datastores/{datastore_name}/paths/\"\n        # print(f\"Attempting to list contents of: {path_to_list}\")\n        # contents = fs.ls(\"/\") # Listing root of the specific datastore instance\n        # print(f\"First 5 items from {datastore_name} root: {contents[:5]}...\") \n        # print(f\"Total items found: {len(contents)}\")\n\n    except Exception as e:\n        print(f\"Could not initialize AzureML filesystem (this is common without a full setup): {e}\")\n        print(\"Please ensure AZUREML_SUBSCRIPTION_ID, AZUREML_RESOURCE_GROUP, AZUREML_WORKSPACE_NAME, and AZUREML_DATASTORE_NAME are correctly set as environment variables or passed as arguments.\")\nelse:\n    print(\"\\nSkipping AzureML filesystem initialization and IO: AzureML environment variables not fully set.\")\n    print(\"Set AZUREML_SUBSCRIPTION_ID, AZUREML_RESOURCE_GROUP, AZUREML_WORKSPACE_NAME, and AZUREML_DATASTORE_NAME for a live example.\")\n\nprint(\"\\nDemonstrating direct import of AzureMLFileSystem class:\")\nfrom azureml.fsspec import AzureMLFileSystem\nprint(f\"AzureMLFileSystem class found: {AzureMLFileSystem}\")","lang":"python","description":"This quickstart demonstrates how to import `azureml-fsspec` to register its protocols with `fsspec` and how to obtain an `AzureMLFileSystem` instance. It highlights the need for Azure ML workspace details (subscription ID, resource group, workspace name, datastore name), typically provided via environment variables. Actual I/O operations (like listing files) are commented out, as they require a live Azure ML setup with valid credentials and permissions to succeed without errors."},"warnings":[{"fix":"Ensure your environment is correctly authenticated with Azure. Use `az login` for Azure CLI, or set appropriate environment variables for service principal or managed identity authentication before running your Python code.","message":"AzureML authentication is required and can be complex. This library relies on the underlying AzureML SDK's authentication mechanisms, which can include environment variables (e.g., AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID), Azure CLI login, or managed identity.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Always include `import azureml.fsspec` at the beginning of your script when intending to use `fsspec` with Azure ML datastores. This import performs the necessary protocol registration.","message":"Forgetting to import `azureml.fsspec` will prevent the 'azureml' and 'adl' protocols from being registered with `fsspec`, leading to 'No such protocol' errors.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Ensure your URI follows the pattern `azureml://datastores/<datastore_name>/paths/<path_to_file>` and that workspace details are provided either in the URI (as query parameters), as `fsspec.filesystem` arguments, or via environment variables. `adl://` is specific to ADLS Gen2 datastores.","message":"Incorrect or incomplete Azure ML URI format can lead to errors. The `azureml://` protocol expects specific parameters (subscription ID, resource group, workspace name, datastore name, and path). These can be in the URI itself or passed as keyword arguments to `fsspec.filesystem()`.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Check `pip list` and ensure `fsspec>=2023.10.0` and `azureml-core~=1.51.0` (or compatible versions) are installed. Upgrade with `pip install --upgrade fsspec azureml-core` if needed.","message":"Requires specific minimum versions of `fsspec` and `azureml-core`. Older versions of these dependencies might not provide the necessary APIs or features.","severity":"gotcha","affected_versions":"<1.3.0"}],"env_vars":null,"search_vec":"'1.3.1':48 'access':30 'allow':27 'api':43 'azur':19,36,58,67 'azureml':1,7,68 'azureml-fsspec':6 'broader':57 'cadenc':66 'cloud':71 'current':45 'datastor':22,38 'ecosystem':61 'enabl':10 'familiar':41 'file':34 'filesystem':13,70 'follow':63 'fsspec':2,8,12,42,69 'handler':4 'interact':17 'learn':21 'librari':9,15 'machin':20 'ml':37,59 'part':54 'protocol':3,26 'read':31 'regist':24 'releas':52,65 'sdk':60 'spec':14 'storag':72 'typic':62 'use':39 'user':28 'version':46 'within':35 'write':33","created_at":"2026-04-16T16:59:44.392142+00:00","updated_at":"2026-04-16T16:59:44.392142+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\n  File \"/tmp/tmpdu1gezzf/venv/lib/python3.12/site-packages/azureml/fsspec/__init__.py\", line 9, in <module>\n    from .spec import AzureMachineLearningFileSystem\n  File \"/tmp/tmpdu1gezzf/venv/lib/python3.12/site-packages/azurem"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"1.3.1","cli_name":"","cli_version":null,"type":"library","homepage":"https://docs.microsoft.com/python/api/overview/azure/ml/?view=azure-ml-py","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/azureml-fsspec/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["azure"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"import_fail","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-07-10","install_tag":null}}