{"id":5860,"library":"azureml-dataset-runtime","title":"Azure Machine Learning Dataset Runtime","description":"azureml-dataset-runtime is an internal Python package within the Azure Machine Learning SDK ecosystem, primarily designed to coordinate dependencies among other AzureML packages. It is not intended for direct use by end-users. As of version 1.62.0, released in February 2026, it maintains an active release cadence, reflecting ongoing development in the broader AzureML SDK.","status":"active","version":"1.62.0","language":"python","source_language":"en","source_url":"https://pypi.org/project/azureml-dataset-runtime/","tags":["azure","machine-learning","microsoft","internal","dependency"],"install":[{"cmd":"pip install azureml-dataset-runtime","lang":"bash","label":"Latest stable version"}],"dependencies":[{"reason":"Required for data preparation functionalities within AzureML SDK.","package":"azureml-dataprep","optional":false},{"reason":"Required for numerical operations; specific version constraints are a known footgun.","package":"numpy","optional":false},{"reason":"Required for efficient data handling, particularly with Apache Arrow format.","package":"pyarrow","optional":false},{"reason":"Optional dependency, likely for FUSE-based file system operations.","package":"fusepy","optional":true},{"reason":"Optional dependency, commonly used for data manipulation.","package":"pandas","optional":true},{"reason":"Optional dependency, for integration with Apache Spark.","package":"pyspark","optional":true},{"reason":"Optional dependency, for scientific computing.","package":"scipy","optional":true}],"imports":[],"quickstart":{"code":"# This package is internal and not intended for direct use.\n# Users typically install it as a dependency for other AzureML SDK components.\n# Here's an example of a conda environment file where it might appear as a dependency:\n\n# name: my-azureml-env\n# channels:\n#   - conda-forge\n#   - defaults\n# dependencies:\n#   - python=3.11  # Note: Python 3.12+ might have numpy compatibility issues\n#   - pip\n#   - pip:\n#     - azureml-core==1.58.0\n#     - azureml-dataset-runtime # Pulled in by azureml-core or other components\n#     - numpy==1.23.5 # Pinning numpy is often necessary to avoid conflicts\n#     - pandas\n#     - scikit-learn","lang":"yaml","description":"Given that `azureml-dataset-runtime` is an internal package not meant for direct interaction, there is no typical 'quickstart' code for its use. Instead, it is installed as a dependency by other Azure Machine Learning SDK packages. The example demonstrates how it might appear in a `conda` environment configuration, highlighting the common practice of pinning `numpy` versions to avoid dependency conflicts with `azureml-dataset-runtime` and other SDK components."},"warnings":[{"fix":"Transition to Azure Machine Learning SDK v2 and its corresponding data handling mechanisms. Refer to official Microsoft documentation for migration guides.","message":"Azure Machine Learning SDK v1, which includes `azureml-dataset-runtime`, is deprecated as of March 31, 2025, with support ending on June 30, 2026. Users are strongly advised to migrate to Azure Machine Learning SDK v2 to avoid potential security risks or breaking changes after this date.","severity":"breaking","affected_versions":"<=1.62.0"},{"fix":"Do not directly import or rely on `azureml-dataset-runtime`. Instead, use the public-facing APIs of the main Azure Machine Learning SDK (`azureml-core`, `azure.ai.ml` for v2) which depend on this package internally.","message":"This package is explicitly internal and not intended for direct public use. Attempting to import or use components directly from `azureml-dataset-runtime` is unsupported and highly discouraged, as its internal APIs are subject to change without notice.","severity":"gotcha","affected_versions":"All versions"},{"fix":"When creating environments for AzureML SDK v1, carefully manage `numpy` versions. It's often necessary to explicitly pin `numpy` to a compatible version (e.g., `numpy==1.23.5`) in your `requirements.txt` or `conda` environment file. Check the specific dependency constraints for your `azureml-core` version.","message":"The `azureml-dataset-runtime` package, and by extension other `azureml` SDK components, often have strict compatibility requirements for `numpy` versions (e.g., `!=1.19.3,<1.24`). This can lead to dependency resolution issues with newer Python versions (like 3.12) or other data science libraries that require a different `numpy` version.","severity":"gotcha","affected_versions":"All versions"},{"fix":"When setting up your environment for AzureML SDK v1, prioritize Python versions known to be compatible, typically Python 3.8 to 3.11. Always test your environment rigorously for dependency conflicts.","message":"Compatibility with newer Python versions (e.g., 3.12+) can be challenging due to upstream dependencies like `numpy` required by `azureml-dataset-runtime`. While some AzureML SDK v1 components support Python 3.8+, the practical upper limit for fully compatible environments often lies around Python 3.11 due to these constraints.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'1.62.0':45 '2026':49 'activ':53 'among':27 'azur':1,17,64 'azureml':7,29,62 'azureml-dataset-runtim':6 'broader':61 'cadenc':55 'coordin':25 'dataset':4,8 'depend':26,70 'design':23 'develop':58 'direct':36 'ecosystem':21 'end':40 'end-us':39 'februari':48 'intend':34 'intern':12,69 'learn':3,19,67 'machin':2,18,66 'machine-learn':65 'maintain':51 'microsoft':68 'ongo':57 'packag':14,30 'primarili':22 'python':13 'reflect':56 'releas':46,54 'runtim':5,9 'sdk':20,63 'use':37 'user':41 'version':44 'within':15","created_at":"2026-04-14T18:31:36.304718+00:00","updated_at":"2026-04-15T23:34:37.277560+00:00","problems":[{"fix":"Ensure you are using a compatible Python version (e.g., Python 3.8-3.11). Pin `numpy` to a specific version, such as `numpy==1.23.5`, within your environment. On Apple Silicon Macs, consider using a Rosetta-emulated terminal or a Miniforge environment with x86 architecture. It's often best to install `azureml-sdk` which will manage the `azureml-dataset-runtime` dependency.","cause":"This error frequently occurs when there are underlying dependency conflicts, particularly with `numpy` or `azureml-dataprep` versions, or when the Python environment itself has issues. It's often seen on Apple Silicon Macs or with newer Python versions where direct installation of `azureml-dataset-runtime` (which is an internal package not meant for direct use) or its dependencies fails silently or leads to an inconsistent environment.","error":"Missing required package \"azureml-dataset-runtime\""},{"fix":"Upgrade `pip` and `setuptools` to their latest versions. Ensure your Python version is compatible with the `numpy` version required by `azureml-dataset-runtime` (often older `numpy` versions are required, like `<1.24`). On Apple Silicon Macs, running your terminal with Rosetta or setting up a `conda` environment with `Miniforge` to emulate an Intel environment can resolve compilation issues. Alternatively, explicitly install a compatible `numpy` version before installing other AzureML packages, e.g., `pip install numpy==1.23.5`.","cause":"This error indicates that the `numpy` package, a core dependency for `azureml-dataset-runtime` and related AzureML packages, cannot be built or installed successfully. This is commonly due to incompatible Python versions, outdated `pip` or `setuptools`, or specific architecture challenges (e.g., Apple Silicon Macs) where pre-built wheels for the required `numpy` version are not available.","error":"ERROR: Failed building wheel for numpy / ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects"},{"fix":"Install the full `azureml-sdk` in a clean Python environment, as it is designed to manage these complex inter-dependencies. If you need specific components, create a new virtual environment and install `azureml-sdk` first, then add other packages. If conflicts persist, try explicitly pinning versions of `azureml-dataprep` and `azureml-dataprep-native` that are known to be compatible with each other and `azureml-dataset-runtime`.","cause":"This is a direct dependency conflict reported by `pip` where different AzureML packages, particularly `azureml-dataprep` and `azureml-dataprep-native`, have incompatible version requirements, preventing `azureml-dataset-runtime` from being installed or updated. This often happens when users try to install individual AzureML components rather than the overarching `azureml-sdk`.","error":"ERROR: Cannot install azureml-dataset-runtime because these package versions have conflicting dependencies. The conflict is caused by: azureml-dataprep X.Y.Z depends on azureml-dataprep-native<A.B.C and >=D.E.F."}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":null,"cli_name":"","cli_version":null,"type":"library","homepage":"https://docs.microsoft.com/python/api/overview/azure/ml/?view=azure-ml-py","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/azureml-dataset-runtime/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["azure","ai-ml","data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":null,"verified_at":null,"last_verified":"2026-04-14","next_check":"2026-07-13","install_tag":null}}