{"id":5259,"library":"inference-cli","title":"Roboflow Inference CLI","description":"Roboflow Inference CLI is a command-line interface designed for deploying computer vision models to various devices and environments with minimal machine learning or deployment knowledge. It provides tools to run and manage a local inference server, process data with workflows, benchmark performance, make predictions, and deploy to the cloud. The library is currently at version 1.2.2 and sees active development with frequent releases.","status":"active","version":"1.2.2","language":"python","source_language":"en","source_url":"https://github.com/roboflow/inference","tags":["computer vision","machine learning","cli","inference","roboflow","object detection","segmentation","classification","deep learning"],"install":[{"cmd":"pip install inference-cli","lang":"bash","label":"Basic Installation"},{"cmd":"pip install inference-cli\n# For GPU (CUDA 12.1 example, adjust --extra-index-url for your CUDA version)\npip install torch torchvision --index-url https://download.pytorch.org/whl/cu121\npip install inference-gpu","lang":"bash","label":"GPU Installation (with PyTorch and CUDA)"}],"dependencies":[{"reason":"Required to run the local inference server (`inference server start`) as it pulls and manages Docker images.","package":"Docker","optional":false},{"reason":"Required for GPU inference with the `inference-models` backend. Specific CUDA-compatible versions are necessary.","package":"torch","optional":true},{"reason":"Often installed alongside `torch` for computer vision tasks, particularly with GPU inference.","package":"torchvision","optional":true},{"reason":"A dependency for the `inference-models` backend when leveraging NVIDIA GPUs.","package":"pycuda","optional":true}],"imports":[{"note":"Used for programmatic interaction with an Inference Server (local or hosted) over HTTP.","symbol":"InferenceHTTPClient","correct":"from inference_sdk import InferenceHTTPClient"},{"note":"Used for Python-native, direct inference without Docker, especially for video streams.","symbol":"InferencePipeline","correct":"from inference import InferencePipeline"}],"quickstart":{"code":"import os\nfrom inference_sdk import InferenceHTTPClient\n\n# Ensure you have your Roboflow API key set as an environment variable or replace os.environ.get with your key.\n# You can find your API key on the Roboflow dashboard.\nROBOFLOW_API_KEY = os.environ.get('ROBOFLOW_API_KEY', '')\n\nif not ROBOFLOW_API_KEY:\n    print(\"Warning: ROBOFLOW_API_KEY environment variable not set. Inference may fail.\")\n    # For a quick demo without a real key, you might use a dummy value\n    # or skip this part if you are only running a local server without Roboflow API interaction.\n    # For proper usage, always use a real key.\n\n# 1. Start a local inference server (requires Docker to be running):\n#    Run in your terminal: inference server start\n#    This will typically start on http://localhost:9001\n\n# 2. Initialize the InferenceHTTPClient\nclient = InferenceHTTPClient(\n    api_url=\"http://localhost:9001\",  # Or \"https://serverless.roboflow.com\" for hosted API\n    api_key=ROBOFLOW_API_KEY,\n)\n\n# Example image URL for inference\nimage_url = \"https://media.roboflow.com/inference/soccer.jpg\"\n\n# Replace with your actual model_id (e.g., 'your-project-name/your-model-version')\n# You can find this on your Roboflow model's deploy tab.\nmodel_id = \"soccer-players-5fuqs/1\"\n\n# 3. Perform inference\ntry:\n    print(f\"Running inference on {image_url} with model {model_id}...\")\n    results = client.infer(image_url, model_id=model_id)\n    print(\"Inference successful!\")\n    # Print first few predictions for brevity\n    if results and 'predictions' in results and len(results['predictions']) > 0:\n        print(\"First 3 predictions:\")\n        for i, pred in enumerate(results['predictions'][:3]):\n            print(f\"  - Class: {pred.get('class')}, Confidence: {pred.get('confidence'):.2f}\")\n    else:\n        print(\"No predictions found or unexpected result format.\")\nexcept Exception as e:\n    print(f\"An error occurred during inference: {e}\")\n    print(\"Ensure the local inference server is running ('inference server start') and the model ID/API key are correct.\")","lang":"python","description":"This quickstart demonstrates how to perform inference programmatically using the `inference_sdk.InferenceHTTPClient`. It assumes a local inference server is running (started via `inference server start` in the terminal, which requires Docker) or uses the Roboflow hosted API. It takes an image URL and a model ID, then prints the inference results. Ensure your `ROBOFLOW_API_KEY` is set as an environment variable."},"warnings":[{"fix":"To continue using the old inference backend, set the environment variable `USE_INFERENCE_MODELS=False`. For GPU users, ensure `torch` and `torchvision` are installed *before* `inference-gpu` with versions compatible with your CUDA toolkit.","message":"Starting with v1.2.0, `inference-models` became the default inference engine. This change impacts performance, resource usage, and may require adjustments for GPU users. The old backend is available in opt-out mode.","severity":"breaking","affected_versions":">=1.2.0"},{"fix":"Upgrade your Python environment to Python 3.10 or newer (up to <3.13) as specified by the `requires_python` metadata.","message":"Python 3.9 support has been deprecated and is now effectively End-of-Life. Building projects with Python 3.9 and `inference-cli` may lead to build failures or unpatched security vulnerabilities.","severity":"deprecated","affected_versions":">=1.1.0"},{"fix":"Install Docker Desktop (or equivalent) for your operating system and ensure it's running before executing `inference server start`.","message":"Running the local inference server using `inference server start` requires Docker to be installed and running on your system. Without Docker, the server cannot be launched.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Refer to the Roboflow documentation or PyTorch installation guide to ensure you install the correct CUDA Toolkit, cuDNN, and then `torch`, `torchvision`, and `inference-gpu` packages that are compatible with each other and your hardware.","message":"Proper GPU setup for `inference-gpu` is complex, requiring specific NVIDIA CUDA Toolkit and cuDNN installations, and careful selection of `torch` and `torchvision` versions that match your CUDA installation. Incorrect versions can lead to runtime errors or CPU-only inference.","severity":"gotcha","affected_versions":"All versions with GPU usage"},{"fix":"Ensure your `ROBOFLOW_API_KEY` is set as an environment variable or passed directly to the client constructor. Obtain your API key from the Roboflow dashboard.","message":"When performing programmatic inference with `inference_sdk.InferenceHTTPClient` or other SDK components, an `ROBOFLOW_API_KEY` (or `API_KEY`) is typically required for authentication, especially when interacting with Roboflow's hosted services.","severity":"gotcha","affected_versions":"All versions with programmatic API usage"}],"env_vars":null,"search_vec":"'1.2.2':61 'activ':64 'benchmark':46 'classif':79 'cli':3,6,73 'cloud':54 'command':10 'command-lin':9 'comput':16,69 'current':58 'data':43 'deep':80 'deploy':15,29,51 'design':13 'detect':77 'develop':65 'devic':21 'environ':23 'frequent':67 'infer':2,5,40,74 'interfac':12 'knowledg':30 'learn':27,72,81 'librari':56 'line':11 'local':39 'machin':26,71 'make':48 'manag':37 'minim':25 'model':18 'object':76 'perform':47 'predict':49 'process':42 'provid':32 'releas':68 'roboflow':1,4,75 'run':35 'see':63 'segment':78 'server':41 'tool':33 'various':20 'version':60 'vision':17,70 'workflow':45","created_at":"2026-04-14T01:26:05.122583+00:00","updated_at":"2026-04-17T15:06:17.622591+00:00","problems":[{"fix":"Ensure `inference-cli` is installed via `pip install inference-cli` (and also `inference` and `inference-sdk` for full functionality). If it is, activate your virtual environment if using one, or add the Python site-packages script directory to your system's PATH. For example, you can often run `python -m inference --help` as an alternative to `inference --help` if `python -m` is in your PATH.","cause":"The 'inference' command-line tool is not found in your system's PATH environment variable, meaning it's either not installed correctly or the installation directory is not accessible by your shell.","error":"command not found: inference"},{"fix":"Install the necessary packages using pip: `pip install inference inference_cli inference_sdk`. If you are in a virtual environment, ensure it is activated before running the installation.","cause":"This error typically occurs when a Python script tries to `import inference`, but the `inference` Python package (or its required components) is not installed in the active Python environment.","error":"ModuleNotFoundError: No module named 'inference'"},{"fix":"Verify the `model_id` or path you are providing is correct, ensure all necessary model files are present and uncorrupted, and check if any specific dependencies for that model type are missing. Consult the `inference-cli` documentation for supported model formats and additional setup steps for specific models.","cause":"The inference CLI encountered an issue while trying to load the specified model, which could be due to an incorrect model ID, corrupted model files, an unsupported model format, or missing dependencies required by the model.","error":"Error loading model: '{model_name}': {e}"},{"fix":"Set the required API key as an environment variable before running the command, for example: `export ROBOUFLOW_API_KEY='your_api_key_here'` (replace with the correct key and variable name for your specific service). Alternatively, configure it through the CLI's configuration command if available.","cause":"The `inference-cli` or an underlying model requires an API key for authentication, but the necessary environment variable (e.g., `ROBOUFLOW_API_KEY`, `OPENAI_API_KEY`) or configuration setting has not been provided or is incorrect.","error":"API key not set"},{"fix":"Ensure `inference-cli` is installed (`pip install inference-cli`) and that your Python environment's script directory is included in your system's PATH.","cause":"The `inference-cli` executable is not in your system's PATH, or the installation was incomplete.","error":"inference: command not found"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"1.5.1","cli_name":"inference","cli_version":"Traceback (most recent call last):","type":"library","homepage":"https://roboflow.com","github":"https://github.com/roboflow/inference","docs":null,"changelog":null,"pypi":"https://pypi.org/project/inference-cli/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["ai-ml","devops"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-30","next_check":"2026-07-28","install_tag":null}}