{"id":667,"library":"nvidia-nccl-cu12","title":"NVIDIA Collective Communication Library (NCCL) Runtime for CUDA 12","description":"nvidia-nccl-cu12 (version 2.29.7) is the Python package providing the NVIDIA Collective Communication Library (NCCL) runtime specifically built for CUDA 12.x. NCCL is a foundational library for high-performance inter-GPU and inter-node communication primitives, such as all-reduce, all-gather, broadcast, and point-to-point operations, crucial for accelerating distributed deep learning workloads. It features a rapid release cadence, often synchronized with CUDA toolkit and major deep learning framework updates.","status":"active","version":"2.29.7","language":"python","source_language":"en","source_url":"https://github.com/NVIDIA/nccl","tags":["GPU","CUDA","Deep Learning","Distributed Training","Collective Communication","NVIDIA"],"install":[{"cmd":"pip install nvidia-nccl-cu12","lang":"bash","label":"Default Install"},{"cmd":"pip install \"nccl4py[cu12]\" # Official Python bindings","lang":"bash","label":"With NCCL4Py Bindings"}],"dependencies":[{"reason":"Often used implicitly or explicitly for CUDA Python bindings, especially with nccl4py.","package":"cuda-python","optional":false},{"reason":"Commonly used as a backend for PyTorch's distributed training module (torch.distributed).","package":"torch","optional":true},{"reason":"Commonly used as a backend for TensorFlow's distributed strategies (tf.distribute).","package":"tensorflow","optional":true}],"imports":[{"wrong":"from nvidia.ncccl import NcclCommunicator","symbol":"NcclCommunicator","correct":"from nvidia.ncccl import NcclCommunicator"}],"quickstart":{"code":"import os\nimport torch\nimport torch.distributed as dist\n\n# This quickstart assumes a multi-process setup, typically launched\n# via torch.distributed.launch or mpirun, where each process\n# runs this script with a unique rank and world_size.\n\n# Example environment variables (set by launch utility):\n# os.environ['MASTER_ADDR'] = os.environ.get('MASTER_ADDR', 'localhost')\n# os.environ['MASTER_PORT'] = os.environ.get('MASTER_PORT', '29500')\n# os.environ['RANK'] = os.environ.get('RANK', '0')\n# os.environ['WORLD_SIZE'] = os.environ.get('WORLD_SIZE', '1')\n\ndef run_distributed_example(rank, world_size):\n    # Initialize the process group with NCCL backend\n    print(f\"Initializing process group for rank {rank}/{world_size-1}...\")\n    dist.init_process_group(backend='nccl', rank=rank, world_size=world_size)\n    print(f\"Process group initialized on rank {rank}.\")\n\n    # Set device for the current process\n    torch.cuda.set_device(rank)\n\n    # Create a tensor on the GPU\n    tensor = torch.ones(10, device=f'cuda:{rank}') * (rank + 1)\n    print(f\"Rank {rank}: Initial tensor value: {tensor}\")\n\n    # Perform an all_reduce operation (summing tensors across all GPUs)\n    dist.all_reduce(tensor, op=dist.ReduceOp.SUM)\n\n    print(f\"Rank {rank}: Tensor after all_reduce: {tensor}\")\n\n    # Clean up the process group\n    dist.destroy_process_group()\n    print(f\"Rank {rank}: Process group destroyed.\")\n\n# To run this, you would typically use:\n# python -m torch.distributed.launch --nproc_per_node=2 your_script.py\n# Or set environment variables and run:\n# MASTER_ADDR=localhost MASTER_PORT=29500 RANK=0 WORLD_SIZE=2 python your_script.py\n# MASTER_ADDR=localhost MASTER_PORT=29500 RANK=1 WORLD_SIZE=2 python your_script.py\n\n# For simplicity, if running as a single process for structural check:\nif __name__ == '__main__':\n    # In a real scenario, rank and world_size would be provided by a launcher.\n    # This block is for structural demonstration only and will not perform\n    # actual distributed communication without a proper launcher.\n    try:\n        rank = int(os.environ.get('RANK', '0'))\n        world_size = int(os.environ.get('WORLD_SIZE', '1'))\n        if torch.cuda.is_available() and world_size > 0:\n             run_distributed_example(rank, world_size)\n        else:\n             print(\"CUDA not available or world_size is 0. Cannot run distributed example.\")\n    except RuntimeError as e:\n        print(f\"Error initializing distributed environment: {e}. This often happens if not run with a proper distributed launcher like torch.distributed.launch.\")\n","lang":"python","description":"This quickstart demonstrates how NCCL is typically used indirectly via PyTorch's `torch.distributed` module for multi-GPU collective communication, specifically an `all_reduce` operation. NCCL provides the underlying high-performance backend. A proper distributed launcher (e.g., `torch.distributed.launch` or `mpirun`) is required to run this code across multiple processes/GPUs. For direct Python bindings, consider `nccl4py` for explicit NCCL API calls."},"warnings":[{"fix":"Ensure that the `nvidia-nccl-cu12` package, your system's CUDA Toolkit, and the CUDA version used by your deep learning framework are all compatible. Consult the NVIDIA documentation or framework-specific guides for compatibility matrices. For PyTorch, `torch.cuda.is_available()` and `torch.version.cuda` can help verify. For `nccl4py`, use `pip install \"nccl4py[cu12]\"` to ensure correct CUDA 12 support.","message":"NCCL versions are tightly coupled with CUDA Toolkit versions and the CUDA version used to compile deep learning frameworks (like PyTorch or TensorFlow). Mismatches can lead to runtime errors, silent performance degradation, or unexpected behavior.","severity":"breaking","affected_versions":"All versions"},{"fix":"To use NCCL directly from Python, install and import `nccl4py`. If using with a deep learning framework, configure its distributed module to use the NCCL backend. Avoid `import nccl` for direct API calls, as this package is a runtime provider.","message":"The `nvidia-nccl-cu12` package itself primarily provides the `libnccl.so` shared library. Direct Python API calls are not exposed through this package. Instead, Python users interact with NCCL through higher-level libraries like `nccl4py` (official bindings) or as a backend to distributed training modules in frameworks like PyTorch (`torch.distributed`) or TensorFlow (`tf.distribute`).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Prefer using `nvidia-nccl-cu12` installed via pip for consistency within Python environments. If system-wide NCCL is necessary, carefully manage `LD_LIBRARY_PATH` to ensure the correct `libnccl.so` is prioritized. Frameworks like PyTorch often statically link NCCL, mitigating some of these issues, but custom builds might need `USE_SYSTEM_NCCL` flags.","message":"Conflicts can arise if multiple NCCL installations are present on the system (e.g., `nvidia-nccl-cu12` from PyPI, a system-wide `apt`/`dnf` installed NCCL, or one bundled with a deep learning framework). The linker's search path (`LD_LIBRARY_PATH`) can affect which `libnccl.so` is loaded, potentially leading to incorrect versions being used.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Verify the availability of `nccl4py[cu12]` for your specific Python version and OS on PyPI or the official `nccl4py` documentation. If pre-built wheels are not available, you might need to compile `nccl4py` from source (which requires a CUDA Toolkit installation and potentially other build dependencies) or consider using a deep learning framework's distributed module, which often bundles NCCL or manages its own bindings.","message":"The `nccl4py[cu12]` package, while recommended for direct Python interaction with NCCL CUDA 12, may not always have pre-built wheels available for all Python versions, operating systems, or architectures on PyPI. This can lead to `ERROR: Could not find a version that satisfies the requirement` during installation.","severity":"breaking","affected_versions":"All versions of `nccl4py` with `[cu12]` extra"},{"fix":"To install `nvidia-nccl-cu12`, you must first install `nvidia-pyindex` to configure the NVIDIA Python Package Index, or specify the NVIDIA index URL directly. For example: `pip install nvidia-pyindex && pip install nvidia-nccl-cu12`, or `pip install --extra-index-url https://pypi.ngc.nvidia.com nvidia-nccl-cu12`.","message":"The `nvidia-nccl-cu12` package is not directly available on the default PyPI.org repository. It is hosted on the NVIDIA Python Package Index, and attempting to install it without configuring this index will result in a build error indicating it's a \"placeholder project\".","severity":"breaking","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'12':9,32 '2.29.7':15 'acceler':69 'all-gath':57 'all-reduc':54 'broadcast':60 'built':29 'cadenc':79 'collect':2,23,97 'communic':3,24,50,98 'crucial':67 'cu12':13 'cuda':8,31,83,92 'deep':71,87,93 'distribut':70,95 'featur':75 'foundat':37 'framework':89 'gather':59 'gpu':45,91 'high':41 'high-perform':40 'inter':44,48 'inter-gpu':43 'inter-nod':47 'learn':72,88,94 'librari':4,25,38 'major':86 'nccl':5,12,26,34 'node':49 'nvidia':1,11,22,99 'nvidia-nccl-cu12':10 'often':80 'oper':66 'packag':19 'perform':42 'point':63,65 'point-to-point':62 'primit':51 'provid':20 'python':18 'rapid':77 'reduc':56 'releas':78 'runtim':6,27 'specif':28 'synchron':81 'toolkit':84 'train':96 'updat':90 'version':14 'workload':73 'x':33","created_at":"2026-03-28T17:09:38.935742+00:00","updated_at":"2026-04-16T17:29:23.225276+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nModuleNotFoundError: No module named 'nvidia.ncccl'"},"ecosystem":"pypi","meta_description":null,"install_score":23,"quickstart_score":0,"quickstart_tag":"stale","pypi_latest":"2.30.7","cli_name":"","cli_version":null,"type":"library","homepage":"https://developer.nvidia.com/cuda-zone","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/nvidia-nccl-cu12/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["ai-ml","aws"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"import_fail","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-07-10","install_tag":"stale"}}