{"id":7781,"library":"tensorrt-cu12-bindings","title":"TensorRT CUDA 12 Bindings","description":"TensorRT-cu12-bindings provides Python bindings for NVIDIA's TensorRT, a high-performance deep learning inference optimizer and runtime. This specific package targets CUDA 12.x environments. It is actively developed by NVIDIA, with frequent releases aligning with major TensorRT and CUDA versions, typically every few months.","status":"active","version":"10.16.1.11","language":"python","source_language":"en","source_url":"https://github.com/nvidia/tensorrt","tags":["deep learning","inference","gpu","nvidia","cuda","optimization","performance"],"install":[{"cmd":"pip install tensorrt-cu12-bindings","lang":"bash","label":"Install for CUDA 12.x"}],"dependencies":[{"reason":"Used for low-level CUDA API interactions in newer versions.","package":"cuda-python","optional":false},{"reason":"Standard for numerical operations and array handling.","package":"numpy","optional":false},{"reason":"Requires a compatible NVIDIA CUDA Toolkit (12.x) and driver installed on the system.","package":"NVIDIA CUDA Toolkit (system-level)","optional":false}],"imports":[{"wrong":"import tensorrt as trt","symbol":"tensorrt_bindings","correct":"import tensorrt_bindings as trt"}],"quickstart":{"code":"import tensorrt as trt\nimport numpy as np\n\n# 1. Create a logger (TRT_LOGGER = trt.Logger(trt.Logger.INFO) for more verbose output)\nTRT_LOGGER = trt.Logger(trt.Logger.WARNING)\n\n# 2. Create builder, network, and configuration\nbuilder = trt.Builder(TRT_LOGGER)\n# Explicit batch is required for some features (e.g., dynamic shapes)\nnetwork = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))\nconfig = builder.create_builder_config()\n\n# Configure builder options\n# max_workspace_size: The maximum GPU memory size (in bytes) that TensorRT can use for temporary buffers.\nconfig.max_workspace_size = 1 << 20  # 1 MiB (adjust as needed for larger models)\n\n# 3. Define the network: a simple identity layer for demonstration\n# Input shape (batch_size, channels, height, width)\ninput_shape = (1, 3, 224, 224)\ninput_tensor = network.add_input(name=\"input_tensor\", dtype=trt.float32, shape=input_shape)\n\n# Add an identity layer as a simple example operation\nidentity_layer = network.add_identity(input_tensor)\noutput_tensor = identity_layer.get_output(0)\n\n# Mark the output tensor\nnetwork.mark_output(output_tensor)\noutput_tensor.name = \"output_tensor\"\n\n# 4. Build the engine\nprint(f\"Building TensorRT engine with input shape {input_shape}...\")\nengine = builder.build_engine(network, config)\n\nif engine:\n    print(\"TensorRT engine built successfully!\")\n    # Example: serialize the engine to disk\n    # with open(\"my_identity_engine.trt\", \"wb\") as f:\n    #     f.write(engine.serialize())\n    # print(\"Engine serialized to my_identity_engine.trt\")\nelse:\n    print(\"Failed to build TensorRT engine.\")\n\n# Cleanup\ndel network, builder, config, engine\n","lang":"python","description":"This quickstart demonstrates how to initialize the TensorRT builder, define a simple identity network with an explicit batch dimension, configure the builder, and build a TensorRT engine. This is the fundamental process for optimizing and compiling deep learning models for NVIDIA GPUs."},"warnings":[{"fix":"Verify your NVIDIA CUDA Toolkit and driver versions precisely match the `cuXX` suffix of the installed package. Upgrade or downgrade system CUDA components as necessary.","message":"The `tensorrt-cuXX-bindings` packages are tightly coupled with specific CUDA versions (e.g., `cu12` for CUDA 12.x). Using a mismatched CUDA driver or toolkit version on your system will lead to runtime failures or import errors.","severity":"breaking","affected_versions":"All `tensorrt-cuXX-bindings` packages."},{"fix":"For full compatibility and access to samples/demos, ensure you are running a CUDA 12.x (or newer) environment and Python 3.10 or newer.","message":"TensorRT 10.13.2 dropped official support for CUDA 11.X. Additionally, official samples and demos now require Python 3.10 or newer.","severity":"breaking","affected_versions":"TensorRT 10.13.2 and later."},{"fix":"If you maintain custom plugins, review the TensorRT documentation for plugin migration guides and update them to `IPluginV3` or later to ensure forward compatibility.","message":"Custom TensorRT plugins (e.g., those implementing `IPluginV2`) are being migrated to `IPluginV3`. Older plugin versions may be deprecated and removed in future releases.","severity":"breaking","affected_versions":"TensorRT 10.11 and later."},{"fix":"Update any custom Python code that directly uses `pycuda` for CUDA context management or memory operations to use `cuda-python` for alignment with official practices and future compatibility.","message":"Official TensorRT Python samples and tools have transitioned from `pycuda` to `cuda-python` for low-level CUDA interactions.","severity":"deprecated","affected_versions":"TensorRT 10.14 and later."},{"fix":"Access all TensorRT samples, demos, and associated scripts directly from the official NVIDIA TensorRT GitHub repository (github.com/nvidia/tensorrt/tree/main/samples).","message":"Starting with TensorRT 10.14, samples and demos are no longer included directly within the `tensorrt-cuXX-bindings` Python packages.","severity":"gotcha","affected_versions":"TensorRT 10.14 and later."}],"env_vars":null,"search_vec":"'12':3,31 'activ':36 'align':43 'bind':4,8,11 'cu12':7 'cuda':2,30,48,59 'deep':20,54 'develop':37 'environ':33 'everi':51 'frequent':41 'gpu':57 'high':18 'high-perform':17 'infer':22,56 'learn':21,55 'major':45 'month':53 'nvidia':13,39,58 'optim':23,60 'packag':28 'perform':19,61 'provid':9 'python':10 'releas':42 'runtim':25 'specif':27 'target':29 'tensorrt':1,6,15,46 'tensorrt-cu12-bindings':5 'typic':50 'version':49 'x':32","created_at":"2026-04-16T14:13:07.558788+00:00","updated_at":"2026-04-16T14:13:07.558788+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\n  File \"/tmp/tmphcanu_er/venv/lib/python3.12/site-packages/tensorrt_bindings/__init__.py\", line 96, in <module>\n    from .tensorrt import *\nImportError: libnvonnxparser.so.11: cannot open shared object file: No such file or di"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"11.0.0.114","cli_name":"","cli_version":null,"type":"library","homepage":"https://developer.nvidia.com/tensorrt","github":"https://github.com/nvidia/tensorrt","docs":null,"changelog":null,"pypi":"https://pypi.org/project/tensorrt-cu12-bindings/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["ai-ml"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"import_fail","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-07-10","install_tag":null}}