{"id":3329,"library":"databricks-dlt","title":"Databricks Delta Live Tables (DLT) Python Stubs","description":"The `databricks-dlt` library provides Python stubs to facilitate local development of Databricks Delta Live Tables (DLT) pipelines. It offers API specifications, docstring references for IDE autocompletion, and Python data type hints for static type checking. This library is purely for development-time assistance and *does not contain functional implementations*; DLT pipelines must be executed on a Databricks workspace. The underlying DLT product has recently been updated to Lakeflow Spark Declarative Pipelines (SDP) by Databricks, with continuous updates on the platform.","status":"active","version":"0.3.0","language":"python","source_language":"en","source_url":"https://pypi.org/project/databricks-dlt/","tags":["databricks","dlt","delta live tables","etl","data engineering","stub","ide","pyspark"],"install":[{"cmd":"pip install databricks-dlt","lang":"bash","label":"Install the DLT stub library"}],"dependencies":[],"imports":[{"note":"While 'import dlt' is correct for existing DLT code, Databricks now recommends 'from pyspark import pipelines as dp' for new Lakeflow Spark Declarative Pipelines (SDP) development.","wrong":"from dlt import table, read_json","symbol":"dlt","correct":"import dlt"},{"note":"This is the recommended import path for new Lakeflow Spark Declarative Pipelines (SDP) development, replacing the 'dlt' module.","symbol":"pipelines (as dp)","correct":"from pyspark import pipelines as dp"}],"quickstart":{"code":"# This code snippet is designed to run within a Databricks DLT Notebook environment.\n# The 'databricks-dlt' library provides stubs for local development, \n# but actual execution requires a Databricks workspace.\n\nimport dlt\nfrom pyspark.sql.functions import *\n\n# Define a streaming table (Bronze layer)\n@dlt.table\ndef raw_data():\n    # In a real scenario, this would read from a source like Auto Loader\n    # e.g., spark.readStream.format('cloudFiles').option('cloudFiles.format', 'json').load('/databricks-datasets/retail-org/sales_orders/')\n    # For demonstration, we'll simulate a static DataFrame read as this is a stub example\n    return spark.read.format('json').load('/databricks-datasets/retail-org/sales_orders/')\n\n# Define a cleansed table (Silver layer) with expectations\n@dlt.table(comment='Cleansed sales orders with valid order numbers')\n@dlt.expect_or_drop('valid_order_number', 'order_number IS NOT NULL')\ndef cleansed_data():\n    return dlt.read('raw_data').select(col('customer_id'), col('order_number'), col('order_date'))\n\n# Define an aggregated table (Gold layer)\n@dlt.table(name='daily_sales_summary')\ndef daily_sales():\n    return (\n        dlt.read('cleansed_data')\n        .groupBy('order_date')\n        .agg(count('order_number').alias('total_orders'))\n    )\n","lang":"python","description":"This quickstart demonstrates a typical Delta Live Tables pipeline structure using Python decorators within a Databricks notebook. It outlines a medallion architecture (bronze, silver, gold) for data processing. Note that while the `databricks-dlt` library provides IDE support for such code, actual execution and data processing occur only when deployed and run as a DLT pipeline on a Databricks workspace."},"warnings":[{"fix":"Understand that `databricks-dlt` is a development-time aid. To run your pipelines, you need a Databricks workspace and the DLT runtime. Refer to Databricks documentation for deploying and running DLT pipelines.","message":"The `databricks-dlt` PyPI library provides Python *stubs* for local development tools (IDE autocompletion, type checking) and *does not contain functional implementations*. DLT pipelines defined using this stub must be deployed and executed on a Databricks workspace; they cannot be run locally.","severity":"gotcha","affected_versions":"All versions of `databricks-dlt`."},{"fix":"For new DLT/SDP development, use `from pyspark import pipelines as dp` at the top of your Python pipeline files. Replace `@dlt.table` with `@dp.table`, `@dlt.streaming_table` with `@dp.create_streaming_table`, `@dlt.materialized_view` with `@dp.materialized_view`, etc.","message":"The underlying product \"Delta Live Tables (DLT)\" has been rebranded to \"Lakeflow Spark Declarative Pipelines (SDP)\". While existing Python code using `import dlt` will continue to function, Databricks officially recommends migrating new development to use `from pyspark import pipelines as dp` and the corresponding `@dp` decorators and functions for future compatibility and to leverage new features.","severity":"deprecated","affected_versions":"From Databricks Runtime 15.4+ (January 2026 onwards)."},{"fix":"Ensure your Python code is part of a DLT/SDP pipeline definition within a Databricks workspace. Local development with the `databricks-dlt` stub is for syntax validation only, not execution.","message":"The `dlt` (or `pyspark.pipelines`) module and its decorators are only available when your Python code is executed within the context of a Databricks DLT/SDP pipeline. Attempting to import or use these modules in a standalone Python script or a regular Databricks notebook (not configured as a DLT pipeline) will result in an `ImportError` or `NameError`.","severity":"gotcha","affected_versions":"All versions related to Databricks DLT/SDP Python API."},{"fix":"Follow Databricks best practices for dependency management: package Python dependencies as a wheel or egg and specify them in the pipeline libraries settings, or use a `requirements.txt` file attached to the pipeline configuration. Avoid dynamic installation methods within the pipeline code.","message":"Managing external Python dependencies within DLT/SDP pipelines directly with `%pip install` or init scripts on the cluster can be problematic due to potential conflicts and maintenance issues. This can lead to unexpected pipeline failures or inconsistent environments.","severity":"gotcha","affected_versions":"All versions."}],"env_vars":null,"search_vec":"'api':29 'assist':53 'autocomplet':35 'check':44 'contain':57 'continu':86 'data':38,97 'databrick':1,10,21,67,84,91 'databricks-dlt':9 'declar':80 'delta':2,22,93 'develop':19,51 'development-tim':50 'dlt':5,11,25,60,71,92 'docstr':31 'engin':98 'etl':96 'execut':64 'facilit':17 'function':58 'hint':40 'ide':34,100 'implement':59 'lakeflow':78 'librari':12,46 'live':3,23,94 'local':18 'must':62 'offer':28 'pipelin':26,61,81 'platform':90 'product':72 'provid':13 'pure':48 'pyspark':101 'python':6,14,37 'recent':74 'refer':32 'sdp':82 'spark':79 'specif':30 'static':42 'stub':7,15,99 'tabl':4,24,95 'time':52 'type':39,43 'under':70 'updat':76,87 'workspac':68","created_at":"2026-04-11T12:35:54.190894+00:00","updated_at":"2026-04-16T05:16:59.209718+00:00","problems":[{"fix":"DLT pipelines must be created and run on a Databricks workspace as a DLT pipeline, not as a standard notebook on an interactive cluster or locally. For local development, `import dlt` will work with the stub library, but the code will not execute. To perform syntax checks locally, consider mocking the `dlt` object.","cause":"The `dlt` module is only available within a Delta Live Tables pipeline runtime on Databricks. The `databricks-dlt` PyPI package provides local development stubs for IDE assistance but does not contain a functional implementation of the `dlt` module for local execution or on standard Databricks clusters.","error":"ModuleNotFoundError: No module named 'dlt'"},{"fix":"Ensure that your DLT code is executed as part of a Delta Live Tables pipeline on a Databricks workspace. The `dlt` module is only fully functional in this dedicated environment.","cause":"This error occurs when attempting to import the `dlt` module on a Databricks cluster that is not specifically a DLT pipeline runtime, such as a Spark Connect cluster or a standard interactive cluster. The `dlt` module has specific runtime requirements within the DLT framework.","error":"DLTImportException: Delta Live Tables module is not supported on Spark Connect clusters."},{"fix":"DLT decorators and functions are only executed and interpreted within a Delta Live Tables pipeline on a Databricks workspace. When developing locally, the `databricks-dlt` stub library provides type hints and autocompletion. To avoid `AttributeError` during local testing or syntax checking, you might need to mock the `dlt` object and its methods.","cause":"This error occurs when trying to use the `@dlt.table` decorator (or other `dlt` functions like `dlt.read`) in an environment where the full DLT runtime is not present, such as a local Python environment with only the `databricks-dlt` stub library installed, or a standard Databricks notebook that isn't part of a DLT pipeline. The local `databricks-dlt` package provides the interface for `dlt.table` for autocompletion but no functional implementation.","error":"AttributeError: module 'dlt' has no attribute 'table'"},{"fix":"DLT pipelines should be executed via the DLT pipeline interface in the Databricks workspace. For interactive debugging and development of supporting Python code (non-DLT specific functions), consider placing them in separate modules that can be tested independently or using a mocked `dlt` environment.","cause":"Delta Live Tables notebooks are designed to be executed as part of a managed pipeline, not interactively cell-by-cell like standard Databricks notebooks. The DLT runtime manages the execution flow, and individual cells (especially those not directly defining tables or views) are not intended for arbitrary interactive execution.","error":"cannot run a cell when connected to pipeline"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.3.0","cli_name":"","cli_version":null,"type":"library","homepage":"https://www.databricks.com/product/delta-live-tables","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/databricks-dlt/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["type-stubs","data","devops"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-29","next_check":"2026-07-28","install_tag":null}}