{"id":1331,"library":"apache-airflow-providers-sqlite","title":"Apache Airflow SQLite Provider","description":"The `apache-airflow-providers-sqlite` package provides the necessary components to interact with SQLite databases within Apache Airflow DAGs. It includes the `SqliteHook` for programmatic access and `SqliteOperator` for defining tasks. This provider is currently at version 4.3.1 and typically releases alongside major Apache Airflow provider updates.","status":"active","version":"4.3.1","language":"python","source_language":"en","source_url":"https://github.com/apache/airflow/tree/main/airflow/providers/sqlite","tags":["airflow","database","sqlite","provider"],"install":[{"cmd":"pip install apache-airflow-providers-sqlite","lang":"bash","label":"Install provider"}],"dependencies":[{"reason":"Required for provider functionality; this package extends Apache Airflow core.","package":"apache-airflow","optional":false}],"imports":[{"symbol":"SqliteHook","correct":"from airflow.providers.sqlite.hooks.sqlite import SqliteHook"},{"symbol":"SqliteOperator","correct":"from airflow.providers.sqlite.operators.sqlite import SqliteOperator"}],"quickstart":{"code":"from __future__ import annotations\nimport pendulum\nfrom airflow.models.dag import DAG\nfrom airflow.providers.sqlite.operators.sqlite import SqliteOperator\n\nwith DAG(\n    dag_id=\"sqlite_example_dag\",\n    start_date=pendulum.datetime(2023, 1, 1, tz=\"UTC\"),\n    schedule=None,\n    catchup=False,\n    tags=[\"sqlite\", \"example\"],\n) as dag:\n    create_table = SqliteOperator(\n        task_id=\"create_table\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"\"\"\n            CREATE TABLE IF NOT EXISTS test_table (\n                id INTEGER PRIMARY KEY,\n                name TEXT\n            );\n        \"\"\",\n    )\n\n    insert_data = SqliteOperator(\n        task_id=\"insert_data\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"INSERT INTO test_table (name) VALUES ('Airflow'), ('SQLite');\",\n    )\n\n    query_data = SqliteOperator(\n        task_id=\"query_data\",\n        sqlite_conn_id=\"sqlite_default\",\n        sql=\"SELECT * FROM test_table;\",\n        handler=lambda x: [print(row) for row in x] # Example handler to print results to logs\n    )\n\n    create_table >> insert_data >> query_data\n","lang":"python","description":"This quickstart demonstrates a basic Airflow DAG using the `SqliteOperator` to create a table, insert data, and query it. Ensure an Airflow connection named `sqlite_default` is configured, or it will default to an in-memory database. For a persistent database file, specify the path in the 'Host' field of the connection or in the 'Extra' field (e.g., `{\"database\":\"/path/to/my.db\"}`)."},"warnings":[{"fix":"Configure the `sqlite_default` Airflow connection to specify a file path for the 'Host' field or in the 'Extra' field as `{\"database\":\"/path/to/my.db\"}`.","message":"SQLite database persistence and location: By default, if the `sqlite_default` connection's 'Host' or 'Extra' field is not configured with a database path, the tasks will use an in-memory SQLite database (`:memory:`). This means all data will be lost after the task completes. For persistent storage, you *must* specify a file path (e.g., `/opt/airflow/dags/data/my_db.db`) in the Airflow connection.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Use SQLite only for single-node Airflow environments or for temporary, task-local data. For shared, persistent state in distributed setups, consider a client-server database like PostgreSQL or MySQL.","message":"Not suitable for distributed Airflow: SQLite is a file-based database. In distributed Airflow setups (e.g., using CeleryExecutor or KubernetesExecutor with multiple workers), the SQLite database file must reside on a shared, mounted filesystem accessible by all workers. Otherwise, each worker will operate on its own independent copy of the database, leading to data inconsistency. It's generally recommended only for single-node Airflow deployments or for ephemeral, task-local data.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your Airflow environment is version 2.0 or newer. Upgrade Airflow if necessary.","message":"Airflow 1.x incompatibility: This provider package (like all `apache-airflow-providers-*` packages) is designed exclusively for Apache Airflow 2.0 and later. Attempting to install or use it in an Airflow 1.x environment will result in import errors and general incompatibility.","severity":"breaking","affected_versions":"< 2.0.0 (Airflow)"},{"fix":"Implement custom sensing logic using a `PythonOperator` calling the `SqliteHook` or combine with other Airflow features like `ExternalTaskSensor` if monitoring external processes that modify the database.","message":"No SQLite Sensor: Unlike some other database providers (e.g., Postgres, MySQL), the `apache-airflow-providers-sqlite` package currently only offers `SqliteHook` and `SqliteOperator`. There is no dedicated `SqliteSensor` available for polling database states directly.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Use a supported Python version for Airflow 2.x (e.g., Python 3.8, 3.9, 3.10, 3.11). Refer to the official Apache Airflow documentation for the latest supported Python versions.","message":"Incompatible Python version: Apache Airflow 2.x and its provider packages have specific Python version requirements. Python 3.13 is currently not supported by Apache Airflow 2.x or its providers. Attempting to install or use the provider with an unsupported Python version will result in installation failures or `ModuleNotFoundError`.","severity":"breaking","affected_versions":"> 3.12 (Python)"}],"env_vars":null,"search_vec":"'4.3.1':43 'access':31 'airflow':2,8,23,50,53 'alongsid':47 'apach':1,7,22,49 'apache-airflow-providers-sqlit':6 'compon':15 'current':40 'dag':24 'databas':20,54 'defin':35 'includ':26 'interact':17 'major':48 'necessari':14 'packag':11 'programmat':30 'provid':4,9,12,38,51,56 'releas':46 'sqlite':3,10,19,55 'sqlitehook':28 'sqliteoper':33 'task':36 'typic':45 'updat':52 'version':42 'within':21","created_at":"2026-04-09T03:43:20.622810+00:00","updated_at":"2026-04-15T20:40:18.856471+00:00","problems":[{"fix":"Install the package using 'pip install apache-airflow-providers-sqlite'.","cause":"The 'apache-airflow-providers-sqlite' package is not installed.","error":"ModuleNotFoundError: No module named 'airflow.providers.sqlite'"},{"fix":"Ensure you are importing 'SqliteOperator' from 'airflow.providers.sqlite.operators.sqlite'.","cause":"The 'SqliteOperator' class is not available in the specified module.","error":"ImportError: cannot import name 'SqliteOperator' from 'airflow.providers.sqlite.operators.sqlite'"},{"fix":"Verify that 'SqliteHook' is correctly defined in 'airflow.providers.sqlite.hooks.sqlite'.","cause":"The 'SqliteHook' class is not found in the 'airflow.providers.sqlite.hooks.sqlite' module.","error":"AttributeError: module 'airflow.providers.sqlite.hooks.sqlite' has no attribute 'SqliteHook'"},{"fix":"Ensure the database file path specified in your Airflow connection (`sqlite_conn_id`) is absolute, the file exists, and the Airflow user has read/write permissions to both the file and its parent directory. For example, in Airflow Connections, set the Host field to an absolute path like `/path/to/your_database.db`.","cause":"Airflow's SQLite provider cannot access the specified database file, often due to incorrect file path, insufficient file permissions, or the database file not existing at the given location. This can also happen if the database path is relative when Airflow expects an absolute path.","error":"sqlite3.OperationalError: unable to open database file"},{"fix":"Update your `AIRFLOW__DATABASE__SQL_ALCHEMY_CONN` configuration or your custom SQLite connection (e.g., `sqlite_default`) to use an absolute path for the SQLite database file. For example, change `sqlite:///airflow.db` to `sqlite:////home/user/airflow/airflow.db`.","cause":"Airflow's SQLite configuration, particularly for the metadata database or a connection, was set with a relative path, but Airflow requires an absolute path for SQLite database files.","error":"airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///C:\\airflow/airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`"}],"ecosystem":"pypi","meta_description":null,"install_score":95,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"4.3.3","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/apache/airflow","docs":"https://airflow.apache.org/docs/apache-airflow-providers-sqlite/4.3.2","changelog":"https://airflow.apache.org/docs/apache-airflow-providers-sqlite/4.3.2/changelog.html","pypi":"https://pypi.org/project/apache-airflow-providers-sqlite/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["workflow","database","data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-06-28","next_check":"2026-07-28","install_tag":"verified"}}