{"id":1960,"library":"chdb","title":"chDB","description":"chDB is an in-process OLAP SQL Engine powered by ClickHouse, enabling users to embed a powerful analytical database directly within their Python applications. It allows running SQL queries on various data formats (Parquet, CSV, JSON, Pandas DataFrames) without needing a separate database server. Currently at version 4.1.6, chDB maintains an active development and release cadence, frequently adding features and improvements.","status":"active","version":"4.1.6","language":"python","source_language":"en","source_url":"https://github.com/chdb-io/chdb","tags":["database","olap","sql","clickhouse","in-process","data-analytics","embedded"],"install":[{"cmd":"pip install chdb","lang":"bash","label":"Install chDB"}],"dependencies":[{"reason":"chDB builds upon chdb-core, which provides the underlying ClickHouse engine. While `pip install chdb` handles this automatically, awareness can be useful for debugging or advanced scenarios.","package":"chdb-core","optional":false},{"reason":"Highly recommended for seamless integration with Pandas DataFrames, including direct querying and results output. Essential for the DataStore API.","package":"pandas","optional":true},{"reason":"Recommended for efficient data exchange and integration with Apache Arrow, especially when working with columnar data formats and DataFrame outputs.","package":"pyarrow","optional":true}],"imports":[{"symbol":"chdb","correct":"import chdb"},{"note":"Used for stateful sessions to maintain database state across queries.","symbol":"Session","correct":"from chdb import session as chs"},{"note":"For using chDB with the Python DB-API 2.0 interface.","symbol":"dbapi","correct":"import chdb.dbapi as dbapi"},{"note":"While `import chdb.datastore as pd` is often suggested for a Pandas-like API, directly importing `DataStore` or `chdb` for `chdb.query` is the primary usage for the core engine features. The `as pd` pattern aims for a drop-in replacement, which might mask `chdb`'s distinct behaviors.","wrong":"import chdb.datastore as pd","symbol":"DataStore","correct":"from chdb.datastore import DataStore"}],"quickstart":{"code":"import chdb\nimport pandas as pd\n\n# Run a simple SQL query and get results as a Pandas DataFrame\nresult_df = chdb.query(\"SELECT 1 as id, 'Hello chDB!' as message, version() as chdb_version\", \"DataFrame\")\nprint(\"Query Result (DataFrame):\\n\", result_df)\n\n# Query an existing Pandas DataFrame directly\ndata = {'col1': [1, 2, 3], 'col2': ['A', 'B', 'C']}\nmypandas_df = pd.DataFrame(data)\nsql_on_df = \"SELECT col1, upper(col2) FROM python(mypandas_df) WHERE col1 > 1\"\nqueried_df_from_pandas = chdb.query(sql_on_df, \"DataFrame\")\nprint(\"\\nQuery Result from Pandas DataFrame (DataFrame):\\n\", queried_df_from_pandas)","lang":"python","description":"This quickstart demonstrates how to execute a basic SQL query using `chdb.query` and receive the results directly as a Pandas DataFrame. It also shows how to query an existing Pandas DataFrame using ClickHouse SQL syntax via the `python(df_name)` table function."},"warnings":[{"fix":"Monitor memory usage for complex queries. For extremely large datasets, consider pre-processing or using a full ClickHouse server. Optimize SQL queries to reduce memory footprint where possible.","message":"chDB is an in-process engine and shares memory with your application. Running complex queries that process large datasets (e.g., aggregating 10GB on an 8GB RAM machine) can lead to out-of-memory crashes for the entire Python process, unlike server-side databases that can spill to disk.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Design your application with chDB as a single-user, embedded analytical tool. Implement access control and resource management at the application layer if necessary, or opt for a full ClickHouse server for multi-tenant scenarios.","message":"chDB operates in a single process and lacks built-in authentication, multi-tenancy, or fine-grained resource limits per user. This makes it unsuitable for multi-user applications or highly concurrent environments where resource isolation and access control are critical.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be mindful of chained DataFrame operations. For large datasets, consider explicitly performing operations that avoid intermediate materialization or breaking down complex chains into optimized SQL queries where possible.","message":"When using the DataStore (Pandas-compatible API) with chained operations in chDB v4.x, each intermediate step can materialize a new DataFrame in memory. This can lead to higher memory consumption than anticipated for large datasets, potentially negating some performance benefits.","severity":"gotcha","affected_versions":"4.0.0 and later"},{"fix":"Review your code for any direct references to `chdb-core` components. Ensure your environment correctly resolves dependencies after upgrading to 4.1.0 or later.","message":"In version 4.1.0, the `chdb` package was decoupled from `chdb-core`. While this was primarily an architectural change for packaging, users who had deep integrations or relied on specific internal structures related to `chdb-core` might experience breaking changes.","severity":"breaking","affected_versions":"Prior to 4.1.0"},{"fix":"Upgrade to chDB version 4.1.0 or newer to benefit from the fix for exit-related crashes.","message":"Versions prior to 4.1.0 were known to experience crashes when exiting the Python process, particularly with persistent sessions.","severity":"gotcha","affected_versions":"Prior to 4.1.0"},{"fix":"Ensure you are using chDB version 4.1.4 or newer to avoid potential module import issues after package upgrades.","message":"An issue in versions prior to 4.1.4 could lead to a broken module after upgrading, due to a missing `chdb/__init__.py` file.","severity":"gotcha","affected_versions":"Prior to 4.1.4"}],"env_vars":null,"search_vec":"'4.1.6':50 'activ':54 'ad':60 'allow':28 'analyt':20,73 'applic':26 'cadenc':58 'chdb':1,2,51 'clickhous':13,67 'csv':37 'current':47 'data':34,72 'data-analyt':71 'databas':21,45,64 'datafram':40 'develop':55 'direct':22 'emb':17 'embed':74 'enabl':14 'engin':10 'featur':61 'format':35 'frequent':59 'improv':63 'in-process':5,68 'json':38 'maintain':52 'need':42 'olap':8,65 'panda':39 'parquet':36 'power':11,19 'process':7,70 'python':25 'queri':31 'releas':57 'run':29 'separ':44 'server':46 'sql':9,30,66 'user':15 'various':33 'version':49 'within':23 'without':41","created_at":"2026-04-09T18:37:31.750221+00:00","updated_at":"2026-04-16T01:56:26.944742+00:00","problems":[{"fix":"Reduce the dataset size, process data in smaller chunks, or increase the available memory for the Python process. For datasets exceeding available RAM, consider using a ClickHouse server which can spill to disk.","cause":"chDB runs in-process and shares memory with the Python application. Queries that attempt to process or aggregate data larger than available RAM can cause the entire Python process to crash due to out-of-memory conditions or segmentation faults.","error":"Python process crashed (Out of Memory / Segmentation fault)"},{"fix":"Install the missing dependency using pip: `pip install pandas` or `pip install pyarrow`.","cause":"The chdb.query function, when used with output_format=\"DataFrame\" or output_format=\"ArrowTable\", requires the 'pandas' or 'pyarrow' library, respectively, to be installed.","error":"ImportError: If pyarrow or pandas aren't installed."},{"fix":"Review the SQL query for typos, ensure correct ClickHouse SQL syntax, use backticks (`) to escape reserved keywords if used as identifiers, or check the chDB/ClickHouse documentation for supported functions and versions.","cause":"The SQL query provided contains syntax errors, uses reserved keywords without proper escaping, or attempts to use a function that is not supported or enabled in the underlying ClickHouse engine version embedded in chDB.","error":"Code: X. DB::Exception: Syntax error: Syntax error near 'KEYWORD'"},{"fix":"Check if the database exists before attempting to create it (e.g., `CREATE DATABASE IF NOT EXISTS default;`), or use a temporary in-memory session if persistence is not required for that specific operation.","cause":"This error occurs when a chdb.Session attempts to create a database (e.g., `CREATE DATABASE default;`) that already exists, particularly when a session is re-initialized with persistent storage in the same location.","error":"Code: 82. DB::Exception: Database default already exists. (DATABASE_ALREADY_EXISTS)"},{"fix":"Enable the `engine_file_allow_create_multiple_files` setting before inserting to allow ClickHouse to create new files for each insert: `chdb.query(\"SET engine_file_allow_create_multiple_files = 1;\")` before your INSERT statements.","cause":"The Parquet file format, by default, does not support appending data directly. When using a File table engine with Parquet format in ClickHouse (and thus chDB), subsequent INSERT statements will fail if they try to append to the same file.","error":"DB::Exception: Cannot append data in format Parquet to file, because this format doesn't support appends. (CANNOT_APPEND_TO_FILE)"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"4.3.0","cli_name":"chdb","cli_version":"sh: 1: chdb: not found","type":"library","homepage":"https://clickhouse.com/chdb","github":"https://github.com/chdb-io/chdb","docs":"https://chdb.readthedocs.io/en/latest/index.html","changelog":"https://github.com/chdb-io/chdb/releases","pypi":"https://pypi.org/project/chdb/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["database","data","serialization"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-28","next_check":"2026-07-28","install_tag":null}}