{"id":3514,"library":"ibis-framework","title":"Ibis: The Portable Python Dataframe Library","description":"Ibis is a portable Python dataframe library that provides a Pythonic way to build and execute operations on data in various backends, including SQL databases, data warehouses, and data lakes. It offers a familiar dataframe API that compiles into the backend's native language, enabling local iteration and remote deployment by changing a single line of code. It is currently at version 12.0.0 and maintains an active release cadence with frequent updates.","status":"active","version":"12.0.0","language":"python","source_language":"en","source_url":"https://github.com/ibis-project/ibis","tags":["dataframe","sql","data-warehousing","etl","duckdb","polars","bigquery"],"install":[{"cmd":"pip install 'ibis-framework[duckdb,examples]'","lang":"bash","label":"Recommended for local development with example data"},{"cmd":"pip install ibis-framework","lang":"bash","label":"Core library only"},{"cmd":"pip install 'ibis-framework[all]'","lang":"bash","label":"All backends (large install)"}],"dependencies":[{"reason":"Default and recommended local backend for high performance.","package":"duckdb","optional":true},{"reason":"Commonly used for converting Ibis expressions to in-memory DataFrames with `.to_pandas()`.","package":"pandas","optional":true},{"reason":"Used for efficient data handling and interoperability between Ibis and other data libraries.","package":"pyarrow","optional":true},{"reason":"Backend for Apache Spark integration.","package":"pyspark","optional":true},{"reason":"Underpins many SQL database connections.","package":"sqlalchemy","optional":true}],"imports":[{"note":"Installing 'ibis' (a web templating framework) instead of 'ibis-framework' (the dataframe library) is a common mistake. Always install 'ibis-framework' from PyPI.","wrong":"import ibis","symbol":"ibis","correct":"import ibis"},{"note":"Enables eager evaluation for interactive exploration, though it can be slower for production workloads and is not strictly necessary.","symbol":"ibis.options.interactive","correct":"ibis.options.interactive = True"}],"quickstart":{"code":"import ibis\n\n# Connect to an in-memory DuckDB database (default backend)\ncon = ibis.duckdb.connect(':memory:')\n\n# Load example data (e.g., the 'penguins' dataset)\nt = ibis.examples.penguins.fetch()\n\n# Create a table in the connected database\ncon.create_table('penguins', t.to_pyarrow(), overwrite=True)\n\n# Get a table expression from the connection\ntable = con.table('penguins')\n\n# Perform a lazy computation: group by species and calculate mean bill length\nresult_expr = table.group_by('species').agg(avg_bill_length=table.bill_length_mm.mean())\n\n# Execute the expression and fetch results into a pandas DataFrame\ndf = result_expr.to_pandas()\n\nprint(df)\n","lang":"python","description":"This quickstart demonstrates how to connect to an in-memory DuckDB backend, load an example dataset, define a lazy dataframe expression to calculate the average bill length per penguin species, and then execute it to retrieve the results as a pandas DataFrame. It showcases Ibis's core lazy evaluation pattern."},"warnings":[{"fix":"Upgrade your Python environment to 3.10 or a later version.","message":"Python 3.9 is no longer supported. Users must upgrade to Python 3.10 or newer.","severity":"breaking","affected_versions":">=12.0.0"},{"fix":"If using the PySpark backend, ensure PySpark is version 3.5 or newer.","message":"PySpark versions older than 3.5 are no longer supported when using the PySpark backend.","severity":"breaking","affected_versions":">=12.0.0"},{"fix":"Replace `dt.name` with `dt.__class__.__name__` where `dt` is an Ibis DataType object.","message":"The `DataType.name` attribute has been removed. Use `DataType.__class__.__name__` instead to get the string name of a data type.","severity":"breaking","affected_versions":">=11.0.0"},{"fix":"Instead of explicitly naming ephemeral in-memory tables, use `connection.create_table()` or `connection.create_view()` for persistent or named views.","message":"Explicit naming of `memtable`s is no longer supported. Use `create_table` or `create_view` for named objects.","severity":"breaking","affected_versions":">=11.0.0"},{"fix":"Always append `.execute()` or `.to_pandas()` (or similar materialization methods) to your Ibis expression chain to trigger computation and retrieve results.","message":"Ibis uses lazy evaluation. Operations define an expression graph but do not execute immediately. To get results, you must explicitly call methods like `.execute()` or `.to_pandas()`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use `pip install ibis-framework` (with relevant extras) or `conda install ibis-framework` to install the correct library.","message":"There is a different PyPI package also named `ibis` which is a web templating framework. Ensure you install `ibis-framework` for the dataframe library.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'12.0.0':69 'activ':73 'api':42 'backend':28,47 'bigqueri':87 'build':20 'cadenc':75 'chang':58 'code':63 'compil':44 'current':66 'data':25,32,35,82 'data-wareh':81 'databas':31 'datafram':5,12,41,79 'deploy':56 'duckdb':85 'enabl':51 'etl':84 'execut':22 'familiar':40 'frequent':77 'ibi':1,7 'includ':29 'iter':53 'lake':36 'languag':50 'librari':6,13 'line':61 'local':52 'maintain':71 'nativ':49 'offer':38 'oper':23 'polar':86 'portabl':3,10 'provid':15 'python':4,11,17 'releas':74 'remot':55 'singl':60 'sql':30,80 'updat':78 'various':27 'version':68 'wareh':83 'warehous':33 'way':18","created_at":"2026-04-11T17:32:52.613827+00:00","updated_at":"2026-04-16T15:39:55.993403+00:00","problems":[{"fix":"First, uninstall any incorrect `ibis` installations: `pip uninstall ibis`. Then, install the correct Ibis data framework: `pip install ibis-framework` or `conda install -c conda-forge ibis-framework`.","cause":"This error often occurs because users mistakenly install the `ibis` package (a web templating framework) instead of `ibis-framework` (the data analytics library), or both packages are installed, leading to a conflict as they share the `ibis` module name.","error":"ModuleNotFoundError: No module named 'ibis'"},{"fix":"Install the `ibis-framework` with the specific backend extra you intend to use. For Impala, this would be: `pip install 'ibis-framework[impala]'` or `conda install -c conda-forge ibis-framework-impala`. Replace `impala` with the desired backend (e.g., `duckdb`, `postgres`).","cause":"This error indicates that a specific backend (like Impala, or `connect` methods for other backends) was not installed along with the core `ibis-framework` package. Ibis backends need to be installed as 'extras' or separately.","error":"AttributeError: module 'ibis' has no attribute 'impala'"},{"fix":"Ensure the necessary backend for reading parquet files (commonly DuckDB) is installed as an extra: `pip install 'ibis-framework[duckdb]'` or `conda install -c conda-forge ibis-framework-duckdb`. You might also need to explicitly connect to the backend: `import ibis; con = ibis.duckdb.connect()`.","cause":"This error typically arises when attempting to perform I/O operations like `read_parquet` without an appropriate backend (e.g., DuckDB) being installed or explicitly configured as the default backend for Ibis to use for such operations.","error":"AttributeError: 'NoneType' object has no attribute 'read_parquet'"},{"fix":"Verify that the table name and its schema/database context are correct and that the table actually exists in the backend you are connected to. Ensure correct casing and full path if required by the database. If inserting, the table must exist prior to the `insert` call unless the backend supports schema creation on the fly.","cause":"This IbisError occurs when a table referenced in an Ibis expression or operation (e.g., `c.insert('t', df, database='db.mydbo')`) does not exist in the connected database or the fully qualified name is incorrect.","error":"ibis.common.exceptions.IbisError: Table not found: <table_name>"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"12.0.0","cli_name":"ibis","cli_version":"sh: 1: ibis: not found","type":"library","homepage":"https://ibis-project.org","github":"https://github.com/ibis-project/ibis","docs":"https://ibis-project.org","changelog":"https://ibis-project.org/release_notes","pypi":"https://pypi.org/project/ibis-framework/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","database"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-29","next_check":"2026-07-28","install_tag":null}}