{"id":1240,"library":"fastparquet","title":"fastparquet","description":"fastparquet is a Python library providing performant read/write support for the Parquet file format, without needing a Python-Java bridge. It integrates well with Python-based big data workflows, particularly Dask and Pandas (versions < 3.0). As of March 2026, with Pandas 3.0 explicitly depending on PyArrow, `fastparquet` is being retired, and no further development is anticipated, though it remains usable for Pandas 2.x users.","status":"deprecated","version":"2026.3.0","language":"python","source_language":"en","source_url":"https://github.com/dask/fastparquet/","tags":["parquet","data processing","dataframe","io","dask","pandas"],"install":[{"cmd":"pip install fastparquet","lang":"bash","label":"PyPI"},{"cmd":"conda install -c conda-forge fastparquet","lang":"bash","label":"Conda"}],"dependencies":[{"reason":"Required for numerical operations, recommend installing first for pip.","package":"numpy"},{"reason":"Core dependency for DataFrame integration, specifically versions <3.0.","package":"pandas","optional":true},{"reason":"Required if building from Pyx files or installing from source.","package":"cython"},{"reason":"Required for compression algorithms.","package":"cramjam"},{"reason":"Required for filesystem abstraction.","package":"fsspec"}],"imports":[{"note":"Used for reading existing Parquet files.","symbol":"ParquetFile","correct":"from fastparquet import ParquetFile"},{"note":"Used for writing Pandas DataFrames to Parquet files.","symbol":"write","correct":"from fastparquet import write"}],"quickstart":{"code":"import pandas as pd\nfrom fastparquet import write, ParquetFile\nimport os\n\n# Create a sample DataFrame\ndf = pd.DataFrame({\n    'col1':,\n    'col2': ['A', 'B', 'C', 'D'],\n    'col3': [True, False, True, False]\n})\n\nfilename = \"example.parquet\"\n\n# Write the DataFrame to a Parquet file with Snappy compression\nwrite(filename, df, compression='SNAPPY')\nprint(f\"DataFrame successfully written to '{filename}'.\")\n\n# Read the Parquet file back into a DataFrame\npf = ParquetFile(filename)\ndf_read = pf.to_pandas()\nprint(f\"DataFrame successfully read from '{filename}':\")\nprint(df_read)\n\n# Clean up the created file\nos.remove(filename)","lang":"python","description":"This quickstart demonstrates how to create a Pandas DataFrame, write it to a Parquet file using `fastparquet.write`, and then read the data back into a new DataFrame using `fastparquet.ParquetFile.to_pandas()`. It also includes basic file cleanup."},"warnings":[{"fix":"Downgrade Pandas to a version less than 3.0, or migrate your Parquet I/O operations to use PyArrow or other alternatives like Dask with the PyArrow engine.","message":"The `fastparquet` project is being retired and is incompatible with `pandas` 3.0 and newer versions. Pandas 3.0 now explicitly depends on `pyarrow`, superseding `fastparquet` for many common workflows. Users should target `pandas<3.0` for continued use or migrate to `pyarrow`.","severity":"breaking","affected_versions":"pandas >= 3.0"},{"fix":"Pre-process data to minimize NULLs or use appropriate sentinel values. Consider fixed-length string types if supported by your data and downstream systems.","message":"Performance can be significantly impacted by the presence of NULL values and variable-length string encoding in your data. For optimal performance, consider using sentinel values (e.g., NaN) for data types that support them, or fixed-length strings where compatible with your ecosystem.","severity":"gotcha","affected_versions":"All versions"},{"fix":"`pip install numpy` before `pip install fastparquet`. Ensure you have a C compiler (e.g., GCC on Linux, Xcode command-line tools on macOS, MSVC on Windows) and `pip install cython` if encountering compilation errors.","message":"When installing `fastparquet` via `pip`, it's advisable to install `numpy` first to aid the dependency resolver. If pre-compiled wheels are not available for your system/Python version, or when installing directly from the GitHub repository, a C compiler toolchain and `cython` are required for compilation.","severity":"gotcha","affected_versions":"All versions (especially when wheels are not available)"},{"fix":"Review and correct the `SyntaxError: invalid syntax` in the test script at `/script.py` line 7. Ensure the Python code adheres to valid syntax rules.","message":"The test script itself contains a `SyntaxError`, preventing the library from being properly evaluated. This issue is with the test script's Python syntax rather than a direct problem with the installed library or its dependencies.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'2':66 '2026':42 '3.0':38,45 'anticip':59 'base':29 'big':30 'bridg':22 'dask':34,74 'data':31,70 'datafram':72 'depend':47 'develop':57 'explicit':46 'fastparquet':1,2,50 'file':14 'format':15 'integr':24 'io':73 'java':21 'librari':6 'march':41 'need':17 'panda':36,44,65,75 'parquet':13,69 'particular':33 'perform':8 'process':71 'provid':7 'pyarrow':49 'python':5,20,28 'python-bas':27 'python-java':19 'read/write':9 'remain':62 'retir':53 'support':10 'though':60 'usabl':63 'user':68 'version':37 'well':25 'without':16 'workflow':32 'x':67","created_at":"2026-04-06T16:56:02.510172+00:00","updated_at":"2026-04-16T14:58:40.464740+00:00","problems":[{"fix":"Downgrade `fastparquet` to an older, compatible version like 0.7.2. Ensure `pyarrow` is also at a compatible version (e.g., `pyarrow==5.0.0` with `fastparquet==0.7.2`).\n\n`pip install fastparquet==0.7.2 pyarrow==5.0.0`","cause":"This error typically occurs due to version incompatibility between `fastparquet`, `pyarrow`, and `pandas`, especially when `fastparquet` version 0.8.0 or newer is used with `pyarrow` versions older than 5.0.0 and specific Python/Pandas versions (e.g., Python 3.6.9, Pandas 1.1.5).","error":"AttributeError: module 'fastparquet.parquet_thrift' has no attribute 'SchemaElement'"},{"fix":"Install `fastparquet` using pip or conda, making sure the installation targets the correct Python environment.\n\n`pip install fastparquet` (or `conda install -c conda-forge fastparquet` if using Anaconda)","cause":"This error means the `fastparquet` library is not installed in the Python environment where the code is being run, or the environment is not correctly activated.","error":"ModuleNotFoundError: No module named 'fastparquet'"},{"fix":"Install the missing compression library. For 'snappy', install `python-snappy`. For others, install the corresponding Python package (e.g., `lz4`, `zstandard`, `brotli`).\n\n`pip install python-snappy` (or `conda install -c conda-forge python-snappy`)","cause":"This error indicates that a specified compression library (e.g., `snappy`, `lz4`, `zstandard`, `brotli`) is not installed or properly configured in the environment, even though `fastparquet` supports it.","error":"RuntimeError: Compression 'snappy' not available. Options: ['GZIP', 'UNCOMPRESSED']"},{"fix":"Explicitly convert the problematic column(s) to a string type (e.g., `str`) or another simple, consistent type before writing to Parquet, or preprocess the data to ensure uniform, compatible types.\n\n`df['problematic_column'] = df['problematic_column'].astype(str)`","cause":"This error arises when `fastparquet` encounters a column in a Pandas DataFrame that contains complex or mixed data types (like lists, tuples, or objects that it cannot automatically convert to a Parquet-compatible type).","error":"ValueError: Can't infer object conversion type: 0 (6.0, 1.0, 1.0, 1.0, 1.0)"},{"fix":"Ensure both `pyarrow` and `fastparquet` are installed and accessible in your Python environment. While `pandas >= 3.0` explicitly depends on `pyarrow`, for `pandas < 3.0` either engine is sufficient. If `fastparquet` is intended, ensure it's installed.\n\n`pip install pyarrow fastparquet`","cause":"This error from `pandas.read_parquet` indicates that neither `pyarrow` nor `fastparquet` is found or correctly installed in the environment for Pandas to use as a Parquet engine.","error":"ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'. A suitable version of pyarrow or fastparquet is required for parquet support."}],"ecosystem":"pypi","meta_description":null,"install_score":100,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"2026.5.0","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/dask/fastparquet","docs":null,"changelog":null,"pypi":"https://pypi.org/project/fastparquet/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","serialization"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-27","next_check":"2026-07-28","install_tag":"verified"}}