{"id":7282,"library":"h3-pyspark","title":"PySpark Bindings for H3","description":"h3-pyspark provides PySpark bindings for Uber's H3 hierarchical hexagonal geospatial indexing system. It allows for efficient geospatial operations and analysis directly within Spark data pipelines by exposing H3 functions as Spark UDFs and native Spark functions. The library is currently at version 1.2.6 and receives active development and maintenance, with recent releases addressing bug fixes and edge cases.","status":"active","version":"1.2.6","language":"python","source_language":"en","source_url":"https://github.com/kevinschaich/h3-pyspark","tags":["pyspark","h3","geospatial","indexing","spark","gis"],"install":[{"cmd":"pip install h3-pyspark","lang":"bash","label":"PyPI"},{"cmd":"conda install -c conda-forge h3-pyspark","lang":"bash","label":"Conda"}],"dependencies":[{"reason":"Provides the Spark DataFrame API and execution environment for the H3 operations. This library is a binding to PySpark.","package":"pyspark","optional":false},{"reason":"The core Python binding for the H3 geospatial indexing system, which h3-pyspark wraps and extends.","package":"h3","optional":false}],"imports":[{"wrong":"import h3","symbol":"h3_pyspark","correct":"import h3_pyspark"}],"quickstart":{"code":"from pyspark.sql import SparkSession, functions as F\nimport h3_pyspark\nimport os\n\n# Initialize Spark Session (adjust master for your environment, e.g., 'local[*]'):\nspark = SparkSession.builder.master(os.environ.get('SPARK_MASTER', 'local[*]')).appName(\"H3PySparkQuickstart\").getOrCreate()\n\n# Create a DataFrame with latitude, longitude, and desired H3 resolution\ndata = [{\"lat\": 37.769377, \"lng\": -122.388903, 'resolution': 9}]\ndf = spark.createDataFrame(data)\n\n# Convert geographic coordinates to H3 index\ndf_with_h3 = df.withColumn('h3_index', h3_pyspark.geo_to_h3(F.col('lat'), F.col('lng'), F.col('resolution')))\n\ndf_with_h3.show()\n\n# Example of an extension function: index_shape for GeoJSON polygons\ngeojson_polygon = \"{\\\"type\\\":\\\"Polygon\\\",\\\"coordinates\\\":[[[-122.4,37.8],[-122.3,37.8],[-122.3,37.7],[-122.4,37.7],[-122.4,37.8]]]}\"\npolygon_df = spark.createDataFrame([{'id': 1, 'geometry': geojson_polygon, 'resolution': 9}])\n\npolygon_h3_df = polygon_df.withColumn(\n    'h3_cells',\n    h3_pyspark.index_shape(F.col('geometry'), F.col('resolution'))\n)\n\npolygon_h3_df.show(truncate=False)\n\nspark.stop()","lang":"python","description":"This quickstart demonstrates how to initialize a SparkSession, create a DataFrame with geospatial coordinates, and use `h3_pyspark.geo_to_h3` to convert latitude and longitude to an H3 index. It also includes an example of `h3_pyspark.index_shape` for indexing GeoJSON polygons. Ensure `pyspark` is configured correctly for your environment."},"warnings":[{"fix":"Refer to the `h3-py` migration guide for changes between H3 v3.x and v4.x. Adapt your code to the new function names and error handling. Verify the version of `h3-py` installed alongside `h3-pyspark` to ensure compatibility.","message":"The underlying `h3-py` library (which `h3-pyspark` wraps) introduced significant breaking changes in its 4.x versions, primarily around function naming conventions (e.g., `kRing` became `gridDisk`) and error handling.","severity":"breaking","affected_versions":"Users upgrading `h3-py` dependency to 4.x alongside h3-pyspark 1.x, or migrating code from `h3-py` 3.x to `h3-pyspark` 1.x with implicit `h3-py` 4.x."},{"fix":"Upgrade to `h3-pyspark` version 1.2.4 or newer to benefit from improved null value handling. Ensure your input data is clean, or explicitly handle nulls (e.g., `na.drop()`, `fillna()`) before passing to H3 functions. [cite: `1.2.4` release notes]","message":"Prior to version 1.2.4, `h3-pyspark` functions might not robustly handle null values in input columns to UDFs, potentially leading to errors or unexpected behavior.","severity":"gotcha","affected_versions":"< 1.2.4"},{"fix":"Upgrade to `h3-pyspark` version 1.2.3 or newer, which includes a fix for this bug and improved error handling for malformed geometries. [cite: `1.2.3` release notes]","message":"The `index_shape` function in versions prior to 1.2.3 had a known bug where it might miss H3 cells for long line segments, leading to incomplete or inaccurate spatial indexing for complex geometries.","severity":"gotcha","affected_versions":"< 1.2.3"},{"fix":"Ensure that your geometry data is formatted as GeoJSON strings before passing it to functions like `h3_pyspark.index_shape`. Convert from other formats if necessary.","message":"h3-pyspark assumes that geospatial geometries are represented as GeoJSON strings within a Spark DataFrame column, rather than other formats like WKT.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'1.2.6':50 'activ':53 'address':60 'allow':21 'analysi':27 'bind':2,10 'bug':61 'case':65 'current':47 'data':31 'develop':54 'direct':28 'edg':64 'effici':23 'expos':34 'fix':62 'function':36,43 'geospati':17,24,68 'gis':71 'h3':4,6,14,35,67 'h3-pyspark':5 'hexagon':16 'hierarch':15 'index':18,69 'librari':45 'mainten':56 'nativ':41 'oper':25 'pipelin':32 'provid':8 'pyspark':1,7,9,66 'receiv':52 'recent':58 'releas':59 'spark':30,38,42,70 'system':19 'uber':12 'udf':39 'version':49 'within':29","created_at":"2026-04-16T13:51:34.104194+00:00","updated_at":"2026-04-16T13:51:34.104194+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\n  File \"/tmp/tmp55x3t6vi/venv/lib/python3.12/site-packages/h3_pyspark/__init__.py\", line 1, in <module>\n    import h3\nModuleNotFoundError: No module named 'h3'"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"1.2.6","cli_name":"","cli_version":null,"type":"library","homepage":"https://h3geo.org","github":"https://github.com/kevinschaich/h3-pyspark","docs":null,"changelog":null,"pypi":"https://pypi.org/project/h3-pyspark/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","database"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"import_fail","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-07-10","install_tag":null}}