{"id":10351,"library":"whylogs-sketching","title":"whylogs-sketching Library","description":"whylogs-sketching is a foundational Python library that provides efficient, probabilistic data structures for common data sketching tasks, such as approximate cardinality estimation (HyperLogLog), frequent item counting, and histogram generation. It serves as a core dependency for the `whylogs` data logging and profiling library, providing the underlying sketching primitives. While it can be used independently for low-level sketching operations, most users interact with its functionalities through the higher-level APIs of `whylogs`. The current PyPI version is 3.4.1.dev3, and its development generally aligns with the `whylogs` release cycle.","status":"active","version":"3.4.1.dev3","language":"python","source_language":"en","source_url":"https://github.com/whylabs/whylogs-sketching","tags":["data sketching","probabilistic data structures","hyperloglog","frequent items","histogram","whylogs"],"install":[{"cmd":"pip install whylogs-sketching","lang":"bash","label":"Direct installation"},{"cmd":"pip install whylogs","lang":"bash","label":"Installed as a dependency of whylogs"}],"dependencies":[{"reason":"Numerical operations for sketches","package":"numpy"},{"reason":"Underlying data sketching implementations (e.g., for Frequent Strings Sketch)","package":"datasketches"},{"reason":"Type hinting support for Python < 3.11","package":"typing-extensions","optional":true}],"imports":[{"note":"For HyperLogLog cardinality estimation","symbol":"HllSketch","correct":"from whylogs_sketching.hll_sketch import HllSketch"},{"note":"For approximate frequent item counting","symbol":"FrequentItemsSketch","correct":"from whylogs_sketching.frequent_items_sketch import FrequentItemsSketch"},{"note":"For approximate histogram generation","symbol":"Histogram","correct":"from whylogs_sketching.histogram import Histogram"},{"note":"Specific frequent strings sketch implementation from datasketches subpackage","symbol":"FrequentStringsSketch","correct":"from whylogs_sketching.datasketches.frequent_strings_sketch import FrequentStringsSketch"}],"quickstart":{"code":"from whylogs_sketching.hll_sketch import HllSketch\n\n# Create an HLL sketch for approximate unique count\nsketch = HllSketch()\n\n# Add various values to the sketch\nsketch.update(\"user_a\")\nsketch.update(\"user_b\")\nsketch.update(\"user_a\") # Adding a duplicate\nsketch.update(\"user_c\")\nsketch.update(123)\nsketch.update(456.78)\n\n# Get the approximate number of unique items\ncardinality_estimate = sketch.get_estimate()\nprint(f\"Approximate unique items: {cardinality_estimate}\")\n\n# You can also merge two sketches\nsketch2 = HllSketch()\nsketch2.update(\"user_c\") # Another duplicate\nsketch2.update(\"user_d\")\n\nsketch.merge(sketch2)\nmerged_cardinality = sketch.get_estimate()\nprint(f\"Approximate unique items after merge: {merged_cardinality}\")\n","lang":"python","description":"This quickstart demonstrates how to initialize an `HllSketch`, add various types of elements to it, and retrieve the approximate count of unique items. It also shows how to merge two sketches to combine their unique counts."},"warnings":[{"fix":"Consider if your use case truly requires direct interaction with sketching primitives or if `whylogs` can provide the necessary functionality more easily.","message":"whylogs-sketching is primarily an internal dependency of the `whylogs` library. While direct usage is possible for low-level operations, most users will interact with its functionalities through `whylogs`'s higher-level APIs, which might offer a more convenient and integrated experience.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Monitor the `whylogs` project for stable version announcements or refer to the `whylogs` `pyproject.toml` for the exact `whylogs-sketching` version it depends on.","message":"The latest PyPI version (e.g., 3.4.1.dev3) often reflects a development release. While functional, it might indicate that the library is still under active iteration or that stable releases are primarily tied to `whylogs`'s major versions.","severity":"gotcha","affected_versions":"3.x.x.devY releases"},{"fix":"Always consult the official `whylogs-sketching` (or `whylogs`) GitHub repository for specific API changes between major versions when upgrading. Re-test code against new versions.","message":"Major version changes (e.g., from 0.x to 3.x) have introduced significant API changes, particularly in class constructors, method signatures, and internal data representations.","severity":"breaking","affected_versions":"< 3.0.0 to >= 3.0.0"}],"env_vars":null,"search_vec":"'3.4.1':86 'align':92 'api':78 'approxim':26 'cardin':27 'common':20 'core':40 'count':32 'current':82 'cycl':97 'data':17,21,45,98,101 'depend':41 'dev3':87 'develop':90 'effici':15 'estim':28 'foundat':10 'frequent':30,104 'function':72 'general':91 'generat':35 'higher':76 'higher-level':75 'histogram':34,106 'hyperloglog':29,103 'independ':60 'interact':69 'item':31,105 'level':64,77 'librari':4,12,49 'log':46 'low':63 'low-level':62 'oper':66 'primit':54 'probabilist':16,100 'profil':48 'provid':14,50 'pypi':83 'python':11 'releas':96 'serv':37 'sketch':3,7,22,53,65,99 'structur':18,102 'task':23 'under':52 'use':59 'user':68 'version':84 'whylog':2,6,44,80,95,107 'whylogs-sketch':1,5","created_at":"2026-04-17T01:23:40.430950+00:00","updated_at":"2026-04-17T01:23:40.430950+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nModuleNotFoundError: No module named 'whylogs_sketching.hll_sketch'; 'whylogs_sketching' is not a package"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"3.4.1.dev3","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/whylabs/whylogs-sketching","docs":null,"changelog":null,"pypi":"https://pypi.org/project/whylogs-sketching/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","ai-ml","observability"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"skip","verified_at":"2026-06-28","last_verified":"2026-06-28","next_check":"2026-08-01","install_tag":null}}