{"id":2803,"library":"tensorflow-metadata","title":"TensorFlow Metadata","description":"TensorFlow Metadata (TFMD) provides standard representations for metadata that are useful when training machine learning models with TensorFlow. This includes formats for describing tabular data schemas (e.g., `tf.Examples`), collections of summary statistics over datasets, and problem statements. It is a foundational library used by other TensorFlow Extended (TFX) components like TensorFlow Data Validation (TFDV) and ML Metadata (MLMD). The library is actively maintained, with version 1.17.3 being the current release.","status":"active","version":"1.17.3","language":"python","source_language":"en","source_url":"https://github.com/tensorflow/metadata","tags":["tensorflow","metadata","mlops","schema","statistics","protobuf","tfx"],"install":[{"cmd":"pip install tensorflow-metadata","lang":"bash","label":"Latest stable version"}],"dependencies":[{"reason":"Crucial for defining and serializing metadata schemas and statistics. Often a source of version conflicts in the TensorFlow ecosystem.","package":"protobuf"},{"reason":"Used for various foundational utilities within the TensorFlow ecosystem.","package":"absl-py"},{"reason":"Provides common protobuf definitions shared across Google APIs and projects.","package":"googleapis-common-protos"}],"imports":[{"wrong":"from tf_metadata.proto.v0 import schema_pb2","symbol":"schema_pb2","correct":"from tensorflow_metadata.proto.v0 import schema_pb2"}],"quickstart":{"code":"from tf_metadata.proto.v0 import schema_pb2\n\n# Create a simple schema definition\nschema = schema_pb2.Schema()\n\n# Add a feature named 'age' of type INT\nfeature_age = schema.feature.add()\nfeature_age.name = \"age\"\nfeature_age.type = schema_pb2.FeatureType.INT\nfeature_age.int_domain.is_categorical = False\nfeature_age.presence.min_fraction = 1.0 # 'age' must always be present\nfeature_age.int_domain.min = 0\nfeature_age.int_domain.max = 120\n\n# Add a feature named 'city' of type BYTES (string), which is categorical\nfeature_city = schema.feature.add()\nfeature_city.name = \"city\"\nfeature_city.type = schema_pb2.FeatureType.BYTES\nfeature_city.string_domain.is_categorical = True\nfeature_city.string_domain.value.extend([\"New York\", \"London\", \"Tokyo\"])\n\nprint(\"Generated Schema (protobuf format):\")\nprint(schema)\n\n# Serialize the schema to bytes\nserialized_schema = schema.SerializeToString()\nprint(f\"\\nSerialized Schema (bytes): {len(serialized_schema)} bytes\")\n\n# Deserialize the schema back from bytes\ndeserialized_schema = schema_pb2.Schema()\ndeserialized_schema.ParseFromString(serialized_schema)\nprint(\"\\nDeserialized Schema:\")\nprint(deserialized_schema)","lang":"python","description":"This quickstart demonstrates how to define a simple data schema using `tensorflow-metadata`'s protobuf definitions. It shows how to add features with different types and constraints, then serializes and deserializes the schema for storage or transfer."},"warnings":[{"fix":"Consult `tensorflow-metadata`'s `setup.py` or `RELEASE.md` for exact `protobuf` version requirements for your Python version (e.g., `protobuf>=4.25.2,<5` for Python 3.11). Consider using a virtual environment and carefully managing dependencies.","message":"Frequent and critical dependency conflicts with the `protobuf` library. `tensorflow-metadata` often pins `protobuf` to specific major/minor versions, which can clash with other libraries in the TensorFlow ecosystem.","severity":"breaking","affected_versions":"All versions, especially when used in complex environments."},{"fix":"Upgrade to Python 3.9 or higher. The library currently supports Python >=3.9,<4.","message":"Support for Python 3.8 was deprecated starting from version 1.15.0.","severity":"deprecated","affected_versions":"1.15.0 and later."},{"fix":"Always use the stable versions available on PyPI (`pip install tensorflow-metadata`) for production or reliable development. Only use nightly builds if you need the absolute latest features and are prepared to handle instability.","message":"Nightly builds of `tensorflow-metadata` (and related TF projects) are explicitly stated to be unstable and prone to breakages, with fixes potentially taking a week or more.","severity":"gotcha","affected_versions":"Nightly builds."},{"fix":"If you rely on statistics for nested features, re-evaluate existing pipelines and logic when upgrading to 1.15.0 or later, as the reported values might change.","message":"Version 1.15.0 introduced a semantic change to how `min/max/avg/tot num-values` are calculated for nested features, now relying on the innermost level.","severity":"breaking","affected_versions":"1.15.0 and later."},{"fix":"Remove any usage of `NaturalLanguageDomain.location_constraint_regex` from your code.","message":"The field `NaturalLanguageDomain.location_constraint_regex` was removed in version 1.15.0. It was previously documented as 'please do not use' and was never fully implemented.","severity":"breaking","affected_versions":"1.15.0 and later."}],"env_vars":null,"search_vec":"'1.17.3':68 'activ':64 'collect':31 'compon':51 'current':71 'data':27,54 'dataset':36 'describ':25 'e.g':29 'extend':49 'format':23 'foundat':43 'includ':22 'learn':17 'librari':44,62 'like':52 'machin':16 'maintain':65 'metadata':2,4,10,59,74 'ml':58 'mlmd':60 'mlop':75 'model':18 'problem':38 'protobuf':78 'provid':6 'releas':72 'represent':8 'schema':28,76 'standard':7 'statement':39 'statist':34,77 'summari':33 'tabular':26 'tensorflow':1,3,20,48,53,73 'tf.examples':30 'tfdv':56 'tfmd':5 'tfx':50,79 'train':15 'use':13,45 'valid':55 'version':67","created_at":"2026-04-11T01:43:27.084002+00:00","updated_at":"2026-04-16T22:47:56.152012+00:00","problems":null,"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"1.21.0","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/tensorflow/metadata","docs":null,"changelog":null,"pypi":"https://pypi.org/project/tensorflow-metadata/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["ai-ml","data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-30","last_verified":"2026-08-28","next_check":"2026-07-30","install_tag":null}}