{"id":7433,"library":"msgpack-numpy-opentensor","title":"msgpack-numpy-opentensor","description":"msgpack-numpy-opentensor provides efficient serialization and deserialization routines for NumPy array and scalar data types using the MessagePack binary format. It is functionally derived from the `msgpack-numpy` library, offering compatibility with NumPy data structures. The current PyPI version is `0.5.0`. While its related GitHub repository shows more recent development, PyPI releases are infrequent, with the latest over a year old.","status":"active","version":"0.5.0","language":"python","source_language":"en","source_url":"https://github.com/opentensor/msgpack-numpy","tags":["serialization","numpy","msgpack","binary","data-interchange","opentensor"],"install":[{"cmd":"pip install msgpack-numpy-opentensor","lang":"bash","label":"PyPI Installation"}],"dependencies":[{"reason":"Core MessagePack serialization library.","package":"msgpack"},{"reason":"Provides the array and numerical types for serialization.","package":"numpy"}],"imports":[{"wrong":"import msgpack_numpy_opentensor as m; m.patch()","symbol":"patch","correct":"from msgpack_numpy import patch"},{"wrong":"import msgpack_numpy_opentensor as m; m.Packer","symbol":"Packer","correct":"from msgpack_numpy import Packer"},{"wrong":"import msgpack_numpy_opentensor as m; m.Unpacker","symbol":"Unpacker","correct":"from msgpack_numpy import Unpacker"}],"quickstart":{"code":"import numpy as np\nimport msgpack\nimport msgpack_numpy_opentensor as m\n\n# Create a NumPy array\nx = np.random.rand(5, 5)\n\n# Pack the NumPy array using msgpack-numpy-opentensor's encoder\n# Optionally, you can call m.patch() to monkey-patch msgpack globally\n# m.patch()\npacked_x = msgpack.packb(x, default=m.encode)\n\n# Unpack the bytes back into a NumPy array using the decoder\nunpacked_x = msgpack.unpackb(packed_x, object_hook=m.decode, raw=False)\n\nprint(\"Original array:\\n\", x)\nprint(\"Unpacked array:\\n\", unpacked_x)\nprint(\"Arrays are equal:\", np.array_equal(x, unpacked_x))\nprint(\"Unpacked array is read-only:\", not unpacked_x.flags['WRITEABLE'])\n","lang":"python","description":"This quickstart demonstrates how to serialize a NumPy array into MessagePack binary format and then deserialize it back into a NumPy array using the `msgpack-numpy-opentensor` library. It shows both packing (`packb`) and unpacking (`unpackb`) using the provided encoder and decoder functions, highlighting that deserialized arrays are typically read-only."},"warnings":[{"fix":"Review `dtype='O'` usage and consider custom serializers for such arrays if you rely on pickle for compatibility. Be prepared to update serialization/deserialization logic if upgrading to a version with this change.","message":"The upstream `opentensor/msgpack-numpy` GitHub repository, linked as this package's source, released `v1.0.0` with a breaking change: it disables `pickle` by default. This will prevent deserialization of NumPy arrays with `dtype='O'` (object arrays) that were serialized with pickle enabled in older versions. While `msgpack-numpy-opentensor` on PyPI is currently `0.5.0`, this change may propagate to future versions.","severity":"breaking","affected_versions":"Potentially `1.0.0+` (if adopted by `msgpack-numpy-opentensor`)"},{"fix":"Avoid `dtype='O'` for sensitive or performance-critical data. Consider explicitly converting object arrays to more primitive types or implementing custom, secure encoders/decoders for specific object types.","message":"NumPy arrays with `dtype='O'` (object arrays) are serialized/deserialized using Python's `pickle` module as a fallback by `msgpack-numpy` (and, by extension, likely `msgpack-numpy-opentensor`). This introduces significant performance overhead and poses security risks when deserializing data from untrusted sources due to pickle's arbitrary code execution capabilities.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If modification is required, explicitly create a writable copy of the array after deserialization, e.g., `modified_array = unpacked_array.copy()`.","message":"NumPy arrays deserialized by `msgpack-numpy` (and thus, `msgpack-numpy-opentensor`) are read-only by default. Attempting to modify them directly will raise a `ValueError` or `AttributeError`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For extremely large NumPy arrays, consider chunking them into smaller pieces before serialization, using alternative serialization formats designed for larger-than-memory data, or streaming solutions.","message":"The underlying `msgpack` library has limitations on the maximum size of individual binary or string objects, typically around 4.3 GB. Attempting to serialize a single NumPy array that exceeds this limit may result in serialization errors.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'0.5.0':48 'array':17 'binari':25,72 'compat':38 'current':44 'data':20,41,74 'data-interchang':73 'deriv':30 'deseri':13 'develop':57 'effici':10 'format':26 'function':29 'github':52 'infrequ':61 'interchang':75 'latest':64 'librari':36 'messagepack':24 'msgpack':2,6,34,71 'msgpack-numpi':33 'msgpack-numpy-opentensor':1,5 'numpi':3,7,16,35,40,70 'offer':37 'old':68 'opentensor':4,8,76 'provid':9 'pypi':45,58 'recent':56 'relat':51 'releas':59 'repositori':53 'routin':14 'scalar':19 'serial':11,69 'show':54 'structur':42 'type':21 'use':22 'version':46 'year':67","created_at":"2026-04-16T13:58:05.786185+00:00","updated_at":"2026-04-16T13:58:05.786185+00:00","problems":null,"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.5.0","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/opentensor/msgpack-numpy","docs":null,"changelog":null,"pypi":"https://pypi.org/project/msgpack-numpy-opentensor/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["serialization","data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-30","last_verified":"2026-06-30","next_check":"2026-07-30","install_tag":null}}