{"id":5949,"library":"gptcache","title":"GPTCache","description":"GPTCache is a powerful caching library designed to speed up and lower the cost of chat applications that rely on Large Language Model (LLM) services. It functions as a semantic cache, storing and retrieving responses for similar (not just exact) queries using embedding algorithms and vector stores. The library is actively maintained with frequent minor releases.","status":"active","version":"0.1.44","language":"python","source_language":"en","source_url":"https://github.com/zilliztech/GPTCache","tags":["LLM","cache","AI","performance","cost reduction","semantic cache"],"install":[{"cmd":"pip install gptcache","lang":"bash","label":"Install core library"},{"cmd":"pip install gptcache[openai]","lang":"bash","label":"Install with OpenAI support"},{"cmd":"pip install gptcache[langchain]","lang":"bash","label":"Install with LangChain support"},{"cmd":"pip install gptcache[redis]","lang":"bash","label":"Install with Redis support"}],"dependencies":[{"reason":"Required Python version.","package":"python","version":">=3.8.1"},{"reason":"Optional dependency for distributed caching or using Redis as a cache store.","package":"redis","optional":true},{"reason":"Optional dependency for integration with LangChain.","package":"langchain","optional":true},{"reason":"Transitive dependency, often related to LangChain integrations; specific versions might cause conflicts.","package":"pydantic","optional":true}],"imports":[{"note":"Commonly used pre-configured global cache instance for quick setup.","symbol":"cache","correct":"from gptcache import cache"},{"note":"The main class for creating a GPTCache instance, allowing custom configuration.","symbol":"GPTCache","correct":"from gptcache import GPTCache"},{"note":"Adapter to integrate GPTCache with the OpenAI API calls.","symbol":"openai","correct":"from gptcache.adapter import openai"}],"quickstart":{"code":"import os\nfrom gptcache import cache\nfrom gptcache.adapter import openai\n\n# Set your OpenAI API key from an environment variable\nos.environ[\"OPENAI_API_KEY\"] = os.environ.get(\"OPENAI_API_KEY\", \"sk-...\")\n\n# Initialize GPTCache\ncache.init()\n\n# The gptcache.adapter.openai module automatically wraps the openai library\n# Subsequent OpenAI API calls will use the cache\nresponse1 = openai.ChatCompletion.create(\n    model=\"gpt-3.5-turbo\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"Hello, what is the capital of France?\"}\n    ]\n)\nprint(f\"First response (likely from LLM): {response1.choices[0].message.content}\")\n\n# A second identical request will hit the cache for faster response and cost savings\nresponse2 = openai.ChatCompletion.create(\n    model=\"gpt-3.5-turbo\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"Hello, what is the capital of France?\"}\n    ]\n)\nprint(f\"Second response (likely from cache): {response2.choices[0].message.content}\")","lang":"python","description":"This quickstart demonstrates how to integrate GPTCache with the OpenAI API. After initializing the cache, subsequent OpenAI calls will automatically leverage the semantic caching capabilities. The first query will likely go to the LLM, while identical or semantically similar subsequent queries will be served from the cache."},"warnings":[{"fix":"Upgrade GPTCache to version 0.1.43 or newer to resolve known compatibility issues with Pydantic v2 and LangChain.","message":"When integrating with LangChain, particularly with Pydantic v2, older versions of GPTCache might have caused 'metaclass conflict errors' or 'LangChain chat pydantic bugs'.","severity":"gotcha","affected_versions":"<0.1.43"},{"fix":"Ensure `pip install gptcache[redis]` if you plan to use Redis. Upgrade to at least 0.1.36 for critical Redis connection fixes, and 0.1.43 to benefit from `redis` being an optional dependency, avoiding unnecessary installs.","message":"Using certain features like remote Redis cache stores or distributed caching might require explicit installation of `redis` and can encounter connection issues in older versions.","severity":"gotcha","affected_versions":"<0.1.43 (for optional Redis install), <0.1.36 (for Redis connection fix)"},{"fix":"Regularly update GPTCache to its latest version to ensure compatibility with evolving LLM APIs. Version 0.1.38 addressed specific OpenAI API base changes.","message":"Changes in external LLM APIs (e.g., OpenAI's API base for embeddings) can cause unexpected behavior or errors if GPTCache is not updated to reflect these changes.","severity":"gotcha","affected_versions":"<0.1.38"}],"env_vars":null,"search_vec":"'activ':52 'ai':60 'algorithm':45 'applic':18 'cach':6,32,59,65 'chat':17 'cost':15,62 'design':8 'embed':44 'exact':41 'frequent':55 'function':28 'gptcach':1,2 'languag':23 'larg':22 'librari':7,50 'llm':25,58 'lower':13 'maintain':53 'minor':56 'model':24 'perform':61 'power':5 'queri':42 'reduct':63 'releas':57 'reli':20 'respons':36 'retriev':35 'semant':31,64 'servic':26 'similar':38 'speed':10 'store':33,48 'use':43 'vector':47","created_at":"2026-04-14T18:35:25.883336+00:00","updated_at":"2026-04-16T15:26:27.352465+00:00","problems":[{"fix":"Ensure that `cache.init()` is called at the beginning of your application or before any operations that interact with the cache. For semantic caching, you might use `init_similar_cache()`. \n```python\nfrom gptcache import cache\nfrom gptcache.adapter.api import init_similar_cache\n\n# For basic exact caching\ncache.init()\n\n# Or for semantic caching\n# init_similar_cache()\n\n# Your code that uses gptcache\n```","cause":"This error occurs when an attempt is made to use the GPTCache instance (or an adapter that relies on it) before the cache has been properly initialized using the `cache.init()` method.","error":"gptcache.utils.error.NotInitError: The cache should be inited before using."},{"fix":"You can either downgrade your `openai` package to a version less than 1.0.0 (e.g., `pip install openai==0.28.1`) or update your `gptcache` integration code to be compatible with `openai` version 1.0.0+ syntax, which often involves using `client.chat.completions.create` instead of `openai.ChatCompletion.create`.","cause":"This error indicates a version incompatibility between your installed `openai` library (which is version 1.0.0 or higher) and code that expects the older `openai` API syntax (typically pre-1.0.0), often encountered when `gptcache` adapters or examples were written for an older OpenAI library version.","error":"openai.lib._old_api.APIRemovedInV1: You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0"},{"fix":"This often requires ensuring your input text for embedding generation is within the model's supported sequence length. You might need to preprocess the input to truncate or split it, or configure `gptcache` with a preprocessing function that handles input lengths (e.g., `pre_embedding_func` or `pre_func` in `cache.init()`).","cause":"This specific ONNXRuntimeError arises when the input tensor's dimensions, particularly for `token_type_ids` during embedding generation, do not match the expected dimensions of the underlying ONNX model used by GPTCache for similarity search, often due to an input text exceeding the model's maximum sequence length.","error":"InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: token_type_ids for the following indices index: 1 Got: 1772 Expected: 512 Please fix either the inputs or the model."},{"fix":"Review the custom `pre_embedding_func` or `post_process_messages_func` passed to `cache.init()` or directly to LangChain's `GPTCache` adapter. Ensure these functions always return a valid, subscriptable object (e.g., a dictionary, list, or string) rather than `None`, especially for edge cases or unexpected inputs. Default functions like `get_prompt` or `last_content` are provided by `gptcache` to correctly handle common scenarios.","cause":"This common Python error, in the context of `gptcache` and LangChain integration, often means that a function like `pre_embedding_func` or `post_process_messages_func` has returned `None` when a dictionary or another subscriptable object (like a list) was expected by a subsequent operation. This can happen if the function's logic doesn't cover all input scenarios or is incorrectly configured.","error":"TypeError: 'NoneType' object is not subscriptable"},{"fix":"Reduce the dimension of the embeddings used in your vector store, or adjust PostgreSQL's configuration if possible (though increasing `MAXALIGN` is complex and often not recommended). If using `pgvector`, ensure your `VectorBase` is configured with a suitable `dimension`. Consider alternative vector stores or strategies for handling very high-dimensional embeddings.","cause":"This error, specifically from `psycopg2` (PostgreSQL adapter), occurs when using `gptcache` with a PostgreSQL-backed vector store (like `PGVector`) where the size of an index entry (e.g., for an embedding) exceeds the maximum allowed size for an index row in PostgreSQL. This is often due to very large embedding dimensions combined with other indexed data.","error":"psycopg2.errors.ProgramLimitExceeded: index row requires X bytes, maximum size is Y"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.1.44","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"https://github.com/zilliztech/GPTCache","docs":null,"changelog":null,"pypi":"https://pypi.org/project/gptcache/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["llm-agents","ai-ml","vector-search"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-06-28","next_check":"2026-07-28","install_tag":null}}