{"id":5262,"library":"ipex-llm","title":"IPEX-LLM","description":"IPEX-LLM is a PyTorch-based library developed by Intel for optimizing Large Language Models (LLMs) on Intel CPUs and GPUs (XPUs). It provides tools for efficient inference and fine-tuning, leveraging Intel hardware accelerations. The current stable version is 2.2.0, with frequent nightly builds and updates, often released in conjunction with the broader BigDL project.","status":"active","version":"2.2.0","language":"python","source_language":"en","source_url":"https://github.com/intel-analytics/ipex-llm","tags":["LLM","AI","Intel","optimization","deep-learning","pytorch","transformers"],"install":[{"cmd":"pip install ipex-llm[cpu]","lang":"bash","label":"For CPU inference"},{"cmd":"pip install --pre --upgrade ipex-llm[xpu]","lang":"bash","label":"For Intel GPUs (XPUs), often with nightly builds"}],"dependencies":[{"reason":"IPEX-LLM is built on PyTorch and requires a compatible version.","package":"torch","optional":false}],"imports":[{"wrong":"from ipex_llm import LLM","symbol":"LLM","correct":"from ipex_llm import LLM"}],"quickstart":{"code":"from ipex_llm import LLM\n\n# Instantiate LLM model\nmodel = LLM(\n    model_name='/path/to/your/model',\n    optimize_type='int4',\n    dtype='auto',\n    trust_remote_code=True\n)\n\n# Example for text generation\nprompt = \"What is the capital of France?\"\noutput = model(prompt)\nprint(output)\n\n# For AutoModel/AutoTokenizer\nfrom ipex_llm.transformers import AutoModel, AutoTokenizer\n\nmodel_id = \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\"\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModel.from_pretrained(\n    model_id,\n    load_in_4bit=True, # or load_in_low_bit, quantize=4 etc.\n    torch_dtype='auto'\n)\n\ninput_ids = tokenizer.encode(prompt, return_tensors=\"pt\")\noutput = model.generate(input_ids, max_new_tokens=32)\nprint(tokenizer.decode(output[0], skip_special_tokens=True))","lang":"python","description":"This quickstart demonstrates how to load an LLM using the `ipex_llm.LLM` class for simple inference or `ipex_llm.transformers.AutoModel` and `AutoTokenizer` for more fine-grained control and compatibility with Hugging Face Transformers. Ensure your `model_name` or `model_id` points to a valid local path or Hugging Face model."},"warnings":[{"fix":"Update your `pip install` commands from `bigdl-llm` to `ipex-llm`. Adjust import statements from `from bigdl.llm...` to `from ipex_llm...`.","message":"The library was rebranded from `BigDL-LLM` to `ipex-llm`. This changes package names, import paths, and some CLI tools.","severity":"breaking","affected_versions":"All versions prior to 2.x (BigDL-LLM) when migrating to 2.x (ipex-llm)."},{"fix":"Always use `pip install ipex-llm[cpu]` or `pip install ipex-llm[xpu]` as appropriate for your system. Refer to the official documentation for detailed hardware requirements.","message":"IPEX-LLM installations are hardware-specific. Users must install the correct extras for their target platform (`[cpu]` for Intel CPUs or `[xpu]` for Intel GPUs). Installing without the correct extra may lead to missing dependencies or suboptimal performance.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always check the official IPEX-LLM documentation and release notes for the recommended PyTorch and oneAPI versions for your IPEX-LLM release. Consider using the `--pre` flag during installation to get the latest compatible builds.","message":"IPEX-LLM's performance is highly dependent on specific PyTorch and underlying Intel oneAPI library versions. Incompatible versions can lead to errors or degraded performance.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For new code, prefer using `ipex_llm.LLM` or `ipex_llm.transformers.AutoModel.from_pretrained()` with `load_in_4bit`/`load_in_low_bit` for easier integration and more streamlined workflows.","message":"While still functional, the `ipex_llm.optimize_model` API is being superseded by the higher-level `ipex_llm.LLM` and `ipex_llm.transformers.AutoModel` APIs for model loading and quantization.","severity":"deprecated","affected_versions":"2.x onwards"}],"env_vars":null,"search_vec":"'2.2.0':47 'acceler':41 'ai':64 'base':11 'bigdl':61 'broader':60 'build':51 'conjunct':57 'cpus':24 'current':43 'deep':68 'deep-learn':67 'develop':13 'effici':32 'fine':36 'fine-tun':35 'frequent':49 'gpus':26 'hardwar':40 'infer':33 'intel':15,23,39,65 'ipex':2,5 'ipex-llm':1,4 'languag':19 'larg':18 'learn':69 'leverag':38 'librari':12 'llm':3,6,63 'llms':21 'model':20 'night':50 'often':54 'optim':17,66 'project':62 'provid':29 'pytorch':10,70 'pytorch-bas':9 'releas':55 'stabl':44 'tool':30 'transform':71 'tune':37 'updat':53 'version':45 'xpus':27","created_at":"2026-04-14T01:26:12.861251+00:00","updated_at":"2026-04-16T15:47:14.911656+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\n  File \"/tmp/tmp08mwxef6/venv/lib/python3.12/site-packages/ipex_llm/__init__.py\", line 22, in <module>\n    from .convert_model import llm_convert\n  File \"/tmp/tmp08mwxef6/venv/lib/python3.12/site-packages/ipex_llm/convert_mode"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"2.2.0","cli_name":"ipex-llm","cli_version":"sh: 1: ipex-llm: not found","type":"library","homepage":null,"github":"https://github.com/intel-analytics/ipex-llm","docs":null,"changelog":null,"pypi":"https://pypi.org/project/ipex-llm/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["ai-ml","llm-agents"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-07-03","last_verified":"2026-08-30","next_check":"2026-07-10","install_tag":null}}