{"id":2951,"library":"flashtext","title":"Flashtext Keyword Processor","description":"Flashtext is a Python library designed for efficient keyword extraction and replacement in sentences. It employs a custom algorithm based on Aho-Corasick and Trie data structures, providing significant performance gains over regular expressions, especially for large dictionaries of keywords. The current stable version is 2.7, released in 2018, and it is largely in a maintenance state, though still widely used.","status":"maintenance","version":"2.7","language":"python","source_language":"en","source_url":"https://github.com/vi3k6i5/flashtext","tags":["text-processing","keyword-extraction","keyword-replacement","nlp","performance"],"install":[{"cmd":"pip install flashtext","lang":"bash","label":"Install stable version"}],"dependencies":[],"imports":[{"symbol":"KeywordProcessor","correct":"from flashtext import KeywordProcessor"}],"quickstart":{"code":"from flashtext import KeywordProcessor\n\n# Initialize the keyword processor (case_sensitive=False by default)\nkeyword_processor = KeywordProcessor()\n\n# Add keywords. Can map multiple 'unclean' names to one 'clean' name.\nkeyword_processor.add_keyword('Big Apple', 'New York')\nkeyword_processor.add_keyword('Bay Area')\nkeyword_processor.add_keyword('New Delhi', 'NCR region')\n\n# Extract keywords\ntext_to_extract = 'I love Big Apple and Bay Area. New Delhi is also great.'\nkeywords_found = keyword_processor.extract_keywords(text_to_extract)\nprint(f\"Extracted keywords: {keywords_found}\") # Expected: ['New York', 'Bay Area', 'NCR region']\n\n# Replace keywords\ntext_to_replace = 'I love Big Apple and new delhi.'\nnew_sentence = keyword_processor.replace_keywords(text_to_replace)\nprint(f\"Replaced sentence: {new_sentence}\") # Expected: 'I love New York and NCR region.'\n\n# Extract with span information\nkeywords_with_span = keyword_processor.extract_keywords('I love Big Apple.', span_info=True)\nprint(f\"Keywords with span: {keywords_with_span}\") # Expected: [('New York', 7, 16)]","lang":"python","description":"This example demonstrates how to initialize the `KeywordProcessor`, add keywords with optional clean names, and then use it to extract or replace keywords in a given text. It also shows how to get span information for extracted keywords."},"warnings":[{"fix":"For non-Latin languages or custom boundary needs, initialize `KeywordProcessor` with a modified `non_word_boundaries` set. Example: `kp = KeywordProcessor(non_word_boundaries=set(['@', '#']))`.","message":"Flashtext's default word boundary definition (`[A-Za-z0-9_]`) might not be suitable for all languages (e.g., Chinese, Japanese) or custom requirements. It may fail to identify keywords correctly if they are not separated by these specific non-word characters. Users can customize `non_word_boundaries`.","severity":"gotcha","affected_versions":"2.0 - 2.7"},{"fix":"Benchmark performance for your specific use case. If keyword count is low or complex pattern matching is needed, consider standard regex. If keyword count is high, Flashtext is highly optimized for speed.","message":"Flashtext generally outperforms regex for keyword extraction/replacement when the number of keywords is large (typically >500). For a small number of keywords or when complex patterns (like partial matches or special character handling) are required, regular expressions might be equally or more efficient, or simply the only solution.","severity":"gotcha","affected_versions":"2.0 - 2.7"},{"fix":"Ensure that `add_keyword()` is only provided with string `clean_name` values if `replace_keywords()` functionality is intended. Tuple clean names are primarily for enhanced extraction information.","message":"If `add_keyword()` is used with a tuple as the `clean_name` (e.g., `add_keyword('Taj Mahal', ('Monument', 'Taj Mahal'))`), the `replace_keywords()` method will not function as expected because it anticipates a string replacement, not a tuple.","severity":"gotcha","affected_versions":"2.0 - 2.7"},{"fix":"Consider migrating to `flashtext2` for improved performance and broader language support, especially if hitting performance bottlenecks or unicode issues with the original `flashtext`. Be aware of potential API differences or slight behavior changes, though the core API is similar.","message":"A separate, community-driven package `flashtext2` (and `flashtextr`) exists, which is a rewrite in Rust, offering significant performance improvements (3-10x faster) and better Unicode handling. While not an official successor from the original author, it addresses some limitations of `flashtext`.","severity":"deprecated","affected_versions":"All versions of `flashtext` (2.x)"}],"env_vars":null,"search_vec":"'2.7':50 '2018':53 'aho':26 'aho-corasick':25 'algorithm':22 'base':23 'corasick':27 'current':46 'custom':21 'data':30 'design':9 'dictionari':42 'effici':11 'employ':19 'especi':39 'express':38 'extract':13,71 'flashtext':1,4 'gain':35 'keyword':2,12,44,70,73 'keyword-extract':69 'keyword-replac':72 'larg':41,57 'librari':8 'mainten':60 'nlp':75 'perform':34,76 'process':68 'processor':3 'provid':32 'python':7 'regular':37 'releas':51 'replac':15,74 'sentenc':17 'signific':33 'stabl':47 'state':61 'still':63 'structur':31 'text':67 'text-process':66 'though':62 'trie':29 'use':65 'version':48 'wide':64","created_at":"2026-04-11T09:14:35.530010+00:00","updated_at":"2026-04-16T15:05:58.712675+00:00","problems":[{"fix":"Install the library using pip: `pip install flashtext`","cause":"The 'flashtext' library is not installed in the Python environment, or the environment where the code is run does not have it installed.","error":"ModuleNotFoundError: No module named 'flashtext'"},{"fix":"Ensure that `case_sensitive` is set appropriately, especially for text with special characters. If the issue persists, consider pre-processing non-ASCII characters or ensuring the Flashtext version is compatible with your Python environment's string handling. One reported fix involved adjusting string length assumptions in the library's internal code.","cause":"This error typically occurs when `replace_keywords` is used with a sentence containing non-ASCII characters or in case-insensitive mode, where the length of the lowercased string differs from the original, causing an incorrect index lookup.","error":"IndexError: string index out of range (in flashtext.keyword.py)"},{"fix":"Ensure the `KeywordProcessor` object is correctly initialized and the extraction function is applied properly across the DataFrame. For PySpark, this often involves creating the `KeywordProcessor` instance and using a UDF to apply `extract_keywords` to a column.","cause":"When integrating `flashtext` with dataframes (e.g., Pandas or PySpark), the `KeywordProcessor` might not be correctly applied across all rows or partitions, leading to incomplete extraction or 'NaN' values. This often happens due to incorrect application of UDFs (User Defined Functions) in distributed environments or subtle data type mismatches.","error":"flashtext.KeywordProcessor not extracting all keywords or returning NaN in DataFrame"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"2.7","cli_name":"","cli_version":null,"type":"library","homepage":null,"github":"http://github.com/vi3k6i5/flashtext","docs":null,"changelog":null,"pypi":"https://pypi.org/project/flashtext/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data","ai-ml"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-28","next_check":"2026-07-28","install_tag":null}}