{"id":4206,"library":"pysbd","title":"pysbd (Python Sentence Boundary Disambiguation)","description":"pysbd (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection library that works out-of-the-box across many languages. It aims to provide accurate sentence segmentation even with complex text, abbreviations, and varied punctuation, offering an alternative to neural network-based approaches. The current version is 0.3.4, and the project appears to be actively maintained.","status":"active","version":"0.3.4","language":"python","source_language":"en","source_url":"https://github.com/nipunsadvilkar/pySBD","tags":["nlp","sentence segmentation","text processing","rule-based","multilingual"],"install":[{"cmd":"pip install pysbd","lang":"bash","label":"PyPI"},{"cmd":"conda install anaconda::pysbd","lang":"bash","label":"Conda"}],"dependencies":[{"reason":"Optional, for integration as a spaCy pipeline component.","package":"spacy","optional":true}],"imports":[{"note":"Primary class for sentence segmentation","symbol":"Segmenter","correct":"import pysbd; segmenter = pysbd.Segmenter(...)"},{"note":"PySBDFactory is specifically for spaCy integration and resides in pysbd.utils, not directly under pysbd or pysbd.segmenter.","wrong":"from pysbd.segmenter import PySBDFactory","symbol":"PySBDFactory","correct":"from pysbd.utils import PySBDFactory"}],"quickstart":{"code":"import pysbd\n\ntext = \"Dr. Smith went to the U.S. last week. He said, 'Hello!' How are you?\"\n\n# Initialize segmenter for English\nsegmenter = pysbd.Segmenter(language=\"en\", clean=False)\n\nsentences = segmenter.segment(text)\n\nfor i, sent in enumerate(sentences):\n    print(f\"Sentence {i+1}: {sent}\")","lang":"python","description":"This example demonstrates basic sentence segmentation using the `Segmenter` class for English text. The `clean=False` parameter is used to prevent aggressive text cleaning."},"warnings":[{"fix":"For spaCy v3.x, use `@Language.factory('pysbd_segmenter')` decorator on a custom factory function, then `nlp.add_pipe('pysbd_segmenter')`. Refer to spaCy's updated documentation for custom pipeline components.","message":"When integrating with spaCy, examples for spaCy v2.x using `nlp.add_pipe(PySBDFactory(nlp))` are not compatible with spaCy v3.x. The `add_pipe` API changed to expect a string name of the registered component factory.","severity":"breaking","affected_versions":"spaCy v3.x and later"},{"fix":"If precise, identical segmentation is critical across different integration methods, thoroughly test both approaches with your specific text data. Direct usage (`pysbd.Segmenter().segment(text)`) generally aligns with the expected rule-based output.","message":"Segmentation results might differ slightly when using `pysbd.Segmenter` directly compared to using `pysbd` as a spaCy pipeline component, especially with quoted text or complex punctuation.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If you need to preserve the original text as much as possible, set `clean=False` (which is the default in some examples but not universally). Implement custom cleaning steps if specific pre-processing is required.","message":"The `clean=True` parameter in the `Segmenter` constructor performs aggressive pre-filtering of the input text, removing repeated punctuation, line breaks, URLs, and HTML tags. This might alter the original text more than desired for certain NLP tasks.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Initialize the segmenter as `segmenter = pysbd.Segmenter(language='en', char_span=True)`. The output will then be a list of `TextSpan` objects, each containing the segmented sentence, its start, and end character indices.","message":"By default, `pysbd.Segmenter` returns a list of strings. If you require character offsets into the original text for non-destructive tokenization, you must initialize `Segmenter` with `char_span=True`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Consider performance implications for very large text corpora or real-time applications. Benchmark `pysbd` against other tools if speed is a primary concern. For high accuracy where speed is not the absolute bottleneck, `pysbd` remains a strong choice.","message":"While highly accurate, `pysbd` is a rule-based system implemented in Python. It may be slower compared to some sentence boundary detection alternatives that are implemented in lower-level languages like C++ or optimized using Cython.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'0.3.4':58 'abbrevi':41 'accur':34 'across':27 'activ':65 'aim':31 'altern':47 'appear':62 'approach':53 'base':15,52,74 'boundari':4,9,17 'box':26 'complex':39 'current':55 'detect':18 'disambigu':5,10 'even':37 'languag':29 'librari':19 'maintain':66 'mani':28 'multilingu':75 'network':51 'network-bas':50 'neural':49 'nlp':67 'offer':45 'out-of-the-box':22 'process':71 'project':61 'provid':33 'punctuat':44 'pysbd':1,6 'python':2,7 'rule':14,73 'rule-bas':13,72 'segment':36,69 'sentenc':3,8,16,35,68 'text':40,70 'vari':43 'version':56 'work':21","created_at":"2026-04-12T03:45:29.145635+00:00","updated_at":"2026-04-16T19:47:52.335392+00:00","problems":{"verify_error":"/tmp/tmp6e1dt_gj/venv/lib/python3.12/site-packages/pysbd/segmenter.py:66: SyntaxWarning: invalid escape sequence '\\s'\n  for match in re.finditer('{0}\\s*'.format(re.escape(sent)), self.original_text):\n/tmp/tmp6e1dt_gj/venv/lib/python3.12/site-packages/pysbd/lang/arabic.py:29: SyntaxWarning: invalid e"},"ecosystem":"pypi","meta_description":null,"install_score":null,"quickstart_score":null,"quickstart_tag":null,"pypi_latest":"0.3.4","cli_name":"","cli_version":null,"type":"library","homepage":"http://nipunsadvilkar.github.io/","github":null,"docs":null,"changelog":null,"pypi":"https://pypi.org/project/pysbd/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-06-28","last_verified":"2026-08-29","next_check":"2026-07-09","install_tag":null}}