nnsplit
| Crates.io | nnsplit |
| lib.rs | nnsplit |
| version | 0.5.9 |
| source | src |
| created_at | 2020-02-10 13:23:46.887377 |
| updated_at | 2023-03-05 19:32:03.775901 |
| description | A tool to split text using a neural network. For sentence boundary detection, compound splitting and more. |
| homepage | https://bminixhofer.github.io/nnsplit |
| repository | https://github.com/bminixhofer/nnsplit |
| max_upload_size | |
| id | 207002 |
| size | 75,528 |
Benjamin Minixhofer (bminixhofer)
documentation
https://docs.rs/nnsplit/
README
NNSplit
A tool to split text using a neural network. The main application is sentence boundary detection, but e. g. compound splitting for German is also supported.
Features
- Robust: Not reliant on proper punctuation, spelling and case. See the metrics.
- Small: NNSplit uses a byte-level LSTM, so weights are small (< 4MB) and models can be trained for every unicode encodable language.
- Portable: NNSplit is written in Rust with bindings for Rust, Python, and Javascript (Browser and Node.js). See how to get started in the usage section.
- Fast: Up to 2x faster than Spacy sentencization, see the benchmark.
- Multilingual: NNSplit currently has models for 9 different languages (German, English, French, Norwegian, Swedish, Simplified Chinese, Turkish, Russian and Ukrainian). Try them in the demo.
Documentation has moved to the NNSplit website: https://bminixhofer.github.io/nnsplit.
License
NNSplit is licensed under the MIT license.