rten-text
| Crates.io | rten-text |
| lib.rs | rten-text |
| version | 0.22.1 |
| created_at | 2023-12-31 20:31:52.485095+00 |
| updated_at | 2025-09-18 18:22:12.921066+00 |
| description | Text tokenization and other ML pre/post-processing functions |
| homepage | https://github.com/robertknight/rten |
| repository | https://github.com/robertknight/rten |
| max_upload_size | |
| id | 1084992 |
| size | 146,808 |
Robert Knight (robertknight)
documentation
README
rten-text
Library containing text tokenization and related functionality, for preparing inputs and decoding outputs for text models (eg. BERT).
The functionality is a subset of that found in Hugging Face Tokenizers. It has less functionality, but also fewer dependencies, and none that require C/C++.