rten-text
| Crates.io | rten-text |
| lib.rs | rten-text |
| version | 0.24.0 |
| created_at | 2023-12-31 20:31:52.485095+00 |
| updated_at | 2025-12-23 15:00:24.001453+00 |
| description | Text tokenization and other ML pre/post-processing functions |
| homepage | https://github.com/robertknight/rten |
| repository | https://github.com/robertknight/rten |
| max_upload_size | |
| id | 1084992 |
| size | 149,523 |
Robert Knight (robertknight)
documentation
README
rten-text
Library containing text tokenization and related functionality, for preparing inputs and decoding outputs for text models (eg. BERT).
The functionality is a subset of that found in Hugging Face Tokenizers. It has less functionality, but also fewer dependencies, and none that require C/C++.