text-tokenizer
| Crates.io | text-tokenizer |
| lib.rs | text-tokenizer |
| version | 0.5.6 |
| source | src |
| created_at | 2023-12-20 11:59:26.645572 |
| updated_at | 2024-09-06 10:13:46.749153 |
| description | Custom text tokenizer |
| homepage | https://github.com/merl-twin/text-tokenizer |
| repository | https://github.com/merl-twin/text-tokenizer |
| max_upload_size | |
| id | 1075234 |
| size | 399,415 |
(merl-twin)
documentation
README
Custom text tokenizer