Crates.io | morsels_lang_chinese |
lib.rs | morsels_lang_chinese |
version | 0.7.3 |
source | src |
created_at | 2021-12-09 08:06:52.220837 |
updated_at | 2022-10-24 05:10:04.34345 |
description | Chinese tokenizer for morsels. |
homepage | |
repository | https://github.com/ang-zeyu/morsels |
max_upload_size | |
id | 495028 |
size | 118,572 |
This is a tokenizer crate for the morsels_indexer
cli tool, not intended to be used directly.