ragit-korean

Crates.ioragit-korean
lib.rsragit-korean
version
sourcesrc
created_at2024-12-30 14:17:57.361309
updated_at2025-02-01 12:18:53.865197
descriptionkorean tokenizer for ragit
homepage
repository
max_upload_size
id1499190
Cargo.toml error:TOML parse error at line 17, column 1 | 17 | autolib = false | ^^^^^^^ unknown field `autolib`, expected one of `name`, `version`, `edition`, `authors`, `description`, `readme`, `license`, `repository`, `homepage`, `documentation`, `build`, `resolver`, `links`, `default-run`, `default_dash_run`, `rust-version`, `rust_dash_version`, `rust_version`, `license-file`, `license_dash_file`, `license_file`, `licenseFile`, `license_capital_file`, `forced-target`, `forced_dash_target`, `autobins`, `autotests`, `autoexamples`, `autobenches`, `publish`, `metadata`, `keywords`, `categories`, `exclude`, `include`
size0
(baehyunsol)

documentation

README

ragit-korean

Ragit-korean is a very simple korean tokenizer.

Ragit used to use charabia to tokenize cjk documents, but it has too many issues.

  1. Charabia bundles cjk dictionaries in the binary, which makes the file 70MiB bigger.
  2. It silently converts 완성형 korean to 조합형 korean. That silently messes up tfidf searches.
Commit count: 0

cargo fmt