lindera-sqlite
| Crates.io | lindera-sqlite |
| lib.rs | lindera-sqlite |
| version | |
| source | src |
| created_at | 2024-12-06 06:38:19.644845 |
| updated_at | 2024-12-06 06:38:19.644845 |
| description | Lindera tokenizer for SQLite FTS5 extention |
| homepage | https://github.com/lindera/lindera-sqlite |
| repository | https://github.com/lindera/lindera-sqlite |
| max_upload_size | |
| id | 1473966 |
| Cargo.toml error: | TOML parse error at line 17, column 1 | 17 | autolib = false | ^^^^^^^ unknown field `autolib`, expected one of `name`, `version`, `edition`, `authors`, `description`, `readme`, `license`, `repository`, `homepage`, `documentation`, `build`, `resolver`, `links`, `default-run`, `default_dash_run`, `rust-version`, `rust_dash_version`, `rust_version`, `license-file`, `license_dash_file`, `license_file`, `licenseFile`, `license_capital_file`, `forced-target`, `forced_dash_target`, `autobins`, `autotests`, `autoexamples`, `autobenches`, `publish`, `metadata`, `keywords`, `categories`, `exclude`, `include` |
| size | 0 |
Minoru Osuka (mosuka)
documentation
https://docs.rs/lindera-sqlite
README
Overview
lindera-sqlite is a C ABI library which exposes a FTS5 tokenizer function.
When used as a custom FTS5 tokenizer this enables application to support Chinese, Japanese and Korean in full-text search.
Build extension
% cargo build --features=ipadic,ko-dic,cc-cedict,compress,extension
Set enviromment variable for Lindera configuration
% export LINDERA_CONFIG_PATH=./resources/lindera.yml
Then start SQLite
% sqlite3 example.db
Load extension
sqlite> .load ./target/debug/liblindera_sqlite lindera_fts5_tokenizer_init
Create table using FTS5 with Lindera tokenizer
sqlite> CREATE VIRTUAL TABLE example USING fts5(content, tokenize='lindera_tokenizer');
Insert data
sqlite> INSERT INTO example(content) VALUES ("Ｌｉｎｄｅｒａは形態素解析ｴﾝｼﾞﾝです。ユーザー辞書も利用可能です。");
Search data
sqlite> SELECT * FROM example WHERE content MATCH "Lindera" ORDER BY bm25(example) LIMIT 10;