ngram-search

Crates.iongram-search
lib.rsngram-search
version0.1.1
sourcesrc
created_at2020-09-01 21:11:24.400906
updated_at2020-09-01 21:24:10.748124
descriptionNgram-based indexing of strings into a binary file
homepage
repositoryhttps://gitlab.com/remram44/ngram-search
max_upload_size
id283638
size63,792
Remi Rampin (remram44)

documentation

README

This crate allows indexing many strings into a file, and then efficiently fuzzy-matching strings against what's been indexed.

Currently, the structure is built in memory before being written to the file, so that phase uses a lot of RAM.

String search is done from the file and requires little memory.

The index is a trie structure in which trigrams can be looked up; results for each trigrams of the input are matched and sorted to get the most similar strings.

Example:

// Build index
let mut builder = Ngrams::builder();
builder.add("spam", 0);
builder.add("ham", 1);
builder.add("mam", 2);

// Write it to a file
let mut file = BufWriter::new(File::create(path).unwrap());
builder.write(&mut file).unwrap();

// Search our index
let mut data = Ngrams::open(path).unwrap();
assert_eq!(
    data.search("ham", 0.24).unwrap(),
    vec![
        (1, 1.0), // "ham" is an exact match
        (2, 0.25), // "mam" is close
    ],
);
assert_eq!(
    data.search("spa", 0.2).unwrap(),
    vec![
        (0, 0.22222222), // "spam" is close
    ],
);
Commit count: 0

cargo fmt