esaxx-rs

Crates.ioesaxx-rs
lib.rsesaxx-rs
version0.1.10
sourcesrc
created_at2020-06-06 08:14:01.087605
updated_at2023-10-05 13:12:19.68833
descriptionWrapping around sentencepiece's esaxxx library.
homepagehttps://github.com/Narsil/esaxx-rs
repositoryhttps://github.com/Narsil/esaxx-rs
max_upload_size
id250592
size507,207
Nicolas Patry (Narsil)

documentation

https://docs.rs/esaxx-rs

README

esaxx-rs

This code implements a fast suffix tree / suffix array.

This code is taken from sentencepiece and to be used by hugging face.

Small wrapper around sentencepiece's esaxx suffix array C++ library. Usage

let string = "abracadabra";
let suffix = esaxx_rs::suffix(string).unwrap();
let chars: Vec<_> = string.chars().collect();
let mut iter = suffix.iter();
assert_eq!(iter.next().unwrap(), (&chars[..4], 2)); // abra
assert_eq!(iter.next(), Some((&chars[..1], 5))); // a
assert_eq!(iter.next(), Some((&chars[1..4], 2))); // bra
assert_eq!(iter.next(), Some((&chars[2..4], 2))); // ra
assert_eq!(iter.next(), Some((&chars[..0], 11))); // ''
assert_eq!(iter.next(), None);
Commit count: 37

cargo fmt