summavy-sstable

Crates.iosummavy-sstable
lib.rssummavy-sstable
version0.1.0
sourcesrc
created_at2023-01-12 10:17:57.710392
updated_at2023-01-12 10:17:57.710392
descriptionsstable implementation
homepage
repository
max_upload_size
id757024
size72,628
Pasha Podolsky (ppodolsky)

documentation

README

SSTable

The tantivy-sstable crate is yet another sstable crate.

It has been designed to be used in quickwit:

  • as an alternative to the default tantivy fst dictionary.
  • as a way to store the column index for dynamic fast fields.

The benefit compared to the fst crate is locality. Searching a key in the fst crate requires downloading the entire dictionary.

Once the sstable index is downloaded, running a get in the sstable crate only requires a single fetch.

Right now, the block index and the default block size have been thought for quickwit, and the performance of a get is very bad.

Sorted strings?

SSTable stands for Sorted String Table. Strings have to be insert in sorted order.

That sorted order is used in different ways:

  • it makes gets and streaming ranges of keys possible.
  • it allows incremental encoding of the keys
  • the front compression is leveraged to optimize the intersection with an automaton
Commit count: 0

cargo fmt