# tokenizer

The implementation of eson's tokenizer