# bert_tokenizer

\[ [API doc](https://docs.rs/bert_tokenizer/) | [crates.io](https://crates.io/crates/bert_tokenizer/) \]

The crate provides the port of the original BERT tokenizer from the [Google BERT repository](https://github.com/google-research/bert).

## License

MIT license. See [LICENSE](LICENSE) file for full license.