# Changelog

## [9.0.1] - 2024-02-07

- updated dependencies to latest versions
- attempt to improve performance by turning the `Tokenizer` into an iterator, performance worsened, so I reverted to the current implementation

## [9.0.0] - 2023-08-16

- replace hardcoded whitespace characters to configurable

## [10.0.0] - 2024-02-14

- `Duo` struct replacing `duos` macro
- `Tokenizer::next` is now `Tokenizer::seek`
- `Tokenizer::seek` returns `SeekResult`

## [11.0.0] - 2024-02-18

- `SeekResult` nuked
- `Tokenizer::seek` renamed to `Tokenizer::consume`
- `Tokenizer::consume` now returns `Result<Option<Token>>`
- prevent compiler from optimizing away benchmarks properly
- tests are now in separate files
- futurproofing tests by expect matching the whole `Token` struct
- `serialization` feature nuked
- fixed `Tokenizer::tokenize_all` not returning errors properly

## [12.0.0] - 2024-03-11

- implemented `Tokenizer::seek`
- renamed `Tokenizer::tokenize_all` to `Tokenizer::consume_all`

## [12.1.0] - 2024-03-12

- replace `lazy_static` with `OnceLock`, removing a dependency