# 0.1.1
- Added `AwarioSmartBot` to list of crawlers and fixed capitalization `DiffBot -> Diffbot`
- Added option to force crawlers to re-fetch the URL they are redirected to by adding a previously-unseen URL to the end
- Added fn to provide contents of `robots.txt` which will deny access to all "AI" crawlers

# 0.1.0
- Contained initial list of crawlers
- Provided service & layer to redirect them to URL of user's choice