s3grep
| Crates.io | s3grep |
| lib.rs | s3grep |
| version | 0.1.3 |
| created_at | 2025-07-07 19:35:13.127349+00 |
| updated_at | 2025-07-08 01:09:27.311541+00 |
| description | A fast, parallel grep tool for searching logs and unstructured content in AWS S3 buckets. |
| homepage | https://github.com/dacort/s3grep |
| repository | https://github.com/dacort/s3grep |
| max_upload_size | |
| id | 1741673 |
| size | 92,182 |
Damon P. Cortesi (dacort)
documentation
README
s3grep - Fast Parallel grep for S3
I use grep almost daily, but often deal with unstructured content and logs on S3...so I wrote an easy way to grep S3!
s3grep is a parallel CLI tool for searching logs and unstructured content in Amazon S3 buckets. It supports .gz decompression, progress bars, and robust error handlingâmaking it ideal for cloud-native log analysis.
Features
- Parallel, concurrent search across S3 objects
- Supports plain text and
.gzcompressed files - Progress bars for files and bytes processed
- Case-sensitive and insensitive search
- Line number output option
- Graceful handling of binary files and decompression errors
- Colorized match highlighting
Installation
Pre-built Binaries
For now, just git clone and cargo build. đ
Pre-built binaries (will be) available on the Releases page.
Usage
s3grep --pattern "ERROR" --bucket my-logs-bucket --prefix logs/ --concurrent-tasks 16
CLI Options
| Flag | Description |
|---|---|
-p, --pattern |
Search pattern (required) |
-b, --bucket |
S3 bucket name (required) |
-z, --prefix |
S3 prefix to search in (default: "") |
-c, --concurrent-tasks |
Number of concurrent tasks (default: 8) |
-i, --case-sensitive |
Case sensitive search |
-q, --quiet |
Hide progress bar |
-n, --line-number |
Show line numbers in output |
Example
s3grep --pattern "timeout" --bucket my-bucket --prefix logs/2025/06/ --concurrent-tasks 12 --line-number
Testing
Integration tests use Localstack to mock S3. See CONTRIBUTING.md for details.
Contributing
Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
License
This project is licensed under the MIT License.
Acknowledgments
- Inspired by daily use of
grepand the need for cloud-native log search. - Built with Rust, aws-sdk-rust, and Localstack for testing.