spider-lib
| Crates.io | spider-lib |
| lib.rs | spider-lib |
| version | 0.2.0-beta.1 |
| created_at | 2026-01-21 04:40:02.820122+00 |
| updated_at | 2026-01-24 12:02:50.966228+00 |
| description | A Rust-based web scraping framework inspired by Scrapy (Python). |
| homepage | https://github.com/mzyui/spider-lib |
| repository | https://github.com/mzyui/spider-lib |
| max_upload_size | |
| id | 2058321 |
| size | 127,860 |
(mzyui)
documentation
README
spider-lib 🕷️🕸️
spider-lib is an asynchronous web scraping library for Rust, inspired by Scrapy. It features a powerful, modular architecture designed for high-performance data fetching. Currently under active development.
Architecture
Leverages Rust's async capabilities for efficient I/O and parallel scraping. Employs a modular, actor-based design with core components: Downloader, Scheduler, and ItemPipeline for flexible workflow customization.
Quick Start
To begin, clone the repository and execute an example:
cargo run --example books
This command initiates a spider to collect book data from a sample website and exports it to a CSV file.
Contribution
For ideas or bug reports, please open an issue or submit a pull request.