rustfs-workers
| Crates.io | rustfs-workers |
| lib.rs | rustfs-workers |
| version | 0.0.3 |
| created_at | 2025-07-05 07:14:00.860274+00 |
| updated_at | 2025-07-05 07:14:00.860274+00 |
| description | Workers for RustFS, providing background processing capabilities for tasks such as data synchronization, indexing, and more. |
| homepage | https://rustfs.com |
| repository | https://github.com/rustfs/rustfs |
| max_upload_size | |
| id | 1738802 |
| size | 18,665 |
houseme (houseme)
documentation
README
RustFS Workers - Background Job Processing
Distributed background job processing system for RustFS object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS Workers provides a distributed background job processing system for the RustFS distributed object storage system. It handles asynchronous tasks such as data replication, cleanup, healing, indexing, and other maintenance operations across the cluster.
Note: This is a core submodule of RustFS that provides essential background processing capabilities for the distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
🔄 Job Processing
- Distributed Execution: Jobs run across multiple cluster nodes
- Priority Queues: Multiple priority levels for job scheduling
- Retry Logic: Automatic retry with exponential backoff
- Dead Letter Queue: Failed job isolation and analysis
🛠️ Built-in Workers
- Replication Worker: Data replication across nodes
- Cleanup Worker: Garbage collection and cleanup
- Healing Worker: Data integrity repair
- Indexing Worker: Metadata indexing and search
- Metrics Worker: Performance metrics collection
🚀 Scalability Features
- Horizontal Scaling: Add worker nodes dynamically
- Load Balancing: Intelligent job distribution
- Circuit Breaker: Prevent cascading failures
- Rate Limiting: Control resource consumption
🔧 Management & Monitoring
- Job Tracking: Real-time job status monitoring
- Health Checks: Worker health and availability
- Metrics Collection: Performance and throughput metrics
- Administrative Interface: Job management and control
📦 Installation
Add this to your Cargo.toml:
[dependencies]
rustfs-workers = "0.0.3"
🔧 Usage
📋 Requirements
- Rust: 1.70.0 or later
- Platforms: Linux, macOS, Windows
- Network: Cluster connectivity required
- Storage: Persistent queue storage recommended
🌍 Related Projects
This module is part of the RustFS ecosystem:
- RustFS Main - Core distributed storage system
- RustFS Common - Common types and utilities
- RustFS Lock - Distributed locking
📚 Documentation
For comprehensive documentation, visit:
🔗 Links
- Documentation - Complete RustFS manual
- Changelog - Release notes and updates
- GitHub Discussions - Community support
🤝 Contributing
We welcome contributions! Please see our Contributing Guide for details.
📄 License
Licensed under the Apache License, Version 2.0. See LICENSE for details.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with 🔄 by the RustFS Team