callysto
| Crates.io | callysto |
| lib.rs | callysto |
| version | 0.1.11 |
| source | src |
| created_at | 2022-03-27 14:56:51.415535 |
| updated_at | 2024-01-23 20:06:45.271594 |
| description | Stream processing framework. |
| homepage | https://vertexclique.github.io/callysto/ |
| repository | https://github.com/vertexclique/callysto |
| max_upload_size | |
| id | 557293 |
| size | 165,981 |
Ankur Srivastava (ansrivas)
documentation
README
Callysto is stream processing framework for Rust with focus on performance and durability.
NOTE: Callysto is used in production environment of multiple companies and it is production ready. This readme will be updated with more information about usage of Callysto.
Requirements
- cmake
- clang
- libtool
- libstdc++-devel
Rust MSRV
We are using:
cargo 1.60.0 (d1fd9fe 2022-03-01)
Running
With Docker Compose - Nerdctl Compose
- You need to have docker installation locally. With containerd backend, if you want to bring single node Kafka replacement use:
nerdctl compose up
If you want to bring Confluent Kafka single node. You can:
nerdctl compose -f docker-compose.kafkasn.yml
If you want to bring full blown Confluent Kafka cluster. Use:
nerdctl compose -f docker-compose.kafkacluster.yml
Mind that nerdctl command is interchangeable with docker.
With K8S
- You need to have K8S installation locally.
cd k8s && kubectl apply -f . && kubectl port-forward svc/redpanda 9092:9092
- Now you can run the producers in Python.
virtualenv venv
source venv/bin/activate
which pip3 # check that it is pointing to venv
pip3 install aiokafka -U
python examples/producer.py
- Now you can spawn the double agent example (without durability).
RUST_LOG="info,rdkafka::client=warn" RUST_BACKTRACE=full cargo run --example double-agent
Environment variables passed above is optional, but suggested for development.