morax-storage
| Crates.io | morax-storage |
| lib.rs | morax-storage |
| version | |
| source | src |
| created_at | 2024-10-07 21:53:18.845865 |
| updated_at | 2024-10-28 10:23:47.807433 |
| description | The Storage Service for Morax that interacts with the different object storage backends |
| homepage | |
| repository | https://github.com/tisonkun/morax/ |
| max_upload_size | |
| id | 1400501 |
| Cargo.toml error: | TOML parse error at line 18, column 1 | 18 | autolib = false | ^^^^^^^ unknown field `autolib`, expected one of `name`, `version`, `edition`, `authors`, `description`, `readme`, `license`, `repository`, `homepage`, `documentation`, `build`, `resolver`, `links`, `default-run`, `default_dash_run`, `rust-version`, `rust_dash_version`, `rust_version`, `license-file`, `license_dash_file`, `license_file`, `licenseFile`, `license_capital_file`, `forced-target`, `forced_dash_target`, `autobins`, `autotests`, `autoexamples`, `autobenches`, `publish`, `metadata`, `keywords`, `categories`, `exclude`, `include` |
| size | 0 |
tison (tisonkun)
documentation
README
Morax
Morax is aimed at providing message queue and data streaming functionality based on cloud native services:
- Meta service is backed by Postgres compatible relational database services (RDS, Aurora, etc.).
- Data storage is backed by S3 compatible object storage services (S3, MinIO, etc.).
Usage
Currently, Morax supports basic PubSub APIs for Apache Kafka and WAL Service. You can try it out with the following steps.
-
Start the environment that provides a Postgres instance and a MinIO instance:
docker compose -f ./dev/docker-compose.yml up -
Build the
moraxbinary:cargo x build -
Start the broker:
./target/debug/morax start --config-file ./dev/config.toml
Try out the Apache Kafka broker
Now, a Kafka broker is running at localhost:9092. You can use your favorite Kafka client to interact with it.
You can also get an impression of the interaction by reading the test cases in:
Try out the WAL Service broker
Also, a WAL broker is running at localhost:8848. You can talk to it with the morax-wal-client. The wire protocol is HTTP so that all the HTTP ecosystem is ready for use.
You can also get an impression of the interaction by reading the test cases in:
Design
To support multiple providing message queue and data streaming APIs, Morax is designed as a modular system:
- Common functionalities like logging, async runtime, and protos are shared;
- Interfaces of meta service and data storage are shared;
- Each protocol implements their own wire protocol and message format;
- Each protocol shares the basic topic metadata model, with optional additional specific properties;
- Each protocol shares the basic data storage model, the payload is protocol specific, with a common header;
- Thus, each protocol shares similar publishing/producing APIs;
- On the contrary, each protocol implements their own subscription and consumer group management.
License
This project is licensed under Apache License, Version 2.0.