crypto-cli-tools

A collection of CLI tools to process cryptocurrency data.

Install: RUSTFLAGS="-C target-cpu=native" cargo install crypto-cli-tools

crypto-daily-merger

If you have deployed carbonbot to multiple geographical locations, then you need to use this CLI tool to merge and deduplicate data from multiple carbonbots.

Usage: crypto-daily-merger <yyyy-MM-dd> <input_dirs(comma separated)> <output_dir>.

crypto-msg-parser

Parse raw strings from crypto-crawler or re-parse messages generated by crypto-msg-parser. Output messages are sorted by timestamp and deduplicated.

Usage: crypto-msg-parser <input_file> <output_file> [yyyy-MM].

If month is specified, only messages of the month will be kept.

csv-to-proto

This CLI tool converts .csv.xz files to .proto.xz files.

The CSV messages are generated by to_csv_string() functions in crypto-message/src/lib.rs.

crypto-daily-processor(DEPRECATED)

Usage: crypto-daily-processor <exchange> <msg_type> <market_type> <day> <input_dir> <output_dir_raw> <output_dir_parsed>

For example:

crypto-daily-processor bitstamp trade spot 2021-09-02 /mnt/dpool/download /mnt/dpool/daily-raw /mnt/dpool/daily-parsed

This CLI tool is replaced by crypto-daily-merger + crypto-msg-parser.