# crypto-cli-tools

A collection of CLI tools to process cryptocurrency data.

Install: `RUSTFLAGS="-C target-cpu=native" cargo install crypto-cli-tools`

## crypto-daily-merger

If you have deployed [carbonbot](https://github.com/crypto-crawler/carbonbot) to multiple geographical locations, then you need to use this CLI tool to merge and deduplicate data from multiple carbonbots.

Usage: `crypto-daily-merger <yyyy-MM-dd> <input_dirs(comma separated)> <output_dir>`.

## crypto-msg-parser

Parse raw strings from `crypto-crawler` or re-parse messages generated by `crypto-msg-parser`. Output messages are sorted by timestamp and deduplicated.

Usage: `crypto-msg-parser <input_file> <output_file> [yyyy-MM]`.

If month is specified, only messages of the month will be kept.

## csv-to-proto

This CLI tool converts `.csv.xz` files to `.proto.xz` files.

The CSV messages are generated by `to_csv_string()` functions in [crypto-message/src/lib.rs](https://github.com/crypto-crawler/crypto-crawler-rs/blob/main/crypto-message/src/lib.rs).

## crypto-daily-processor(DEPRECATED)

Usage: `crypto-daily-processor <exchange> <msg_type> <market_type> <day> <input_dir> <output_dir_raw> <output_dir_parsed>`

For example:

```bash
crypto-daily-processor bitstamp trade spot 2021-09-02 /mnt/dpool/download /mnt/dpool/daily-raw /mnt/dpool/daily-parsed
```

This CLI tool is replaced by `crypto-daily-merger` + `crypto-msg-parser`.