# JSON Remix

My small contribution to the _Rewrite it in Rust!_ ~bandwagon~ movement. The original was written in [Typescript](https://github.com/vimcommando/json-remix)

## About

This is a Rust command-line interface (CLI) tool that provides four commands for manipulating JSON and NDJSON files: `split`, `merge`, `bundle`, and `unbundle`.

Each command can accept input or output from files, directories, or from standard input/output wherever relevant.

After being rewritten in Rust it is roughly 2x as fast as the Typescript implementation and can handle files larger than 512MB.

### Installation

This is published to [crates.io](https://crates.io/crates/jsrmx) so you can simply do a global install with:

```sh
cargo install jsrmx
```

Then `jsrmx` is executable from your shell

```sh
jsrmx --help
```

## Usage

There are four commands:

1. `merge` - merges multiple JSON objects into a single large JSON object
2. `split` - splits a single JSON object into multiple JSON objects by top-level keys
3. `bundle` - bundles multiple JSON objects ito an NDJSON (newline-delimited JSON) series
4. `unbundle` - unbundles an NDJSON series into a collection of separate JSON objects

### merge

```sh
jsrmx merge <dir> [output]
```

#### Arguments

- `<dir>` - Required input directory
- `[output]` - Optional output file name (default `-` for stdout)

#### Options

- `-c`, `--compact` - Compact single-line output objects
- `-f`, `--filter` - regular expression to filter output keys
- `-p`, `--pretty` - Pretty-print output objects (default)
- `-t`, `--trim` - File extension to trim from object key names

#### Examples

Given a directory named `letters` with six files:

```
letters/alpha.json
letters/bravo.json
letters/charlie.json
letters/delta.json
letters/echo.json
letters/foxtrot.json
```

Where each file contains a few properties:

```jsonc
// cat alpha.json
{
  "uppercase": "A",
  "lowercase": "a",
  "position": 1
}
```

We can `merge` all the files into a single file:

```sh
jsrmx merge letters/ letters.json
```

So the contents of `letters.json` looks like:

```jsonc
// cat letters.json
{
  "alpha": {
    "lowercase": "a",
    "position": 1,
    "uppercase": "A"
  },
  "bravo": {
    "lowercase": "b",
    "position": 2,
    "uppercase": "B"
  },
  "charlie": {
    "lowercase": "c",
    "position": 3,
    "uppercase": "C"
  },
  "delta": {
    "lowercase": "d",
    "position": 4,
    "uppercase": "D"
  },
  "echo": {
    "lowercase": "e",
    "position": 5,
    "uppercase": "E"
  },
  "foxtrot": {
    "lowercase": "f",
    "position": 6,
    "uppercase": "F"
  }
}
```

Note the keys get sorted and have the `.json` extension trimmed from their names.

### split

```sh
jsrmx split [input] [output]
```

#### Arguments

- `[input]` - Optional input file name or `-` for `stdin` (default `-`)
- `[output]` - Optional output directory or `-` for `stdout`(default `-`)

#### Options

- `-c`, `--compact` - Compact single-line output objects
- `-f`, `--filter` - regular expression to filter output keys
- `-p`, `--pretty` - Pretty-print output objects (default)

#### Examples

We can split one file (or object through `stdin`) into individually-named files:

```sh
jsrmx split letters.json letters/
```

Given a the following single-object JSON file:

```jsonc
{
  "alpha": {
    "uppercase": "A",
    "lowercase": "a",
    "position": 1
  },
  "bravo": {
    "uppercase": "B",
    "lowercase": "b",
    "position": 2
  },
  // ... 3 entries omitted
  "foxtrot": {
    "uppercase": "F",
    "lowercase": "f",
    "position": 6
  }
}
```

The output files created will be:

```
letters/alpha.json
letters/bravo.json
letters/charlie.json
letters/delta.json
letters/echo.json
letters/foxtrot.json
```

Where each file contents will be the value from the large JSON:

```jsonc
// cat alpha.json
{
  "uppercase": "A",
  "lowercase": "a",
  "position": 1
}
```

If output to `stdout` the object will keep the top-level key as a parent object. Using `--filter` can extract specific keys.

```sh
jsrmx split --filter delta big_object.json -
```

```jsonc
{
  "delta": {
    "uppercase": "D",
    "lowercase": "d",
    "position": 4
  }
}
```

Combined with `--compact` this can convert a large object into an `.ndjson` file.

```sh
jsrmx split --compact big_object.json > letters.ndjson
```

```jsonc
// cat letters.ndjson
{"foxtrot":{"lowercase":"f","position":6,"uppercase":"F"}}
{"bravo":{"lowercase":"b","position":2,"uppercase":"B"}}
{"charlie":{"lowercase":"c","position":3,"uppercase":"C"}}
{"delta":{"lowercase":"d","position":4,"uppercase":"D"}}
{"echo":{"lowercase":"e","position":5,"uppercase":"E"}}
{"alpha":{"lowercase":"a","position":1,"uppercase":"A"}}
```

### bundle

```sh
jsrmx bundle <dir> [output]
```

#### Arguments

- `<dir>` - Required target input directory
- `[output]` - Optional output filename or `-` for stdout (default `-`)

#### Options

- `-e`, `--escape` -  List of field path to convert from nested JSON to an escaped string

#### Examples

We can convert a directory of `.json` files into a single `.ndjson` (newline-delimited JSON) file:

```sh
jsrmx bundle letters/ letters.ndjson
```

Given the files in the input directory:

```
letters/alpha.json
letters/bravo.json
letters/charlie.json
letters/delta.json
letters/echo.json
letters/foxtrot.json
```

With each file containing:

```jsonc
// cat alpha.json
{
  "letter": {
    "lowercase": "a",
    "uppercase": "A"
  },
  "name": "alpha",
  "position": 1
}
```

The output `letters.ndjson` will contain:

```jsonc
{"name":"alpha","letter":{"uppercase":"A","lowercase":"a"},"position":1}
{"name":"bravo","letter":{"uppercase":"B","lowercase":"b"},"position":2}
{"name":"charlie","letter":{"uppercase":"C","lowercase":"c"},"position":3}
{"name":"delta","letter":{"uppercase":"D","lowercase":"d"},"position":4}
{"name":"echo","letter":{"uppercase":"E","lowercase":"e"},"position":5}
{"name":"foxtrot","letter":{"uppercase":"F","lowercase":"f"},"position":6}
```

> NOTE: the filenames are not retained when bundling `.ndjson` files.

### unbundle

```sh
jsrmx unbundle [options] [intput] [output]
```

#### Arguments

- `[input]` - Optional input file name (default `-` for stdin)
- `[output]` - Optional output directory name (default `-` for stdout)

#### Options

- `-c`, `--compact` - Compact single-line output objects
- `-n`, `--name` - A list of JSON paths to use for filenames (uses first non-null)
- `-p`, `--pretty` - Pretty-print output objects (default)
- `-t`, `--type` - A JSON path to use for filename suffix (before extension)
- `-u`, `--unescape` - List of field paths to convert from escaped string to nested JSON

#### Example

Unbundling a file (or `stdin`) to a directory (or `stdout`):

```sh
jsrmx unbundle letters.ndjson letters/
```

Given the file `letters.ndjson`:

```jsonc
{"name":"alpha","letter":{"uppercase":"A","lowercase":"a"},"position":1}
{"name":"bravo","letter":{"uppercase":"B","lowercase":"b"},"position":2}
{"name":"charlie","letter":{"uppercase":"C","lowercase":"c"},"position":3}
{"name":"delta","letter":{"uppercase":"D","lowercase":"d"},"position":4}
{"name":"echo","letter":{"uppercase":"E","lowercase":"e"},"position":5}
{"name":"foxtrot","letter":{"uppercase":"F","lowercase":"f"},"position":6}
```

Unbundling with:

```sh
jsrmx unbundle letters.ndjson letters/
```

Will create these files:

```sh
letters/object-000001.json
letters/object-000002.json
letters/object-000003.json
letters/object-000004.json
letters/object-000005.json
letters/object-000006.json
```

The contents of each file will be pretty-printed by default:

```jsonc
// cat letters/object-000001.json
{
  "name": "alpha",
  "letter": {
    "uppercase": "A",
    "lowercase": "a"
  },
  "position": 1
}
```

Using the `--compact` option we can keep them as single-line entries:

```sh
jsrmx unbundle --compact letters.ndjson letters/
```

```jsonc
// cat letter/object-000001.json
{"name":"alpha","letter":{"uppercase":"A","lowercase":"a"},"position": 1}
```

For descriptive filenames, use the `--name` option. For a filename made of `${name}.json` run:

```sh
jsrmx unbundle --name=name letters.ndjson letters/
```

Will output `${name}.json` filenames:

```
letters/alpha.json
letters/bravo.json
letters/charlie.json
letters/delta.json
letters/echo.json
letters/foxtrot.json
```

Name values will work on nested values as long as the JSON path is `.` delimited. Periods in the key names will not resolve properly.

```sh
jsrmx unbundle --name=letter.lowercase letters.ndjson letters/
```

Will output `${letter.lowercase}.json` filenames:

```
letters/a.json
letters/b.json
letters/c.json
letters/d.json
letters/e.json
letters/f.json
```