xs (cross-stream)

xs is an event stream store for personal, local-first use. Think of it like sqlite, but specializing in the event sourcing use case.

The focus is on fun and playfulness. Event sourcing provides an immediate connection to what you're creating, making the process feel alive. xs encourages experimentation, allowing you to make messes and explore freely—then gives you tools to organize and make sense of it all.

"You don't so much run it, as poke at it."

Come hang out and play

Installation

You can install the tool with:

cargo install cross-stream

or

brew install cablehead/tap/cross-stream

Usage

Usage: xs <COMMAND>

Commands:
  serve   Provides an API to interact with a local store
  cat     `cat` the event stream
  append  Append an event to the stream
  cas     Retrieve content from Content-Addressable Storage
  remove  Remove an item from the stream
  head    Get the head frame for a topic
  get     Get a frame by ID
  help    Print this message or the help of the given subcommand(s)

Unlike sqlite, which operates directly on the file system, xs requires a running process to manage access to the local store. This enables features like subscribing to real-time updates from the event stream.

% xs serve ./store
11:27:54.464 9zalp xs.start

Basics

Note: xs is designed to be orchestrated with Nushell, but since many are more familiar with bash, here are the very basics that work just fine from bash.

To append items to the stream, use:

% xs append ./store <topic>

The content for the event can be provided via stdin and, if present, will be stored in Content-Addressable Storage (CAS). You can also append events without content. Additionally, you can attach arbitrary metadata to an event using the --meta flag, which accepts metadata in JSON format.

For example:

% echo "content" | xs append ./store my-topic --meta '{"type": "text/plain"}' | jq
{
  "topic": "my-topic",
  "id": "03cq29mdmmkfze8p1plry4maj",
  "hash": "sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M=",
  "meta": {
    "type": "text/plain"
  },
  "ttl": "forever"
}

To fetch the contents of the stream, use the cat command:

% xs cat ./store/ | jq
{
  "topic": "xs.start",
  "id": "03cq29gqsg8ijbkob4krv93k3",
  "hash": null,
  "meta": {
    "expose": null
  },
  "ttl": null
}
{
  "topic": "my-topic",
  "id": "03cq29mdmmkfze8p1plry4maj",
  "hash": "sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M=",
  "meta": {
    "type": "text/plain"
  },
  "ttl": "forever"
}

xs generates a few meta events, such as xs.start, which is emitted whenever the process managing the store starts.

You can also see the my-topic event we just appended, along with a hash, which represents the hash of the stored content.

You can retrieve this content from the Content-Addressable Storage (CAS) using:

% xs cas ./store/ sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M=
content

To append another event to my-topic, you can run:

% echo "more content" | xs append ./store my-topic --meta '{"type": "text/plain"}' | jq
{
  "topic": "my-topic",
  "id": "03cq29ul7bhxrcaeh2ssrvcw1",
  "hash": "sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=",
  "meta": {
    "type": "text/plain"
  },
  "ttl": "forever"
}

Now, to quickly access the most recent event associated with my-topic, you can use the head command:

% xs head ./store/ my-topic | jq
{
  "topic": "my-topic",
  "id": "03cq29ul7bhxrcaeh2ssrvcw1",
  "hash": "sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=",
  "meta": {
    "type": "text/plain"
  },
  "ttl": "forever"
}

The head command retrieves the latest event (or "head") for a specific topic. If you have multiple events under the same topic, head will always return the latest one.

To get the content of the latest version:

% xs head ./store/ my-topic | jq -r .hash | xargs xs cas ./store/
more content

To retrieve a specific event by its ID, use the get command.

For example, to get the event with ID 03clswrgmmkkoqnotna38ldvl:

% xs get ./store/ 03clswrgmmkkoqnotna38ldvl | jq
{
  "topic": "my-topic",
  "id": "03cq29ul7bhxrcaeh2ssrvcw1",
  "hash": "sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=",
  "meta": {
    "type": "text/plain"
  },
  "ttl": "forever"
}

The basics with Nushell

Here's how the previous basics example looks using Nushell. To get started, run the following module import:

$ use xs.nu *

This will add some .command conveniences to your session. The commands default to working with a ./store in your current directory. You can customize this by setting $env.XSPWD.

Appending looks like this:

$ "content" | .append my-topic --meta {type: "text/plain"}
───────┬─────────────────────────────────────────────────────
 topic │ my-topic
 id    │ 03cq29mdmmkfze8p1plry4maj
 hash  │ sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M=
       │ ──────┬────────────
 meta  │  type │ text/plain
       │ ──────┴────────────
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

To .cat the stream:

$ .cat
─#─┬──topic───┬────────────id─────────────┬────────────────────────hash─────────────────────────┬────────meta─────────┬───ttl───
 0 │ xs.start │ 03cq29gqsg8ijbkob4krv93k3 │                                                     │ ────────┬──         │
   │          │                           │                                                     │  expose │           │
   │          │                           │                                                     │ ────────┴──         │
 1 │ my-topic │ 03cq29mdmmkfze8p1plry4maj │ sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M= │ ──────┬──────────── │ forever
   │          │                           │                                                     │  type │ text/plain  │
   │          │                           │                                                     │ ──────┴──────────── │
───┴──────────┴───────────────────────────┴─────────────────────────────────────────────────────┴─────────────────────┴─────────

We have the full expressiveness of Nushell—for example, we can get the content hash of the last frame on the stream using:

$ .cat | last | $in.hash
sha256-7XACtDnprIRfIjV9giusFERzD722AW0+yUMil7nsn3M=

And then use the .cas command to retrieve the content:

$ .cat | last | .cas $in.hash
content

We can also retrieve the content from a frame by piping it directly to .cas:

$ .cat | last | .cas
content

Continuing the basic example, we append an additional my-topic frame:

$ "more content" | .append my-topic --meta {type: "text/plain"}
───────┬─────────────────────────────────────────────────────
 topic │ my-topic
 id    │ 03cq29ul7bhxrcaeh2ssrvcw1
 hash  │ sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=
       │ ──────┬────────────
 meta  │  type │ text/plain
       │ ──────┴────────────
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

And use .head to retrieve the latest version:

$ .head my-topic
───────┬─────────────────────────────────────────────────────
 topic │ my-topic
 id    │ 03cq29ul7bhxrcaeh2ssrvcw1
 hash  │ sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=
       │ ──────┬────────────
 meta  │  type │ text/plain
       │ ──────┴────────────
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

To get the content of the latest version:

$ .head my-topic | .cas
more content

Finally, we have the .get command:

$ .get 03cq29ul7bhxrcaeh2ssrvcw1
───────┬─────────────────────────────────────────────────────
 topic │ my-topic
 id    │ 03cq29ul7bhxrcaeh2ssrvcw1
 hash  │ sha256-LCMWc3yTE5Vt/ACD2joqYs4ln2ZITz4mRA8NGwLdQSg=
       │ ──────┬────────────
 meta  │  type │ text/plain
       │ ──────┴────────────
 ttl   │ forever
───────┴─────────────────────────────────────────────────────

Built with 🙏💚

• fjall: for indexing and metadata
• cacache: for content (CAS)
• hyper: provides an HTTP/1.1 API over a local Unix domain socket for subscriptions, etc.
• Nushell: for scripting and interop