# Sidex

_Sidex_ is a format- and language-agnostic data structure and API definition language with a focus on simplicity, extensibility, and developer ergonomics. Sidex aims to simplify data exchange between different programming languages and platforms via potentially multiple serialization formats.

🚧 **Status:** Although we already use Sidex in production, it is still **experimental**. Use at your own risk!

## ✨ Features

- **Schema-first** definition of data structures and RPC-like APIs.
- Designed for **format- and language-agnostic** definitions.
- Modern **algebraic data types** and **non-null** by default.
- **Extensible** with user-defined opaque types.
- Designed for **interoperability**, e.g., with [JSON Schema](https://json-schema.org/).
- VS Code extension for increased productivity.
- Out-of-the-box support for [Rust](https://rust-lang.org), [TypeScript](https://www.typescriptlang.org), and [JSON](https://www.ecma-international.org/publications-and-standards/standards/ecma-404/).

## 🚀 Getting Started

Sidex is currently distributed via [Cargo](https://doc.rust-lang.org/cargo/) and [crates.io](https://crates.io/). To install Sidex run:

```plain
cargo install sidex-cli
```

Then, to create a new Sidex definition named `my_def` run:

```plain
sidex new my_def
```

Every Sidex definition consists of a flat collection of _modules_ located in the `modules` directory. Here is a simple example of a module you could place in the file `person.sidex`:

```plain
opaque Uuid  // This is an opaque user-defined type.

alias PersonId: Uuid  // This is a type alias.

enum Role {
    Admin,
    User,
}

struct Person {
    id: PersonId,
    name: string,
    email?: string,  // This field is optional.
    role: Role,
    children: [PersonId],  // A sequence of person ids.
}

enum GetPersonResult {
    NotFound,
    Found: Person,
}

fun get_person_by_id(id: PersonId) -> GetPersonResult
```

To check a definition for validity run:

```plain
sidex check
```

Please have a look at the [recipes](https://github.com/silitics/sidex/recipes) for further examples on how to use Sidex.

## ⚙️ The Sidex Language

At the core of Sidex is the Sidex language for defining data types and function types.

The core of Sidex is only concerned with such types and nothing else.

### 📦 Data Types

Sidex is based on five kinds of _data types_:

- _Opaque types_ are opaque to Sidex, i.e., their internal structure is a black box.

  Opaque types are defined with the `opaque` keyword. Opaque types are _nominal_, i.e., opaque types defined separately are always distinct even if they have the same name.

- _Enumeration types_ define unions with tagged _variants_ of different types.

  Enumeration types are defined with the `enum` keyword. Enumeration types are nominal, i.e., enumeration types defined separately are always distinct even if they agree on all their variants.

- _Struct types_ define structures with labeled _fields_ of different types.

  Struct types are defined with the `struct` keyword. Struct types are nominal, i.e., struct types defined separately are always distinct even if they agree on all their fields.

- _Sequence types_ define sequences of _elements_ of the same type.

  Sequence types are created with `[T]` where `T` denotes the _element type_. Sequence types are _structural_, i.e., two sequence types with the same element type are identical.

- _Map types_ define mappings from _keys_ of some type to _values_ of some type.

  Map types are created with `[K: V]` where `K` denotes the _key type_ and `V` denotes the _value type_. Map types are structural, i.e., two map types with the same key and value type are identical.

Sidex comes with built-in _primitive types_ for strings, integers, and booleans.
Technically, these primitive types are not any different from user-defined opaque types. These primitive types are:

- `string`: For sequences of Unicode code points.
- `i8`, `i16`, `i32`, `i64`: For signed integers of different bit width.
- `u8`, `u16`, `u32`, `u64`: For unsigned integers of different bit width.
- `bool`: For booleans.

In addition there is the `void` type for indicating the absence of any data.

Using opaque types, you can define your own primitives, e.g., for UUIDs:

```plain
opaque Uuid
```

The structure of opaques types can be specified externally, e.g., using [JSON Schema](https://json-schema.org/).

### 📡 Function Types

Taking inspiration from RPC and FFI, Sidex allows defining function types with the `fun` keyword. Every function type consists of a sequence of named _arguments_ with their own respective type and a _return type_. At its core, Sidex does not presuppose any protocol or other mechanism for invoking such functions.

## 🤝 Exchanging Data

Data exchange can be quite complex and involves multiple concerns which Sidex aims to separate.

### 📜 Language Mapping

To be useful, Sidex definitions need to be _mapped_ to type or class definitions of some programming language, e.g., [Rust](https://rust-lang.org) or [TypeScript](https://www.typescriptlang.org). We refer to such a mapping as a _language mapping_:

```plain
┌──────────────────┐   Language Mapping    ┌─────────────────┐
│ Sidex Definition │ ────────────────────► │ Target Language │
└──────────────────┘                       └─────────────────┘
```

Note that a language mapping might involve certain tradeoffs to be made. For instance, in case of TypeScript, a map type can be mapped either to `Object` or to `Map`, and, in case of Rust, there are also multiple different types of maps available, e.g., `HashMap` or `BTreeMap`. Furthermore, depending on the language, certain data types may not be mappable at all due to language-specific constraints.

Hence, the goal of the Sidex project is to provide tools and infrastructure for mapping Sidex definitions to different programming languages _without_ imposing any particular mapping. Using the [`sidex`](https://crates.io/crates/sidex) crate as a basis, you can define your own mappings and even generate additional boilerplate such as constructors and getters. If something cannot be sensibly mapped, a tool is free to generate an error as a last resort.

Sidex aims to provide mappings for some languages out-of-the-box with sane defaults.

Note that a language mapping is itself completely independent from how data may be serialized and how functions may be invoked. It can also be useful without ever exchanging any data.

### 📩 Serialization Formats

To exchange data between different languages, it needs to be _serialized_ into some common format. To this end, a _format mapping_ from a Sidex definition to the serialization format is necessary:

```plain
┌──────────────────┐   Format Mapping    ┌──────────────────────┐
│ Sidex Definition │ ──────────────────► │ Serialization Format │
└──────────────────┘                     └──────────────────────┘
```

Note that the format mapping is supposed to be language-independent. It merely describes how certain Sidex types are mapped to the serialization format and its types.

Again, Sidex does not impose any restrictions on the serialization format, however, it aims to provide some out-of-the-box mappings to common formats with sane defaults.

For user-defined opaque types, specific format mappings have to be provided.

### 🔗 Serialization Binding

Once we have fixed a language mapping and a format mapping, we need to bind both together using a _serialization binding_. A serialization binding is language-specific and format-specific. It takes serialized data as per the format mapping and transforms it into data structures as per the language mapping (known as _deserialization_) and vice versa (known as _serialization_).

## 🤔 Rationale

**Why schema-first?**

A schema-first approach has multiple advantages over definitions in a programming language: (1) It allows focusing on the important aspects of the data being exchanged. (2) It allows developing tooling independent of any programming language. (3) It enables the independent evolution and adaption of the definition language. (4) It can be used independently of a particular programming language.

**Why yet another language?**

Existing approaches are often specific to certain serialization formats, do not explicitly support algebraic data types, do not support arbitrary user-defined opaque types, have nullable fields by default, or/and are overly complex by supporting much more structures/types than Sidex.

## ⚖️ Licensing

Sidex is licensed under [MIT](https://github.com/silitics/sidex/blob/main/LICENSE). Unless you explicitly state otherwise, any contributions intentionally submitted for inclusion in this project shall be licensed under MIT without any additional terms or conditions.

---

Made with ❤️ by [Silitics](https://www.silitics.com).