## Specification and Bindings for Swift Binary Protocol

<!-- toc -->

- [Installing sbp2json, json2sbp, json2json and related tools](#installing-sbp2json-json2sbp-json2json-and-related-tools)
- [Building / installing](#building--installing)
  * [Using Docker](#using-docker)
    + [Fetching the prebuilt image from DockerHub](#fetching-the-prebuilt-image-from-dockerhub)
    + [Creating your own image](#creating-your-own-image)
    + [Using the docker image](#using-the-docker-image)
  * [Installing from package managers](#installing-from-package-managers)
  * [Installing development Python versions](#installing-development-python-versions)
  * [Adding development version as a pip dependency](#adding-development-version-as-a-pip-dependency)
  * [Installing from source](#installing-from-source)
- [SBP Development Procedures](#sbp-development-procedures)  
- [SBP Protocol Specification](#sbp-protocol-specification)
- [JSON Schema Definitions](#json-schema-definitions)
- [Kaitai Struct Format Descriptions](#kaitai-struct-format-descriptions)
- [LICENSE](#license)

<!-- tocstop -->

The Swift Navigation Binary Protocol (SBP) is a fast, simple, and minimal
binary protocol for communicating with Swift devices. It is the native binary
protocol used by the [Piksi GPS receiver](http://swiftnav.com/piksi.html) to
transmit solutions, observations, status and debugging messages, as well as
receive messages from the host operating system, such as differential
corrections and the almanac.

This project provides language-agnostic specification and documentation for
messages used with SBP, a compiler for generating message bindings, and client
libraries in a variety of languages. This repository is organized into the
following directory structure:

* [`docs`](./docs): Protocol documentation and message definitions.
* [`spec`](./spec): Machine readable protocol specification in
  [YAML](http://en.wikipedia.org/wiki/YAML).
* [`generator`](./generator): Simple, template-based generator for
  different languages.
* [`python`](./python): Python client and examples.
* [`c`](./c): C client library and examples.
* [`haskell`](./haskell): Haskell client and examples.
* [`java`](./java): Java client library and examples.
* [`javascript`](./javascript): JavaScript client library and examples.
* [`rust`](./rust): Rust client library and examples.
* [`sbpjson`](./sbpjson): Tools for parsing SBP-JSON.
* [`kaitai`](./kaitai): Kaitai Struct Format Description and generated code.

Except for the `generator`, all of the above are generated and should not be modified directly.

## Installing sbp2json, json2sbp, json2json and related tools

This repository also provides the following utilities for comprehending and
inspecting SBP data:

- `sbp2json`: converts SBP binary into a JSON representation, in which field
  names and values are expanded into JSON objects, common fields such as
  "message type" and "payload" are included as well.

- `json2sbp`: uses the "message type", "payload" and related fields from an SBP
  JSON stream to reconstruct the binary representation.

- `json2json`: some tools (notably the Swift GUI Console) produce abbreviated
  JSON logs with only common fields such as "message type" and "payload", the
  `json2json` tool expands these JSON objects to include fields specific the
  individual message.

To install a released version of these tools, visit the [releases
page](https://github.com/swift-nav/libsbp/releases) and download an archive for
your platform.

To install from source, you can use Rust's cargo tool (first [install
Rust](https://www.rust-lang.org/tools/install)), then run:

```
cargo install --git https://github.com/swift-nav/libsbp.git --bins
```

There's also a [Haskell version](./haskell) available which can be installed by
running `stack install` in the [./haskell](./haskell) directory of a checkout
of this repo (after [installing
stack](https://docs.haskellstack.org/en/stable/README/)) or by visiting the
releases by and downloading an `sbp_tools_haskell` archive.  This variant of
the tools predate the Rust and Python versions, and also includes an `sbp2yaml`
tool as well as a `sbp2prettyjson` tool.

Finally, a Python version of the `sbp2json` tool exists, which is installable
on any platform that supports Python via pip, e.g.:

```
pip3 install sbp
```

The tool can then be invoked as follows:

```
python3 -m sbp2json <sbp.bin
```

The performance of the Python version is significantly slower than Rust and Haskell,
but works on all platforms that Python itself supports.

## Building / installing

Before you start, run 
```
git pull --tags
```
in you local libsbp repository to pull the tags. This will ensure the correct
version number is generated.

### Using Docker

Before you begin, make sure you have [Docker](https://docs.docker.com/docker-for-mac/install/) installed.
Start [Docker desktop](https://docs.docker.com/docker-for-mac/).

#### Fetching the prebuilt image from DockerHub

The quickest method to get going is to just pull a prebuilt copy from DockerHub
(no guarantees on freshness) by running the following on your command line:

    docker run --rm -v $PWD:/mnt/workspace -i -t swiftnav/libsbp-build:2023-12-19 /bin/bash

This will mount your local copy of the libsbp repository onto the image.

Check this [link](https://hub.docker.com/r/swiftnav/libsbp-build/tags) for newer tags.
Alternatively, you could run

    docker run --rm -v $PWD:/mnt/workspace -i -t swiftnav/libsbp-build:latest-master /bin/bash

if you are facing issues with compilation and the tags are out of date as well.

#### Creating your own image

Otherwise, the `Dockerfile` will create a docker image that contains all the
necessary dependencies to build libsbp.  You can make a local image fresh from
this file by running `docker build` as such:

    docker build -t libsbp-build - <Dockerfile

Reading the Dockerfile from STDIN prevents docker from pulling in the whole
repostory into the build context (which is then immediately discarded anyway).
You can customize the UID of the user that's created with the docker image
by passing the desired `UID` value to the build:

    docker build -t libsbp-build --build-arg UID=1234 - <Dockerfile

You can then make this image operate on your local workspace like this:

    docker run --rm -v $PWD:/mnt/workspace -i -t libsbp-build:latest /bin/bash

#### Using the docker image

Once in the image, simply type `make all` to generate all the libsbp bindings.
This could take several hours to run.  Alternately, the docker image will run
the `make all` command by default, so you can kick off the `make all` process
by simply running the following command:

    docker run --rm -v $PWD:/mnt/workspace -i -t libsbp-build:2023-12-19

To speed up this process you can attempt to run Python environment tests in
paralell with:

    docker run --rm -v $PWD:/mnt/workspace -i -t -e SBP_TOX_PARALLEL=auto libsbp-build:2023-12-19

When you are finished, quit Docker so that it would not unnecessarily use up
resources on your machine.

If you run into issues during the generation process, try running `make clean`.
Alternatively, you could recompile from a clean, newly-cloned libsbp repository
on your machine, which would minimize the chance of running into compilation
issues from an old build.

### Installing from package managers

Some bindings are available on package managers:

* [`python`](https://github.com/swift-nav/libsbp/tree/HEAD/python): available on pip
* [`haskell`](https://github.com/swift-nav/libsbp/tree/HEAD/haskell): available on Hackage
* [`javascript`](https://github.com/swift-nav/libsbp/tree/HEAD/javascript): available on NPM

### Installing development Python versions

To install the Python binding from source (using pip) run the following command:

```sh
pip install 'file:///path/to/libsbp#subdirectory=python'
```

Or via setuptools directly:
```sh
cd /path/to/libsbp
cd python
python setup.py
```

### Adding development version as a pip dependency

Run the following command:
```sh
pip install "git+https://github.com/swift-nav/libsbp@<GIT_REVISION>#egg=sbp&subdirectory=python"
```

Or add this to `requirements.txt`:
```
git+https://github.com/swift-nav/libsbp@<GIT_REVISION>#egg=sbp&subdirectory=python
```

### Installing from source
You can build one binding at a time or update all at once:

```
make python
```

or

```
make all
```

are both valid. To see a list of all valid targets, run `make help`.

**Python version notes:**
1. By default the Python targets `make python` and `make test-python` (as well
   as `make all`) run tests on all Python versions officially supported by *the
   libsbp Python bindings*, currently **3.6-3.9**, skipping any versions not
   installed. To run tests on just specific Python version(s), specify the
   `TOXENV` environment variable, e.g., `TOXENV=py37 make python`. Travis runs
   Python tests on all supported versions.
2. By default *the code generators* are run on the system's (or virtual env's)
   default Python interpreter. Currently Python versions **2.7, 3.5, and 3.7**
   are officially supported, other versions may or may not work. The generated
   libsbp bindings should be the same on all supported Python versions. To use
   a different version than your default Python interpreter, specify the
   `GENENV` environment variable, e.g., `GENENV=py27 make all` (you must have
   that version of Python installed beforehand).
3. To run both the generator and the Python tests on specific Python versions,
   specify both envs, e.g., `GENENV=py37 TOXENV=py37 make python`

## SBP Development Procedures

See the ["how to"](HOWTO.md) page for instructions on adding new messages,
updating the [documentation](docs/sbp.pdf), and releasing new versions of this
library.

## SBP Protocol Specification

SBP consists of two pieces: (i) an over-the-wire message framing format and
(ii) structured payload definitions. As of Version 1.0, the packet consists of
a 6-byte binary header section, a variable-sized payload field, and a 16-bit
CRC value. SBP uses the CCITT CRC16 (XMODEM implementation) for error
detection.

Please see [the docs](docs/sbp.pdf) for a full description of the packet
structure and the message types. Developer documentation for the
language-specific sbp libraries is [here](http://swift-nav.github.io/libsbp/).
Please refer to [the changelog](CHANGELOG.md) for more information about the
evolution of the library and its messages.

## JSON Schema Definitions

For web clients processing SBP in JSON form, JSON schema definitions are
provided.  Libraries for JavaScript, TypeScript, and Elm generated by the
[QuickType](https://github.com/quicktype/quicktype) tool are provided.  See the
HOWTO for instructions on updating these schemas.

## Kaitai Struct Format Descriptions

[Kaitai Struct](https://kaitai.io/) is a declarative language used to describe
binary data structures.  KSY files containing format specifications for all
supported SBP messages can be generated using the `kaitai` target.  The
resulting files can be found in the [`kaitai/ksy`](./kaitai/ksy) directory.
This target also runs the Kaitai Struct compiler to generate bindings for
perl and python.  These bindings (along with test cases) can be found in the
[`kaitai/perl`](./kaitai/perl) and [`kaitai/python`](./kaitai/python) subdirectories
respectively.

The Kaitai Struct format description files can also potentially be used to
generate bindings for the following targets:

* graphviz
* csharp
* rust
* java
* go
* cpp_stl
* php
* lua
* nim
* html
* ruby
* construct
* javascript

### Notes on python bindings

The python bindings generated by the Kaitai Struct compiler allow for
significantly faster parsing in comparison to the construct-based bindings
included in libsbp.  However, due to the nature of Kaitai Struct, these
bindings do not allow for serialisation (i.e. construction of SBP objects).
Serialisation for java and python targets is supported as an experimental
feature in the development version of Kaitai Struct (> 0.10) so this
restriction should hopefully be removed in the future.

It should also be noted that the names used by the KSY bindings are not 1:1
identical to the construct bindings due constraints which Kaitai Struct places
on various token types (e.g. it is forbidden to use upper case characters in
field names).  The test cases in the [`kaitai/ksy/tests`](./kaitai/ksy/tests) directory
(particularly those in `test_parsers.py`) can be used as a starting point for
working with these bindings, as well as demonstrating the minor differences
between the output generated by each set of bindings.

## LICENSE

Copyright © 2015-2023 Swift Navigation

Distributed under the [MIT open source license](LICENSE).