serde_rustler

serde_rustler provides a Serde Serializer and Deserializer for Rustler types, so you can easily serialize and deserialize native Rust types directly to and from native Elixir terms within your NIFs.

Installation

Install from Crates.io:

[dependencies]
serde_rustler = "0.0.3"

Quick Start

#[macro_use] extern crate rustler;

use serde::{Serialize, Deserialize}
use serde_rustler::{from_term, to_term};

rustler_export_nifs! {
    "Elixir.SerdeRustlerTests",
    [("nif", 1, nif)],
    None
}

#[derive(Serialize, Deserialize)]
struct Animal = { ... };

fn nif<'a>(env: Env<'a>, args: &[Term<'a>]) -> NifResult<Term<'a>> {
    // Deserialize term into a native Rust type.
    let animal: Animal = from_term(args[0])?;

    // Serialize a type into an Elixir term.
    to_term(env, animal).map_err(|err| err.into())
}

Usage

Below is a more comprehensive example of how you might use serde_rustler within a rust NIF...

#[macro_use]
extern crate rustler;

use rustler::{Env, error::Error as NifError, NifResult, Term};
use serde::{Serialize, Deserialize};
use serde_rustler::{from_term, to_term};

rustler_export_nifs! {
    "Elixir.SerdeNif",
    [("readme", 1, readme)],
    None
}

// NOTE: to serialize to the correct Elixir record, you MUST tell serde to
// rename the variants to the full Elixir record module atom.
#[derive(Debug, Serialize, Deserialize)]
enum AnimalType {
    #[serde(rename = "Elixir.SerdeNif.AnimalType.Cat")]
    Cat(String),
    #[serde(rename = "Elixir.SerdeNif.AnimalType.Dog")]
    Dog(String),
}

// NOTE: to serialize to an actual Elixir struct (rather than a just map with
// a :__struct__ key), you MUST tell serde to rename the struct to the full
// Elixir struct module atom.
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename = "Elixir.SerdeNif.Animal")]
struct Animal {
    #[serde(rename = "type")]
    _type: AnimalType,
    name: String,
    age: u8,
    owner: Option<String>,
}

fn readme<'a>(env: Env<'a>, args: &[Term<'a>]) -> NifResult<Term<'a>> {
    let animal: Animal = from_term(args[0])?;
    println!("serialized animal: {:?}", animal);
    to_term(env, animal).map_err(|err| err.into())
}

... and how you might structure your corresponding Elixir types (code structure, imports, aliases and requires simplified or omitted for brevity):

defmodule SerdeNif do
  use Rustler, otp_app: :serde_nif

  def readme(_term), do: :erlang.nif_error(:nif_not_loaded)

  defmodule Animal do
    @type t :: %Animal{
      type: Cat.t() | Dog.t(),
      name: bitstring,
      age: pos_integer,
      owner: nil | bitstring
    }
    defstruct type: Cat.record(), name: "", age: 0, owner: nil

    @doc """
    Deserializes term as a Rust `Animal` struct, then serializes it back into
    an Elixir `Animal` struct. Should return true.
    """
    def test() do
      animal = %Animal{
        type: Animal.Cat.record(),
        name: "Garfield",
        age: 41,
      }

      SerdeNif.readme(animal) == animal
    end
  end

  defmodule AnimalType.Cat do
    require Record
    @type t {__MODULE__, String.t()}
    Record.defrecord(:record, __MODULE__, breed: "tabby")
  end

  defmodule AnimalType.Dog do
    # omitted
  end
end

Conversion Table

1: API still being decided / implemented.

2: When serializing unknown input to terms, atoms will not be created and will instead be replaced with Elixir bitstrings. Therefore "records" will be tuples ({bitstring, ...}) and "structs" will be maps containing %{:__struct__ => bitstring}. The unfortunate consequence of this is that deserialize_any will lack the necessary information needed deserialize many terms without type hints, such as structs, enums and enum variants, and tuples. (Feedback on how best to solve this is very welcome here).

Benchmarks

To run:

cd serde_rustler_tests
MIX_ENV=bench mix run test/benchmarks.exs

Benchmarks were ripped from the Poison repo. The NIFs being called were implemented using serde-transcode to translate between serde_rustler and serde_json and were compiled in :release mode by rustler.

NOTE: If someone can point out any mistakes I made that led to these ridiculous results, please let me know :)

Benchmarks suggest that serde_rustler is somewhat faster than jiffy when encoding JSON, and generally comparable to / no more than ~2-3x as slow as jiffy or jason when decoding JSON, and in almost all cases, serde_rustler seems to use significantly less memory than pure-Elixir alternatives, though this is likely has to do with running a NIF rather than an pure-Elixir function.

Also take note of the results for any test taking longer than 1ms or tests involving the larger inputs govtrack.json (3.74 MB) and issue-90.json (7.75 MB) - the encode_json_compact and decode_json NIFs have significantly higher variation in performance while their dirty equivalents encode_json_compact_dirty and decode_json_dirty are comparable to the originals in speed and have more reliable performance.

TODO

• finalize behaviour around chars, charlists, iolists, map keys
• still getting used to Rust, so may need to improve error handling and ergnomoics around API
• support for i128 and u128
• more extensive (i.e. possible addition of smoke, property-based) testing
• investigate decode_json (Serializer?) performance degradation

Changelog

Contributing

1. Fork it https://github.com/your_username/serde_rustler/fork
2. Create your feature branch (git checkout -b feature/fooBar)
3. Commit your changes (git commit -am 'Add some fooBar')
4. Push to the branch (git push origin feature/fooBar)
5. Create a new Pull Request

Maintainers

• Sunny G - @sunny-g

License

MIT