Truc

Rust code generator for safe, fixed size, evolving records.

Objectives

The objectives of this crate are:

• define fixed size Rust structures to represent records of data
• define strongly typed accessors to data
• allow record evolution: data replacement by other data of different type
• optimize memory copies when records are evolving
• entirely safe memory management, even on panic

Whether these objectives are met or not is still to be proven.

Why and how

Fixed size

Having fixed size records aims at optimizing allocations: it is easy to manage memory when millions of objects have all the same size.

This is achieved by leveraging the const generic Rust feature: the size of records is statically known.

Strong typing

Strong typing is a very interesting aspect of the Rust language. The only thing a developer has to do to have strong typing is to use types. Truc is made to allow the use of as many types as possible, even user-defined types. There may be restrictions, especially on lifetimes, but any owned type can be used.

Record evolution

It is often nice to make a record evolve, keeping only one invariant: its size.

This is achieved by defining record variants and ways to:

• destructure the data
• convert from one variant to the next one
• convert a list of records of one variant to a list of records of another variant

There is no direct way to convert from one variant to any arbitrary other variant (because use cases have to be defined), but the language allows almost anything provided it compiles.

Memory copies

Memory copies are expensive, one should try to avoid them as much as possible.

A datum in a record is always stored at the same location in that record. If it's not at the same location then it is not the same datum. With the help of the compiler, memory copies should be optimized.

To be proven:

• Can the compiler optimize the conversion from one variant to another? That sounds optimistic, but one can be surprised by the level of optimization of LLVM.
• Does it work on both the heap and the stack?

Safe memory management

Code generated by Truc is made to be entirely safe. It is built on top of some heavily unsafe calls but the API only exposes safe functions:

• the types size and alignment is respected (otherwise the code would panic)
• everything held by a record is safely managed: dropping a record drops the data it holds
• the vector conversion is safe even if the conversion panics in the middle of the loop: old data and new data is dropped according to the type system rules, no value is missed

Additional cool features

• cross-compilation enabled

Getting started

Project organization

Usually a project is organized this way:

• generation of the record definitions in build.rs
• import of the generated definitions in a module of the project
• project implementation

Example Cargo.toml:

[package]
name = "readme"
version = "0.1.0"
edition = "2021"

[dependencies]
static_assertions = "1"
truc_runtime = { git = "https://github.com/arnodb/truc.git" }

[build-dependencies]
truc = { git = "https://github.com/arnodb/truc.git" }

Record definitions

First of all you need a type resolver. If you are not cross-compiling then HostTypeResolver will work in most cases.

Then the definitions are built with RecordDefinitionBuilder.

Once you have your definitions set up, you just need to generate the Rust definitions to an output file.

Example build.rs:

use std::{env, fs::File, io::Write, path::PathBuf};

use truc::{
    generator::{config::GeneratorConfig, generate},
    record::{definition::RecordDefinitionBuilder, type_resolver::HostTypeResolver},
};

fn main() {
    let mut definition = RecordDefinitionBuilder::new(&HostTypeResolver);

    // First variant with an integer
    let integer_id = definition.add_datum_allow_uninit::<usize, _>("integer");
    definition.close_record_variant();

    // Second variant with a string
    let string_id = definition.add_datum::<String, _>("string");
    definition.remove_datum(integer_id);
    definition.close_record_variant();

    // Remove the integer and replace it with another
    definition.add_datum_allow_uninit::<isize, _>("signed_integer");
    definition.remove_datum(string_id);
    definition.close_record_variant();

    // Build
    let definition = definition.build();

    // Generate Rust definitions
    let out_dir = env::var("OUT_DIR").expect("OUT_DIR");
    let out_dir_path = PathBuf::from(out_dir);
    let mut file = File::create(out_dir_path.join("readme_truc.rs")).unwrap();
    write!(
        file,
        "{}",
        generate(&definition, &GeneratorConfig::default())
    )
    .unwrap();
}

Project implementation

#[macro_use]
extern crate static_assertions;

#[allow(dead_code)]
#[allow(clippy::borrowed_box)]
#[allow(clippy::module_inception)]
mod truc {
    include!(concat!(env!("OUT_DIR"), "/readme_truc.rs"));
}

fn main() {
    use crate::truc::*;

    for record_2 in (0..42)
        .into_iter()
        .map(|integer| Record0::new(UnpackedRecord0 { integer }))
        .map(|mut record_0| {
            (*record_0.integer_mut()) *= 2;
            record_0
        })
        .map(|record_0| {
            let string = record_0.integer().to_string();
            Record1::from((record_0, UnpackedRecordIn1 { string }))
        })
        .map(|record_1| {
            let UnpackedRecord1 { string } = record_1.unpack();
            Record2::new(UnpackedRecord2 {
                signed_integer: string.parse().unwrap(),
            })
        })
    {
        println!("{}", record_2.signed_integer());
    }
}

So...

Did you see any unsafe code so far?

More complex examples

See the Truc examples and Truc Rust documentation for more information.