Encrypted storage tools for PersonalMediaVault

This library implements a collection of tools to create an encrypted storage, for the PersonalMediaVault project:

• Functions to encrypt and decrypt, using AES-256, with the option to compress the data using ZLIB.
• Read and write streams to create and read encrypted files in chunks.
• Read and write streams to pack multiple small encrypted files into a single container file.

Documentation

Installation

In order to add it to your project, use

cargo add pmv_encryption_rs

File encryption

You can encrypt a buffer of data using encrypt, with a key of 64 bytes.

You can then decrypt it using decrypt and the same key.

use pmv_encryption_rs::{encrypt, decrypt, EncryptionMethod};

fn example() {
    let key: &[u8] = &[0x01; 32]; // Mock key

    let data = "Hello world!".as_bytes();

    let encrypted = encrypt(data, EncryptionMethod::Aes256Flat, key).unwrap();

    let decrypted = decrypt(&encrypted, key).unwrap();

    assert_eq!(
        decrypted.as_slice(),
        data
    );
}

Details

The encrypted data returned by encrypt_file_contents and accepted by decrypt_file_contents is binary-encoded, with the following structure:

The system is flexible enough to allow multiple encryption algorithms. Currently, there are 2 supported ones:

• AES256_ZIP: ID = 1, Uses ZLIB (RFC 1950) to compress the data, and then uses AES with a key of 256 bits to encrypt the data, CBC as the mode of operation and an IV of 128 bits. This algorithm uses a header of 20 bytes, containing the following fields:

• AES256_FLAT: ID = 2, Uses AES with a key of 256 bits to encrypt the data, CBC as the mode of operation and an IV of 128 bits. This algorithm uses a header of 20 bytes, containing the following fields:

Block-Encrypted Files

Block encrypted files are used to encrypt an arbitrarily large file, splitting it's contents in blocks (or chunks) with a set max size. Each block is then encrypted using the file encryption method detailed above.

For creating / writing files:

• You can create a file using FileBlockEncryptWriteStream::new, a function that returns a new instance of FileBlockEncryptWriteStream. It creates and initializes the file.
• Once it is created, you may call FileBlockEncryptWriteStream::write to write data into the file. When the data reached a block limit, that block is encrypted and stored into the file.
• After you wrote all the data, you must call FileBlockEncryptWriteStream::close to write any pending data and close the file.

For reading files:

• You can open a file calling FileBlockEncryptReadStream::new, a function that returns an instance of FileBlockEncryptReadStream, opening the file for reading.
• After it's opened, you may call FileBlockEncryptReadStream::get_file_size, FileBlockEncryptReadStream::get_block_size or FileBlockEncryptReadStream::get_block_count to retrieve the parameters of the file.
• You may call FileBlockEncryptReadStream::read to decrypt and read the data.
• You can call FileBlockEncryptReadStream::seek to change the cursor position. You may also call FileBlockEncryptReadStream::get_cursor_position to retrieve the cursor position if needed.
• After you are done, you may call FileBlockEncryptReadStream::close to close the file. It is also closed on drop.

use pmv_encryption_rs::{FileBlockEncryptWriteStream};

use rand::{RngCore, SeedableRng, rngs::SmallRng};
use tempdir::TempDir;

const MOCK_DATA_SIZE: usize = 1 * 1024 * 1024;
const MOCK_BLOCK_SIZE: u64 = 128 * 1024;

const MOCK_DATA_READ_SIZE: usize = 300 * 1024;
const MOCK_DATA_READ_OFFSET: usize = 500 * 1024;

fn example() {
    // Mock key
    let key: &[u8] = &[0x01; 32];

    // Mock data
    let mut mock_data: Vec<u8> = vec![0; MOCK_DATA_SIZE];
    let mut rng = SmallRng::from_rng(&mut rand::rng());
    rng.fill_bytes(&mut mock_data);
    let file_size = mock_data.len() as u64;

    // Mock file
    let dir = TempDir::new("test_pmv_encryption_rs").unwrap();
    let file_path = dir.path().join("s_1.pma");

    // Write the data

    let mut write_stream =
        FileBlockEncryptWriteStream::new(&file_path, key.to_vec(), file_size, MOCK_BLOCK_SIZE)
            .unwrap();

    let block_count = write_stream.get_block_count();

    write_stream.write(&mock_data).unwrap();
    write_stream.close();

    // Read the data

    let mut read_stream = FileBlockEncryptReadStream::new(&file_path, key.to_vec()).unwrap();

    assert_eq!(read_stream.get_file_size(), MOCK_DATA_SIZE as u64);
    assert_eq!(read_stream.get_block_size(), MOCK_BLOCK_SIZE);
    assert_eq!(read_stream.get_block_count(), block_count);

    let mut mock_data_read_buf: Vec<u8> = vec![0; MOCK_DATA_READ_SIZE];

    let seek_pos = read_stream
        .seek(SeekFrom::Start(MOCK_DATA_READ_OFFSET as u64))
        .unwrap();

    assert_eq!(seek_pos, MOCK_DATA_READ_OFFSET as u64);
    assert_eq!(seek_pos, read_stream.get_cursor());

    let read_size = read_stream.read(&mut mock_data_read_buf).unwrap();

    assert_eq!(read_size, MOCK_DATA_READ_SIZE);

    assert!(mock_data_read_buf.iter().eq(mock_data.iter().skip(MOCK_DATA_READ_OFFSET).take(MOCK_DATA_READ_SIZE)));

    read_stream.close();
}

Details

They are binary files consisting of 3 contiguous sections: The header, the chunk index and the encrypted chunks.

The header contains the following fields:

After the header, the chunk index is stored. For each chunk the file was split into, the chunk index will store a metadata entry, withe the following fields:

After the chunk index, the encrypted chunks are stored following the same structure described above.

This chunked structure allows to randomly access any point in the file as a low cost, since you don't need to decrypt the entire file, only the corresponding chunks.

Multi-File Pack

Multi-file pack container files are used to store multiple small files inside a single container.

For creating / writing files:

• You can create a file by calling MultiFilePackWriteStream::new, a function that returns an instance of MultiFilePackWriteStream. It creates and initializes the file.
• You may call MultiFilePackWriteStream::put_file for each file you want to store, in order.
• After all files are written, you must call MultiFilePackWriteStream::close to white any pending data and close the file.

For reading files:

• You can open a file by calling MultiFilePackReadStream::new, a function that returns an instance of MultiFilePackReadStream, opening the file for reading.
• You may call MultiFilePackReadStream::get_file_count to retrieve the number of stored files.
• You may call MultiFilePackReadStream::get_file to read a file, by its index.
• After you are done, you may call MultiFilePackReadStream::close to close the file. It is also closed on drop.

use pmv_encryption_rs::{MultiFilePackWriteStream};

use rand::{RngCore, SeedableRng, rngs::SmallRng};
use tempdir::TempDir;

fn example() {
    // Mock files
    let mut rng = SmallRng::from_rng(&mut rand::rng());

    let mut mock_file_1: Vec<u8> = vec![0; 100 * 1024];
    rng.fill_bytes(&mut mock_file_1);

    let mut mock_file_2: Vec<u8> = vec![0; 200 * 1024];
    rng.fill_bytes(&mut mock_file_2);

    let mut mock_file_3: Vec<u8> = vec![0; 300 * 1024];
    rng.fill_bytes(&mut mock_file_3);

    // Temp file to store the packed files
    let dir = TempDir::new("test_pmv_encryption_rs").unwrap();
    let file_path = dir.path().join("m_1.pma");

    // Write the files in order

    let mut write_stream = MultiFilePackWriteStream::new(&file_path, 3).unwrap();

    write_stream.write_file(&mock_file_1).unwrap();
    write_stream.write_file(&mock_file_2).unwrap();
    write_stream.write_file(&mock_file_3).unwrap();
    write_stream.close();

    // Read the files (any order)

    let mut read_stream = MultiFilePackReadStream::new(&file_path).unwrap();

    assert_eq!(read_stream.get_file_count(), 3);

    let mock_file_2_r = read_stream.get_file(1).unwrap();
    assert!(mock_file_2.iter().eq(mock_file_2_r.iter()));

    let mock_file_1_r = read_stream.get_file(0).unwrap();
    assert!(mock_file_1.iter().eq(mock_file_1_r.iter()));

    let mock_file_3_r = read_stream.get_file(2).unwrap();
    assert!(mock_file_3.iter().eq(mock_file_3_r.iter()));

    assert!(matches!(read_stream.get_file(3), Err(MultiFilePackReadError::IndexOutOfBounds)));

    read_stream.close();
}

Details

They are binary files consisting of 3 contiguous sections: The header, the file table and the encrypted files.

The header contains the following fields:

After the header, a file table is stored. For each file stored by the asset, a metadata entry is stored, with the following fields:

After the file table, each file is stored following the same structure described above.