Crates.io | cder |
lib.rs | cder |
version | 0.2.1 |
source | src |
created_at | 2023-01-24 13:37:05.468082 |
updated_at | 2023-12-04 02:08:26.00364 |
description | database seed generator that helps create and persist struct-typed instances based on serde-compatible yaml files |
homepage | |
repository | https://github.com/estie-inc/cder |
max_upload_size | |
id | 766676 |
size | 64,004 |
cder (see-der) is a database seeding tool to help you import fixure data in your local environment.
Generating seeds programmatically is a easy task, but maintaining them is not. Everytime when your schema is changed, your seed can be broken. It costs your team extra efforts to keep them updated.
cder has no mechanism for database interaction, so it can works with any type of ORM or database wrapper (e.g. sqlx) your application have.
This embedded-tag mechanism is inspired by fixtures that Ruby on Rails provides for test data generation.
# Cargo.toml
[dependencies]
cder = "0.2"
Suppose you have users table as seeding target:
CREATE TABLE
users (
`id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
`name` VARCHAR(255) NOT NULL,
`email` VARCHAR(255) NOT NULL,
)
In your application you also have:
<T>
(usually a model, built upon a underlying table)Fn(T) -> Result<i64>
First, add DeserializeOwned trait on the struct.
(cder brings in serde as dependencies, so derive(Deserialize)
macro can do the job)
use serde::Deserialize;
#[derive(Deserialize)] // add this derive macro
User {
name: String,
email: String,
}
impl User {
// can be sync or async functions
async fn insert(&self) -> Result<(i64)> {
//
// inserts a corresponding record into table, and returns its id when succeeded
//
}
}
Your User seed is defined by two separate files, data and glue code.
Now create a seed data file 'fixtures/users.yml'
# fixtures/users.yml
User1:
name: Alice
email: 'alice@example.com'
User2:
name: Bob
email: 'bob@example.com'
Now you can insert above two users into your database:
use cder::DatabaseSeeder;
async fn populate_seeds() -> Result<()> {
let mut seeder = DatabaseSeeder::new()
seeder
.populate_async("fixtures/users.yml", |input| {
async move { User::insert(&input).await }
})
.await?;
Ok(())
}
Et voila! You will get the records Alice
and Bob
populated in your database.
If your function is non-async (normal) function, use Seeder::populate
instead of Seeder::populate_async
.
use cder::DatabaseSeeder;
fn main() -> Result<()> {
let mut seeder = DatabaseSeeder::new();
seeder
.populate("fixures/users.yml", |input| {
// this block can contain any non-async functions
// but it has to return Result<i64> in the end
diesel::insert_into(users)
.values((name.eq(input.name), email.eq(input.email)))
.returning(id)
.get_result(conn)
.map(|value| value.into())
})
Ok(())
}
If you want to take more granular control over the deserialized structs before inserting, use StructLoader instead.
use cder::{ Dict, StructLoader };
fn construct_users() -> Result<()> {
// provide your fixture filename followed by its directory
let mut loader = StructLoader::<User>::new("users.yml", "fixtures");
// deserializes User struct from the given fixture
// the argument is related to name resolution (described later)
loader.load(&Dict::<String>::new())?;
let customer = loader.get("User1")?;
assert_eq!(customer.name, "Alice");
assert_eq!(customer.email, "alice@example.com");
let customer = loader.get("User2")?;
assert_eq!(customer.name, "Bob");
assert_eq!(customer.email, "bob@example.com");
ok(())
}
cder replaces certain tags with values based on a couple of rules. This 'pre-processing' runs just before deserialization, so that you can define dynamic values that can vary depending on your local environments.
Currently following two cases are covered:
Let's say you have two records to be inserted in companies
table.
companies.id
s are unknown, as they are given by the local database on insert.
# fixtures/companies.yml
Company1:
name: MassiveSoft
Company2:
name: BuggyTech
Now you have user records that reference to these companies:
# fixtures/users.yml
User1:
name: Alice
company_id: 1 // this might be wrong
You might end up with failing building User1, as Company1 is not guaranteed to have id=1 (especially if you already have operated on the companies table).
For this, use ${{ REF(label) }}
tag in place of undecided values.
User1:
name: Alice
company_id: ${{ REF(Company1) }}
Now, how does Seeder know id of Compnay1 record?
As described earlier, the block given to Seeder must return Result<i64>
. Seeder stores the result value mapped against the record label, which will be re-used later to resolve the tag references.
use cder::DatabaseSeeder;
async fn populate_seeds() -> Result<()> {
let mut seeder = DatabaseSeeder::new();
// you can specify the base directory, relative to the project root
seeder.set_dir("fixtures");
// Seeder stores mapping of companies record label and its id
seeder
.populate_async("companies.yml", |input| {
async move { Company::insert(&input).await }
})
.await?;
// the mapping is used to resolve the reference tags
seeder
.populate_async("users.yml", |input| {
async move { User::insert(&input).await }
})
.await?;
Ok(())
}
A couple of watch-outs:
companies
in above examples) before 'referencing' records (users
).You can also refer to environment variables using ${{ ENV(var_name) }}
syntax.
Dev:
name: Developer
email: ${{ ENV(DEVELOPER_EMAIL) }}
The email is replaced with DEVELOPER_EMAIL
if that environment var is defined.
If you would prefer to use default value, use (shell-like) syntax:
Dev:
name: Developer
email: ${{ ENV(DEVELOPER_EMAIL:-"developer@example.com") }}
Without specifying the defalut value, all the tags that points to undefined environment vars are simply replaced by empty string "".
cder deserializes yaml data based on serde-yaml, that supports powerful serde serialization framework. With serde, you can deserialize pretty much any struct. You can see a few sample structs with various types of attributes and the yaml files that can be used as their seeds.
Below shows a few basics of required YAML format. Check serde-yaml's github page for further details.
Label_1:
name: Alice
email: 'alice@example.com'
Label_2:
name: Bob
email: 'bob@example.com'
Notice that, cder requires to name each record with a label (Label_x). Label can be anything (as long as it is a valid yaml key) but you might want to keep them unique to avoid accidental mis-references.
Enums can be deserialized using YAML's !tag
.
Suppose you have a struct CustomerProfile with enum Contact
.
struct CustomerProfile {
name: String,
contact: Option<Contact>,
}
enum Contact {
Email { email: String }
Employee(usize),
Unknown
}
You can generate customers with each type of contact as follows;
Customer1:
name: "Jane Doe"
contact: !Email { email: "jane@example.com" }
Customer2:
name: "Uncle Doe"
contact: !Employee(10100)
Customer3:
name: "John Doe"
contact: !Unknown
cder is designed to populate seeds in development (or possibly, test) environment. Production use is NOT recommended.
The project is available as open source under the terms of the MIT License.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, shall be licensed as MIT, without any additional terms or conditions.
Bug reports and pull requests are welcome on GitHub at https://github.com/estie-inc/cder