Basic model example using ecto without database

Questions / Help
1

I have the following instance of a simple Person object represented in yaml.

id: Father
person_type: PrimaryOwner
birth_year: 1968
age_assumptions:
    retirement_age: 65
    death_age: 85

I’m new to and just learning elixir and trying to understand some capabilities in terms of modeling data.
Below is the rust definition corresponding to this instance. Effectively that is one representation of the schema. My questions are:

  • Assuming I don’t want any database dependencies, how can ecto and specifically embedded_schema be used to model such a schema? The answer to this makes me think it’s straightforward but since I don’t know answers to questions below I’m not sure how to get started.

  • If it can, is there an adapter that can just keep the repository in memory and read/write it in toto as json or yaml on demand?

  • If that is possible, is all the meta-data of the ecto ebmedded schema available for programmatic access from elixir? For example, could elixir code find all enumerations or all schema fields with a default value?

#[derive(
    Clone, Copy, Debug, Deserialize, Eq, Hash, Ord, PartialEq, PartialOrd, Serialize,
)]
pub enum PersonType {
    /// The primary owner of the `Dossier`
    PrimaryOwner,
    /// The secondary owner of the `Dossier`
    SecondaryOwner,
    /// A dependent - as in child
    Dependent,
}

/// Assumptions regarding the ages of a person _at retirement_, _at death_, etc.
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
pub struct AgeAssumptions {
    /// Age of retirement - where labor incomes are ended.
    ///
    pub retirement_age: i32,
    /// Age at death.
    pub death_age: i32,
}
#[derive(Clone, Debug, Default, Deserialize, PartialEq, Serialize)]
pub struct Person {
    /// Identifier for the person (eg name or nom de guerre)
    pub id: String,
    /// Type of person
    pub person_type: PersonType,
    /// Year of birth of person
    pub birth_year: i32,
    /// Important ages for person
    #[serde(skip_serializing_if = "is_default")]
    pub age_assumptions: Option<AgeAssumptions>,
}

/// Implementation of trait `Default` for type `AgeAssumptions`
impl Default for AgeAssumptions {
    /// A trait for giving a type a useful default value.
    ///
    ///  * _return_ - The default value for the type
    ///
    fn default() -> Self {
        AgeAssumptions {
            retirement_age: 65,
            death_age: 95,
        }
    }
}
1 Like
2

:wave:

I’d probably use ets directly with something like typed_structs, but if you need some functionality from ecto, it would probably look something like wojtekmach/ets_ecto (if you need to “query” the in-memory data store using ecto queries) or Lonestar ElixirConf 2019 - Ecto Without a DB - Greg Vaughn (if you don’t need to query it, but just use changesets and embedded schemas to prepare the incoming data).

8 Likes
3

Thank you. That Lonestar video was the best technical video I’ve seen in a while. Very informative.

3 Likes
4

Hello. I’m the presenter of that referenced talk. Thank you for your kind words!

I have not needed an in-memory datastore of those embedded schema based structs, but any recommendation for that will depend on how sophisticated you expect your queries to be. I agree with @idi527 that ets would be my first choice if you only need to look up records based upon a single key. If your needs are more sophisticated mnesia is included with OTP, though it has a steep learning curve for queries.

4 Likes
5

I will be watching it again this weekend with an iex open to play/learn.

What I am after, other than an excuse to use elixir, is a replacement for some Dart code generation. I have outlined my current task in a previous question to both elixir and clojure communities to help me select a good replacement language. I’m happy with what I’m seeing in elixir and will use it.

I’m trying to decide between using elixir embedded schema or rolling my own modules which I previously implemented in Dart. My focus on in-memory was poorly worded because it’s less about that and more about not wanting the weight of a database. So your talk was a perfect intro. I have maybe 50 schemas and a total of less than 1,000 instances stored in the flat files that I translate via code generation into Rust and other target languages. I can easily edit the instances and regenerate the code. I don’t need any complex queries, just ability to iterate on fields and maybe filter. So the process is like protobufs, capnp, thrift, etc just focusing on the structure and not behavior.

I do like the look of ecto definitions but I’m not sure yet on how to specify/store instances of the schemas conveniently in flat file. From what I gather the choices are basic elixir literal syntax or use json/yaml with ecto as the Mapper.

So, for the sample above the schema in Dart code looks like:

  enum_('person_type', [
    ev('primary_owner', 'The primary owner of the `Dossier`'),
    ev('secondary_owner', 'The secondary owner of the `Dossier`'),
    ev('dependent', 'A dependent - as in child')
  ])
    ..doc = 'Used to categorize people covered by `Dossier`',

  object('age_assumptions', [
    field('retirement_age', Int32)
      ..doc = '''
Age of retirement - where labor incomes are ended.
''',
    field('death_age', Int32)
      ..doc = '''    
Age at death.''',
  ])
    ..doc = '''
Assumptions regarding the ages of a person _at retirement_, _at death_, etc.'''
    ..setProperty('rust_own_module', true)
    ..setProperty('rust_not_derives', ['Default']),

  object('person', [
    field('id')..doc = 'Identifier for the person (eg name or nom de guerre)',
    anonymousField('person_type')..doc = 'Type of person',
    field('birth_year', Int32)..doc = 'Year of birth of person',
    anonymousField('age_assumptions')
      ..doc = 'Important ages for person'
      ..isOptional = true,
  ])
    ..doc = '''
Captures just enough about the `Person` to forecast.
Ability to determine age is important for determining timing of
events - like RMD requirements, statistics on costs such as healthcare,
etc
'''
    ..setProperty('rust_own_module', true),

The code generation turns it into the rust previously shown. So now I want to either convert those schema into elixir embedded schema or just port my existing libraries from Dart. Either way I’ll have to decide how to store my instances of the schemas as elixir literals or json. Any suggestions there would help.

6

Code generation is not something I’d recommend as a first introduction to Elixir, but you gotta do what makes sense for you :slight_smile: I went over it rather quickly, but note the schemaless changeset EventSearchController example in my talk. With that, the core inputs you need to validate incoming data is a map of that data, plus a “types” map consisting of field names to field types. Encoding those as json or yaml will be simple. The complexity comes from extra validate_* calls you might want to add to the changeset. I’m not sure if the schemaless approach would save you effort/complexity, but it’s something to consider.

Whether you choose schemaless changesets or embedded schemas, your result with be a map or a struct (which really is a map). Iterating and filtering are available to you via Enum and Map modules fairly extensively.

1 Like
7

Code generation is not something I’d recommend as a first introduction to Elixir

What would the concerns be particular to code generation? Personally I don’t mind writing something that is not great if it provides an opportunity to learn and I can fix/improve it over time. I’ve been using this project, kojin, my first elixir project to learn the language. I’m sure it is quite raw, noobish and I have a lot to learn.

Any tips on the best way to solicit comments and suggestions on the coding? I find code generation
quite fun so I am less interested in critiques of the idea/intent of the project as I am in learning all the
silly things I’m doing in elixir and how to fix them.