GuardedStruct - A macro allows to build Structs that provide you with a number of important options Validation, Sanitizing, Constructor

[shahryarjb]

GuardedStruct macro allows to build Structs that provide you with a number of important options Validation, Sanitizing, Constructor

The creation of this macro will allow you to build Structs that provide you with a number of important options, including the following:

1. Validation
2. Sanitizing
3. Constructor
4. It provides the capacity to operate in a nested style simultaneously.

Suppose you are going to collect a number of pieces of information from the user, and before doing anything else, you are going to sanitize them. After that, you are going to validate each piece of data, and if there are no issues, you will either display it in a proper output or save it somewhere else. All of the characteristics that are associated with this macro revolve around cleaning and validating the data.

The features that we list below are individually based on a particular strategy and requirement, but thankfully, they may be combined and mixed in any way that you see fit.

My Blog post about GuardedStruct:

Consolidating Input and Output Validation and Sanitization in Elixir with...

Nested Example

defmodule ConditionalFieldComplexTest do
  use GuardedStruct
  alias ConditionalFieldValidatorTestValidators, as: VAL

  guardedstruct do
    field(:provider, String.t())

    sub_field(:profile, struct()) do
      field(:name, String.t(), enforce: true)
      field(:family, String.t(), enforce: true)

      conditional_field(:address, any()) do
        field(:address, String.t(), hint: "address1", validator: {VAL, :is_string_data})

        sub_field(:address, struct(), hint: "address2", validator: {VAL, :is_map_data}) do
          field(:location, String.t(), enforce: true)
          field(:text_location, String.t(), enforce: true)
        end

        sub_field(:address, struct(), hint: "address3", validator: {VAL, :is_map_data}) do
          field(:location, String.t(), enforce: true, derive: "validate(string, location)")
          field(:text_location, String.t(), enforce: true)
          field(:email, String.t(), enforce: true)
        end
      end
    end

    conditional_field(:product, any()) do
      field(:product, String.t(), hint: "product1", validator: {VAL, :is_string_data})

      sub_field(:product, struct(), hint: "product2", validator: {VAL, :is_map_data}) do
        field(:name, String.t(), enforce: true)
        field(:price, integer(), enforce: true)

        sub_field(:information, struct()) do
          field(:creator, String.t(), enforce: true)
          field(:company, String.t(), enforce: true)

          conditional_field(:inventory, integer() | struct(), enforce: true) do
            field(:inventory, integer(),
              hint: "inventory1",
              validator: {VAL, :is_int_data},
              derive: "validate(integer, max_len=33)"
            )

            sub_field(:inventory, struct(), hint: "inventory2", validator: {VAL, :is_map_data}) do
              field(:count, integer(), enforce: true)
              field(:expiration, integer(), enforce: true)
            end
          end
        end
      end
    end
  end
end

Installing the library:

def deps do
  [
    {:guarded_struct, "~> 0.0.1"}
  ]
end

---

Links

Github: GitHub - mishka-group/guarded_struct: GuardedStruct macro allows to build Structs that provide you with a number of important options Validation, Sanitizing, Constructor
Hex: guarded_struct | Hex
LiveBook Document: guarded_struct/guidance/guarded-struct.livemd at master · mishka-group/guarded_struct · GitHub

[dimitarvp]

This looks great, I’ll try it.

Some of the naming is IMO prone to changes, f.ex. :is_map_data should be just fine written as :is_map. Same for :is_string_data. Also not sure about enforce, why not christen it mandatory?

Also the derive thing might need another format but I believe I can work with it as-is.

Thanks for making this.

[shahryarjb]

@dimitarvp

:is_map_data

Hello, thank you for your support.
These functions are related to custom functions that users create; they are just examples. I chose a name to make the user think this is a custom function.

@dimitarvp

Also not sure about enforce, why not christen it mandatory?

I used endforce because it is specific to the struct itself, and we only added additional conditions to it.
https://hexdocs.pm/elixir/structs.html

@dimitarvp

Also the derive thing might need another format but I believe I can work with it as-is.

Please review the LiveBook documentation I prepared. This section is highly flexible, allowing you to create advanced validation conditions using keys like on, auto, from, and domain.

I had implemented this functionality in other libraries before, and about a year ago, I separated it into a new library.

By the way, if the following libraries are also installed, some additional functions will be available in the code:

  {:email_checker, "~> 0.2.4"},
  {:ex_url, "~> 2.0"},
  {:ex_phone_number, "~> 0.4.3"}

I recommend trying it out for a while; you’ll soon realize how much it boosts your workflow speed. I’ve used it in two of my libraries, as well as in other projects, including:

• Mishka Chelekom
• Mishka Installer

Thank you in advance

[dimitarvp]

@shahryarjb

These functions are related to custom functions that users create; they are just examples. I chose a name to make the user think this is a custom function.

Ah, thanks. I’d definitely go for something shorter and self-documenting.

@shahryarjb

I used endforce because it is specific to the struct itself, and we only added additional conditions to it.

I am aware but IMO Elixir blundered the name here. enforce is ambiguous and mandatory is much clearer.

@shahryarjb

Please review the LiveBook documentation I prepared. This section is highly flexible, allowing you to create advanced validation conditions using keys like on, auto, from, and domain.

You got me. I commented before reviewing it, it was just a surface observation. Thanks for elaborating.

@shahryarjb

I recommend trying it out for a while; you’ll soon realize how much it boosts your workflow speed. I’ve used it in two of my libraries, as well as in other projects, including:

Definitely. I believe Elixir’s community really has to unite around one godlike parsing and validation / type+rule enforcement library. Yours looks like a solid candidate.

[Eiji]

Hi, here are my few cents, based on my experience. Hope you would find it helpful.

@shahryarjb

field

@shahryarjb

sub_field

@shahryarjb

conditional_field

There are lots of macros which most probably do similar work. Maybe for you it’s not a problem, but developers using your library would have to use 3 macros instead of one. What’s even more important:

---

@shahryarjb

conditional_field(:address, any()) do

is confusing, as following naming it could be:

conditional_sub_field(:address, struct()) do

Few booleans and a default option are less confusing than few macros.

---

@shahryarjb

    field(:provider, String.t())

Should be:

`    field :provider, String.t()`

All you have to do is to modify .formatter.exs by using locals_without_parens (within your library) and export: [locals_without_parens: …] to enable same thing for apps using your library. Make sure to document import_deps to remind developers and point new developers how to have a clear DSL without so many parentheses. See Importing dependencies configuration | mix format @ Mix documentation for more information.

---

@shahryarjb

{VAL, :is_string_data}

This is of course valid, but not common and therefore confusing for both new developers and those who are used to write &VAL.is_string_data/1. You can easily support such syntax as long as you use Macro.escape/1 for that. Of course maintaining that in nested DSL may be hard, so I would recommend to deal with a flatten list of fields. It could be stored as a list of 2-element Tuple where first element is path and second the data you need for compile callbacks to later generate stuff you need. Many Elixir functions are designed exactly to support such paths when dealing with nested structures.

This way escaping becomes much easier especially if you deal with for example Keyword instead of Map as you don’t have to call Macro.escape/1 on the map, but “somehow” prevent that for a value of validator key. Since lists are considered literals you can store them easily together with escaped functions. No matter if you use anonymous function notation or capture special form.

Also what’s more important as long as there is no bug in your macros the compiler could help a developer using your library to find possible problems (for example when refactoring) the code. Many other tools would also not work with such tuples. While first element is definitely an alias to the module, the second element could be anything and therefore no tool can safely assume it’s a function name without possibly breaking code.

---

@shahryarjb

    sub_field(:profile, struct()) do

For this I would recommend rather something like:

    sub_field(:profile, SomeModule.t())
    # or
    sub_field(:profile, SomeModule.t()) do

This way your DSL would be more similar to ecto:

    embeds_many :children, Child
    # or
    embeds_many :children, Child do
      field :name, :string
      field :age,  :integer
    end

---

Looks like you generate some function like builder. Could you please define a callbacks and document them separately? This way is more clear when reading documentation to quickly find what functions are generated and by using such behaviour you have a compile-time check if your macros generate function(s) you have specified in callbacks.

---

All modules except main one have only specs written and therefore they are not clear at all. Consider using @moduledoc false or document them.

---

Optionally I can also recommend using a credo. It’s very configurable tool and would catch many things like the previously mentioned one.

---

The main file is very big as it almost have 3k lines. It would be definitely much cleaner if you would keep DSL with it’s documentation separately and let other modules do most of the logic. Well … some people may disagree on this as they prefer to have everything in one place. However keep in mind that every developer outside of you team would see “3k lines blob” and … it would not be a good start for such people. I know what I was searching and such amount was a bit too much for me, so I cannot imagine how terrible it may look for less experienced developers.

I also see that lots of this is just a documentation and that’s why I suggested to break it in callbacks, maybe move some parts of documentation to other modules and to keep DSL with it’s documentation separately from the logic.

defmodule MyDSL do
  @moduledoc """
  (…)
  """

  # use/import/alias/require

  # callbacks, typedoc and so on …

  defmacro __using__(_opts \\ []) do
    # …
  end

  @doc """
  (…)
  """
  defmacro my_macro(some, arguments) do
    # some core stuff
    Helper.generate_something(…)
  end

  # nothing more here
end

What do you think about it?

[shahryarjb]

Hello,
I truly appreciate you sharing your feedback—it’s incredibly valuable to me.

Since the initial idea was inspired by another Elixir library and some of the points raised involve certain trade-offs, I’ll do my best to address them where possible. However, unfortunately, I can’t make structural changes at this stage. This library has been in use for about a year now, and rewriting macros would require significant time. Additionally, I am currently actively working on another open-source project.

---

@Eiji

There are lots of macros which most probably do similar work. Maybe for you it’s not a problem, but developers using your library would have to use 3 macros instead of one. What’s even more important:

In my opinion, the name of a macro should be completely transparent about its purpose and usage. This ensures that developers can immediately understand what the macro is and what it does. Additionally, the consumer might be a project manager with limited technical knowledge of Elixir, so clarity is crucial.

I don’t believe having three macros in the entire program is excessive. I agree that there is a learning curve involved, but it’s relatively short. To address this, I’ve made an effort to provide extensive LiveBook examples and detailed documentation to help users understand and utilize the macros effectively.

@Eiji

All you have to do is to modify .formatter.exs by using locals_without_parens (within your library) and export: [locals_without_parens: …] to enable same thing for apps using your library. Make sure to document import_deps to remind developers and point new developers how to have a clear DSL without so many parentheses. See Importing dependencies configuration | mix format @ Mix documentation for more information.

Thank you for this

@Eiji

This is of course valid, but not common and therefore confusing for both new developers and those who are used to write &VAL.is_string_data/1. You can easily support such syntax as long as you use Macro.escape/1 for that. Of course maintaining that in nested DSL may be hard, so I would recommend to deal with a flatten list of fields. It could be stored as a list of 2-element Tuple where first element is path and second the data you need for compile callbacks to later generate stuff you need. Many Elixir functions are designed exactly to support such paths when dealing with nested structures.

The validator feature was the very first functionality I added to the macro. If nothing is explicitly provided, it automatically looks for and reads from the module itself, assuming relevant information is available. If you wish to use an external module, it should be provided as a tuple.

That said, I’d be delighted to support this feature alongside the current ones. I’d be even happier if someone with the time could submit a pull request for it—I think it would turn out to be a very clean and elegant addition!

@Eiji

For this I would recommend rather something like:

    sub_field(:profile, SomeModule.t())
    # or
    sub_field(:profile, SomeModule.t()) do

The macro already supports this functionality. You can pass any type to it as needed.
Including the specific case you mentioned.

@Eiji

Looks like you generate some function like builder. Could you please define a callbacks and document them separately? This way is more clear when reading documentation to quickly find what functions are generated and by using such behaviour you have a compile-time check if your macros generate function(s) you have specified in callbacks.

Currently, the t() type is generated directly within the module. As for documentation, I was aiming for a custom approach to write concise documentation for individual fields. However, this is something I plan to address more thoroughly in the future.
Regarding the callback, I plan to include it in the next version. I already have a task assigned for it.

@Eiji

The main file is very big as it almost have 3k lines. It would be definitely much cleaner if you would keep DSL with it’s documentation separately and let other modules do most of the logic. Well …

I’m sorry, but I’m not a big fan of splitting files excessively. I believe the main macro module is already sufficiently divided into other modules. Further splitting would hinder rapid development, as developers would spend too much time searching for modules and functions.

I prefer structuring modules based on their responsibilities rather than merely reducing the number of lines.

@Eiji

What do you think about it?

I completely agree with many of the points you’ve mentioned. I’d love to add more features and create diverse implementations. However, managing the balance between compile-time and runtime has been quite time-consuming for me.

I’d really appreciate it if the community could contribute by submitting pull requests for the features they need or at least opening an issue before working on a pull request.

Personally, I’m heavily focused on the Mishka Chelekom project at the moment, so I might not be able to implement the features you’ve mentioned in the short term.

---

I hope my responses don’t come across as unwillingness to listen to and address feedback. On the contrary, I truly enjoy improving things based on constructive criticism. After all, projects only get better with input from their users.

I simply tried to explain the thoughts and considerations that have been on my mind.
Thank you in advance

[Eiji]

@shahryarjb

This library has been in use for about a year now, and rewriting macros would require significant time. Additionally, I am currently actively working on another open-source project.

Yeah, I know what you feel

---

@shahryarjb

Additionally, the consumer might be a project manager with limited technical knowledge of Elixir, so clarity is crucial.

Oh, I so wish that everyone with “limited technical knowledge” would at least read documentation. I truly appreciate your consumers.

---

@shahryarjb

If nothing is explicitly provided, it automatically looks for and reads from the module itself

Ah, that makes more sense now. It’s technically possible to do same for capture special form, but I believe it would be a bit too magic solution for end users. There could be an option for a fallback validator module which defaults to current one, so the feature would be really well documented, but this is not really a high priority, so:

@shahryarjb

support this feature alongside the current ones

makes a perfect sense in this case.

---

@shahryarjb

The macro already supports this functionality.

Oh, in that case it may be worth to include it in the example code above. As always I try to write generic advices, so as long as I don’t mention something directly I’m commenting what’s visible now for all readers and not what’s deep in source code or so.

---

@shahryarjb

field(:name, String.t(), enforce: true)

Thee are lots of lines like that, so putting some special cases instead of some fields which are different only by their names is definitely worth to consider. Please pay attention that you have mentioned any() and struct() several times.

---

@shahryarjb

I prefer structuring modules based on their responsibilities rather than merely reducing the number of lines.

Oh, I was not precise. Of course I mean both at the same time (something like Unix way of tiny apps doing very specific things). There could be (not read whole code) few helper modules you could define. One option is to split them based on features, so for example you may have few DSL macros and each one would be documented in in separate module, so you would have “private” modules like GuardedStruct.Field.

The other option is as you said divide by responsibilities, so there could be for example GuardedStruct.AST module. This for example allows contributors to “jump in” to specific feature or type and work only on meta-programming part or just improve documentation. This is especially important when documentation takes over 1k lines. Of course in smaller cases it’s not so important.

---

@shahryarjb

I hope my responses don’t come across as unwillingness to listen to and address feedback. On the contrary, I truly enjoy improving things based on constructive criticism. After all, projects only get better with input from their users.

I simply tried to explain the thoughts and considerations that have been on my mind.

Your response looks good for me. Whatever others say I fully understand, so no worries.

[mudasobwa]

You might want to check Estructura.Nested which allows validation, coercion, access, jason encoding/decoding, and generation.

Example: Estructura.User — estructura v1.6.0

[shahryarjb]

Thank you for sharing your library here. I’ve seen this library before. However, I think I created this macro several months prior to your library as part of another project of mine, which is also available in the forum.

I believe some of our features might share similarities in the core concept, but overall, what I’ve seen is quite different.

I was heavily inspired by libraries on Rust and Scala, particularly the derive style or annotations above functions. I also focused on incorporating certain logic keys under a domain, as well as routing the macro in the file where it needs to be used, aiming to establish a clear boundary between development and usage.

I hope the Elixir community continues to see more libraries like this in the future.
Thank you for introducing it!

[mudasobwa]

@shahryarjb

what I’ve seen is quite different

That’s exactly why I suggested you to take a look at estructura, to maybe gain some ideas for more sophisticated coercers/validators and especially for stream data generation, which is a cornerstone if you want to make apps using this library ever testable.

[shahryarjb]

This is a minor update with no significant changes.

What’s Changed 0.0.4

• Fix deprecated code from Elixir 1.18
• Support overridable messages for the GuardedStruct module with support for multiple languages
• Fix typo

---

Changelog: guarded_struct/CHANGELOG.md at master · mishka-group/guarded_struct · GitHub
Hex: guarded_struct | Hex
LiveBook document: guarded_struct/guidance/guarded-struct.livemd at master · mishka-group/guarded_struct · GitHub

{:guarded_struct, "~> 0.0.4"}