Prototyping and enforcing context function conventions

Phoenix Forum Questions / Help
, , ,
1

I’ve gone through various iterations of experiments to more quickly prototype and enforce conventions in my contexts. Generally, my app has a lot of CRUD and repetition in the contexts. I don’t mind so much in the areas with complexity, churn, etc, but some of the areas rarely get touched. However, sometimes the conventions for all of these CRUD functions might take on slight changes in naming, arguments, specs, etc and it’s not a great time updating throughout the app. While the functions are not difficult to write at all, I’m questioning some of the maintenance burden and the ability to enforce consistency.

I know that there are “admin panels” and such that can generate everything from database updates to templates. However, in most cases I’m not willing to cede that much control. I’d like to be able to add custom functions alongside those CRUD functions generated within the context and easily transition the CRUD functions to custom when needed without friction.

I’ve experimented with the API below and I’m curious if others are able to fairly easily understand it.

In short, these Crud.* macros generate functions (commented above each) that follow the naming conventions and arguments of my app-specific context functions. They are purposefully limited to require all arguments with no custom options. The concept is that if I need the generated function to be named or operate in ANY different way, that it should just be written out. I’ve tried out more customizable macros, but that seems to result in spending more time trying to remember what is/isn’t supported than just writing the function out in the first place. This would hopefully allow easily seeing those functions that follow the conventions 100% and those that necessarily do not. Any updates to the conventions could easily be made in the macros.

defmodule MyApp.Inventory.Unit do
  # In practice, require/alias would be pulled in via a shared context `use`
  require MyApp.Crud
  use MyApp.Crud

  alias MyApp.Inventory.Unit

  # Configure the resource schema and repo.
  @resource Crud.config(Unit, Repo)

  #######################################################################
  # READ QUERIES
  #
  # This is very specific to my app, but all of these generated functions take an optional
  # `queries` argument. This allows passing public preload, join, order, and filter functions.
  #
  # Example:
  #
  # list_units(fn query ->
  #   query
  #   |> preload_unit_property()
  #   |> filter_by_one(property)
  #   |> order_units_by_name()
  # end)
  #######################################################################

  # list_units(queries \\ & &1)
  Crud.list(@resource)
  # paginate_units(page \\ 1, page_size \\ 25, queries \\ & &1)
  Crud.paginate(@resource)

  # get_unit!(id, queries \\ & &1)
  Crud.get!(@resource)
  # get_unit(id, queries \\ & &1)
  Crud.get(@resource)

  # The "get for" functions convention in my app is specifically for cases where there is a
  # one-to-one relationship between two resources. This example would assume that `unit`
  # `belongs_to` a `network.
  #
  # get_unit_for_network!(network, queries \\ & &1)
  Crud.get_for!(@resource, :network)
  # get_unit_for_network(network, queries \\ & &1)
  Crud.get_for(@resource, :network)

  # get_unit_by_name!(name, queries \\ & &1)
  Crud.get_by_attr!(@resource, :name)
  # get_unit_by_name(name, queries \\ & &1)
  Crud.get_by_attr(@resource, :name)

  # preload_unit_property(query)
  Crud.preload(@resource, :property)

  # join_unit_property(query)
  Crud.join(@resource, :property)

  # Only ONE of these three macros would be called depending upon whether one,
  # many, or one or many are allowed. The generated function name is the same
  # for all.
  #
  # filter_units_by_property(query, property)
  Crud.filter_by_one(@resource, :property)
  Crud.filter_by_many(@resource, :property)
  Crud.filter_by_one_or_many(@resource, :property)

  # order_units_by_name(query)
  Crud.order_by(@resource, :name, :desc)

  #######################################################################
  # CREATE, UPDATE, DELETE
  #
  # create, change, and upate are passed a changeset function
  #######################################################################

  # new_unit()
  Crud.new(@resource)

  # create_unit(attrs)
  Crud.create(@resource, &Unit.changeset/2)

  # change_unit(unit, attrs \\ %{})
  Crud.change(@resource, &Unit.changeset/2)

  # update_unit(unit, attrs)
  Crud.update(@resource, &Unit.changeset/2)

  # delete_unit(unit)
  Crud.delete(@resource)
end

UPDATE: The underlying macros can be viewed in this gist. I’m more focused on the desired API at this point and these macros could also use some work (enforcing struct args, maybe specs, documentation, etc).

In practice, a lot of context resource modules would end up as simple as something like the following:

defmodule MyApp.Inventory.Unit do
  use MyApp.Contexts

  alias MyApp.Inventory.Unit

  @resource Crud.config(Unit, Repo)

  # Common CRUD functions
  Crud.list(@resource)
  Crud.get!(@resource)
  Crud.get(@resource)
  Crud.new(@resource)
  Crud.create(@resource, &Unit.changeset/2)
  Crud.change(@resource, &Unit.changeset/2)
  Crud.update(@resource, &Unit.changeset/2)
  Crud.delete(@resource)

  # CRUD helpers
  Crud.paginate(@resource)
  Crud.get_for!(@resource, :network)
  Crud.get_for(@resource, :network)
  Crud.get_by_attr!(@resource, :name)
  Crud.get_by_attr(@resource, :name)
  Crud.preload(@resource, :property)
  Crud.join(@resource, :property)
  Crud.filter_by_one(@resource, :property)
  Crud.filter_by_one_or_many(@resource, :unit_type)
  Crud.order_by(@resource, :name, :desc)
end
1 Like
2

The original post was regarding generating functions in the context. I would still like some tests in place, which tend to be even more repetitive and time-consuming than the context functions. Thus, a corresponding CrudTest module has been added. The idea here is, to the extent possible, match the API of the context function macros. The basic flow for each is to insert a factory/factories for the given resource, call the corresponding function, and then assert the results. As in the case of the context functions, if I need to add, change, or test these functions in ANY other way, I should just write out the tests and not use the macro. All of these macros generate a describe block with the function name and then include any number of test cases that make sense for the context function.

For example, CrudTest.list(@resource) will insert a few unit records and then assert that list_units() returns those records. Factory records are also created for various associations as needed in get_for!, preload, join, filter_by_*, etc. For most data types, order_by will generate data and assert the correct ordering.

The main difference is that the context function macros create, update, and change take a changeset as the second argument, whereas the test macros take actual test data (as a map or function) as that argument.

Most of the tests cover what I usually need for each case. The exception is in create and update. Those are purely smoke tests that ensure that passing valid data can create and update a record. In most cases, I will want to write out tests to cover error cases and such.

Factory names are based off the resource/association schema name. In cases where that does not match for the resource a factory option can be passed to config (ex. factory: :user). In case where associations do not match, an association_factories option can be passed to config (ex. [creator: :user, property: :building]).

Here is an example of tests for the corresponding functions in the original post above.

defmodule MyApp.Inventory.UnitTest do
  use MyApp.DataCase

  alias MyApp.Inventory.Unit

  @resource CrudTest.config(Unit, Repo)

  # Common CRUD function tests
  CrudTest.list(@resource)
  CrudTest.get!(@resource)
  CrudTest.get(@resource)
  CrudTest.new(@resource)
  CrudTest.create(@resource, %{name: "My Place"})
  CrudTest.change(@resource, %{name: "My Place"})
  CrudTest.update(@resource, %{name: "My Place"})
  CrudTest.delete(@resource)

  # CRUD helper tests
  CrudTest.paginate(@resource)
  CrudTest.get_for!(@resource, :network)
  CrudTest.get_for(@resource, :network)
  CrudTest.get_by_attr!(@resource, :name)
  CrudTest.get_by_attr(@resource, :name)
  CrudTest.preload(@resource, :property)
  CrudTest.join(@resource, :property)
  CrudTest.filter_by_one(@resource, :property)
  CrudTest.filter_by_one_or_many(@resource, :unit_type)
  CrudTest.order_by(@resource, :name, :desc)
end
3

That is interesting. Are you still using this? Pros/cons? :slight_smile:

I was working on something similar, it’s a bit more code but I stayed away from macros to keep it explicit:

defmodule CrudController do
  use MyAppWeb, :controller
  alias MyApp.{Repo, Query}

  def index(conn, params) do
    render(assign_entries(conn, params), "index.html")
  end

  def new(conn, params) do
    render(assign_changeset(conn, params), "new.html")
  end

  def create(conn, params) do
    case controller_module(conn).create_entry(conn, params) do
      # ...  
    end
  end

  def show(conn, params) do
    render(assign_entry(conn, params), "show.html")
  end

  def edit(conn, params) do
    render(assign_changeset(conn, params), "edit.html")
  end

  def update(conn, params) do
    conn = assign_entry(conn, params)

    case controller_module(conn).update_entry(conn, params) do
      # ...
  end

  def delete(conn, params) do
    # ...
  end

  def action_path(conn, action_name, query \\ []) do
    assigns = Map.put(conn.assigns, :conn, conn)
    view_module(conn).action_path(assigns, action_name, query)
  end

  defp assign_entries(conn, params) do
    assign(conn, :entries, find_entries(conn, params))
  end

  defp assign_entry(conn, params) do
    assign(conn, :entry, find_entry(conn, params))
  end

  defp assign_changeset(conn, %{"id" => _id} = params) do
    conn = assign_entry(conn, params)
    assign(conn, :changeset, controller_module(conn).edit_changeset(conn, params))
  end

  defp assign_changeset(conn, params) do
    assign(conn, :changeset, controller_module(conn).new_changeset(conn, params))
  end

  def find_entries(conn, params) do
    controller_module(conn).query_entries(conn, params) |> Repo.all()
  end

  def find_entry(conn, %{"id" => id} = params) do
    controller_module(conn).query_entries(conn, params)
    |> Query.where(:id, id)
    |> Repo.one!()
  end
end

Then in a controller:

defmodule MyController do
  use MyAppWeb, :controller

  defdelegate index(conn, params), to: CrudController
  defdelegate new(conn, params), to: CrudController
  defdelegate create(conn, params), to: CrudController
  defdelegate show(conn, params), to: CrudController
  defdelegate edit(conn, params), to: CrudController
  defdelegate update(conn, params), to: CrudController
  defdelegate delete(conn, params), to: CrudController

  def query_entries(_conn, %{"organization_id" => organization_id}) do
    # ... returns an Ecto.Query ...
  end

  def new_changeset(_conn, _params) do
    # ...
  end

  def edit_changeset(conn, _params) do
    # ...
  end

  def create_entry(_conn, _params) do
    # ...
  end

  def update_entry(conn, _params) do
    # ...
  end

  def delete_entry(conn, _params) do
    # ...
  end
end

The point here is that the crud actions simply call a function on CrudController:

  def index(conn, params) do
    CrudController.index(conn, params)
  end

Then there are a couple of functions that the controller should implement (plan to add @behaviour for this):

  • query_entries/2
  • new_changeset/2
  • edit_changeset/2
  • create_entry/2
  • update_entry/2
  • delete_entry/2

The crud controller uses these functions via controller_module(conn).<function>, not much magic going on there either.

This is where you can implement controller specific stuff such as scoping by a parent id.

For me this is a good balance between explicit and easy to follow while still saving a ton of code in the controller (thus being more consistent across the code base) and still providing enough flexibility to implement stuff like nested resources.

I might stil go for use CrudController which would add the @behaviour + default implementations for crud actions but ultimately that would make the code harder to follow so not sure.

Thoughts? let me know:)

4

I’m still using it, but haven’t been adding much to the app where it was added recently. It makes adding new resources quite easy and ensures a consistent naming scheme. It’s the abstraction I wanted for my context functions to enforce conventions, but is just as easy to replace any one function with something more custom as needed. Outside the context, the exposed function names are the same so anything using them is none the wiser if generated via macro or later transitions to something more custom.

What you’re doing there is a little different in going at it at the controller level as opposed to the context level. While I have quite a bit of crud at the context level, my controllers tend to bring together data from a number of different contexts such that I don’t get much mileage out of streamlining that. I also just don’t access the repo directly from controllers; only contexts have access to call those. The benefit being that controllers can focus on shuttling requests and responses rather than dealing with domain concerns. This also means multiple controllers, apps, APIs can all use the same public context that is decoupled from the web app. At any rate, especially if dealing with very repetitive basic resources, there are multiple ways to approach it from just a few minimal niceties to a full-stack admin panel.

1 Like