Hello,
After a following several get started tutorials and trying a few things, I’m seriously considering migrating my project to Ash framework.
Before I really start to migrate, I would like a clean view on how to test my app under Ash framework.
I’ve rode a few posts but not a lot talking about that:
- Testing Ash framework apps
- Ash Api testing setup (intermittent unrelated test failures in application)
And from the official doc:
I learned with Elixir community that tests are a very important part of an application, and I like the TDD approach even if I don’t follow it all the time. Tests remain essential in a development process, I don’t invent anything.
That’s why I create this post.
My goal is to regroup feedbacks from the community on how you handle the tests of your application, when using Ash framework, before than in-depth articles and tutorials are written on this subject.
I have my own questions to feed the subject:
Usage
1. Resource generation (fixture)
Within a classical Phoenix application, fixtures are written as follow:
defmodule MyApp.SupportFixtures do
@moduledoc """
This module defines test helpers for creating
entities via the `MyApp.Support` context.
"""
@doc """
Generate a ticket.
"""
def ticket_fixture(attrs \\ %{}) do
{:ok, ticket} =
attrs
|> Enum.into(ticket_valid_attrs())
|> MyApp.Support.create_ticket()
ticket
end
def ticket_valid_attrs() do
%{subject: "some subject"}
end
end
Then you can test like that, and it works:
defmodule MyApp.SupportTest do
use MyApp.DataCase
alias MyApp.Support
import MyApp.SupportFixtures
@tag :phoenix
describe "tickets" do
test "list_tickets/0 returns all tickets" do
ticket = ticket_fixture()
assert Support.list_tickets() == [ticket]
end
end
end
Now with Ash Framework, generating the fixture can looks like:
defmodule MyApp.SupportFixtures do
@moduledoc """
This module defines test helpers for creating
entities via the `MyApp.Support` context.
"""
alias MyApp.Support
alias MyApp.Support.Ticket
@doc """
Generate a ticket.
"""
def ticket_fixture(attrs \\ %{}) do
final_attrs = attrs |> Enum.into(ticket_valid_attrs())
Ash.Changeset.for_create(Ticket, :create, final_attrs)
|> Support.create!()
end
def ticket_valid_attrs() do
%{subject: "some subject"}
end
end
And the test could be:
defmodule MyApp.SupportTest do
use MyApp.DataCase
alias MyApp.Support
import MyApp.SupportFixtures
@tag :ash
describe "tickets" do
alias MyApp.Support.Ticket
test "read returns all tickets" do
ticket = ticket_fixture()
assert Ash.Query.for_read(Ticket, :read) |> Support.read!() ==
[ticket]
end
end
end
But this test will fail, with the following error message:
(because coloration doesn’t help here, the only difference here is on the metadata field)
Assertion with == failed
code: assert Ash.Query.for_read(Ticket, :read) |> Support.read!() == [ticket]
left: [
%MyApp.Support.Ticket{
__lateral_join_source__: nil,
__meta__: #Ecto.Schema.Metadata<:loaded, "tickets">,
__metadata__: %{
selected: [:id, :subject, :status, :representative_id]
},
__order__: nil,
aggregates: %{},
calculations: %{},
id: "9ec80178-b60d-48f2-b86e-892300d6ba90",
representative: #Ash.NotLoaded<:relationship, field: :representative>,
representative_id: nil,
status: :open,
subject: "some subject"
}
]
right: [
%MyApp.Support.Ticket{
__lateral_join_source__: nil,
__meta__: #Ecto.Schema.Metadata<:loaded, "tickets">,
__metadata__: %{},
__order__: nil,
aggregates: %{},
calculations: %{},
id: "9ec80178-b60d-48f2-b86e-892300d6ba90",
representative: #Ash.NotLoaded<:relationship, field: :representative>,
representative_id: nil,
status: :open,
subject: "some subject"
}
]
The basic solution I found to fix it is querying the ticket from the database after it’s creation inside the fixture:
defmodule MyApp.SupportFixtures do
@moduledoc """
This module defines test helpers for creating
entities via the `MyApp.Support` context.
"""
alias MyApp.Support
alias MyApp.Support.Ticket
@doc """
Generate a ticket.
"""
def ticket_fixture(attrs \\ %{}) do
final_attrs = attrs |> Enum.into(ticket_valid_attrs())
ticket =
Ash.Changeset.for_create(Ticket, :create, final_attrs)
|> Support.create!()
Ash.Query.for_read(Ticket, :by_id, %{id: ticket.id})
|> Support.read_one!()
end
def ticket_valid_attrs() do
%{subject: "some subject"}
end
end
But I find this solution is redundant. I wonder if anyone has a better solution for such a case. How would you handle the variations of the metadata field?
2. JSON schema assertion
Within a classical Phoenix App, each schema has it’s own controller to handle requests at the server level, and JSON module to render the data:
Controller
defmodule MyAppWeb.Api.Support.TicketController do
use MyAppWeb, :controller
alias MyApp.Support
action_fallback MyAppWeb.Api.FallbackController
def index(conn, _params) do
tickets = Support.list_tickets()
render(conn, :index, tickets: tickets)
end
end
JSON view
defmodule MyAppWeb.Api.Support.TicketJSON do
alias MyApp.Support.Ticket
@doc """
Renders a list of tickets.
"""
def index(%{tickets: tickets}) do
%{data: for(ticket <- tickets, do: data(ticket))}
end
def data(%Ticket{} = ticket) do
%{
id: ticket.id,
subject: ticket.subject,
status: ticket.status,
}
end
end
Then you can test it like that:
defmodule MyAppWeb.Api.Support.TicketControllerTest do
use MyAppWeb.ConnCase
import MyApp.SupportFixtures
alias MyAppWeb.Api.Support.TicketJSON
setup %{conn: conn} do
{:ok, conn: put_req_header(conn, "accept", "application/json")}
end
setup [:create_and_set_api_key]
describe "index" do
setup [:create_ticket]
@tag :phoenix
test "lists all tickets", %{conn: conn, ticket: ticket} do
conn = get(conn, ~p"/api/v1/support/tickets")
assert json_response(conn, 200)["data"] == [
ticket |> to_normalised_json()
]
end
defp create_ticket(_) do
ticket = ticket_fixture()
%{ticket: ticket}
end
defp to_normalised_json(data) do
data
|> TicketJSON.data()
|> normalise_json()
end
end
With normalise_json/1 defined in MyAppWeb.ConnCase module:
defmodule MyAppWeb.ConnCase do
@moduledoc """ ... """
use ExUnit.CaseTemplate
import Plug.Conn
[...]
@doc """
Format data to be normalised JSON and pass asserts in tests.
"""
def normalise_json(data) do
data
|> Jason.encode!()
|> Jason.decode!()
end
[...]
end
It makes the assertion easy with any kind of schema since we pass the created ticket to the same function used by the controller to render the data.
With AshJsonApi, rendered resources are autogenerated from the resource description, so how would you make the assertion easy with data coming from a newly created ticket for instance? It would be nice to be able to use the same function that is used by AshJsonApi.Controllers.Get to generate the JsonApi schema for instance, but I’m currently not used to how it works.
defmodule MyAppWeb.AshControllerTest do
use MyAppWeb.ConnCase
import MyApp.SupportFixtures
defp create_ticket(_) do
ticket = ticket_fixture()
%{ticket: ticket}
end
setup %{conn: conn} do
{:ok, conn: put_req_header(conn, "accept", "application/vnd.api+json")}
end
setup [:create_and_set_api_key]
describe "index" do
setup [:create_ticket]
@tag :ash
test "lists all tickets", %{conn: conn, ticket: ticket} do
conn = get(conn, ~p"/ash/tickets")
assert json_response(conn, 200)["data"] == "data"
end
end
end
This code renders an error message:
Assertion with == failed
code: assert json_response(conn, 200)["data"] == "data"
left: [%{"attributes" => %{"representative_id" => nil, "status" => "open", "subject" => "some subject"}, "id" => "4ed2d671-2961-4ffc-9435-40413862ecf4", "links" => %{}, "meta" => %{}, "relationships" => %{"representative" => %{"links" => %{}, "meta" => %{}}}, "type" => "ticket"}]
right: "data"
I could copy/paste the left side and re-arrange it to populate values with variables such as "id" => ticket.id, but I’m looking for something that I don’t need to modify in tests each time I will bring modifications to the JSON rendered schema.
Design
1. Design Ash controllers tests
As we have seen in my previous examples, a common design for API controllers in Phoenix applications is to have:
- 1 Controller file per schema
- 1 JSON file per schema
With Ash, and more specifically AshJsonApi, we have:
- 1 generic (dependency level) Controller file per action
- 1 generic (dependency level) JSON generator → for all resources? (to clarify, my comprehension is not perfect on it for now)
So I wonder what would be the best strategy for tests:
- create a ControllerTest file for each resource (equivalent to ControllerTest file for each schema which I currently have on Phoenix)
OR - create a ControllerTest file for each action
While writing it, I think the first option is the best, as I find it logic to regroup tests by resources (it creates a better overview of what you can / cannot do with your resource’s API, just as you have an overview of each resource in their respective Ash.Resource file).
Maybe some of you came with a third approach that I haven’t thought of, feedbacks are more than welcome to share the best practices!
I put again my questions here:
Usage
- How would you handle the variations of the metadata field?
- How would you normalize the assigns with JSON renders in controllers tests?
Design
- How do you organize your controller tests with Ash (AshJsonApi or AshGraphQL even if I’m not sure to use the second in the short term)
. As part of the documentation overhaul, we will be adding some testing guidelines. I believe that testing is very situational, but there are a few things that make testing Ash code a bit unique.
. Let me know your thoughts and I will respond tomorrow 
