Combining different data layers possible?

Ash Forum Ash Questions / Help
,
1

Let me set some context:

# Resource User
defmodule User
  # uses a postgresql table
  relationships do
    has_many :reports, Report
  end
end

# Resource Report
defmodule Report
  # uses a postgresql table
  attributes do
    attribute :date, :date
    attribute :duration, :duration # <- a custom type but not relevant here
  end

  relationships do
    belongs_to :user, User
  end 
end

In my app I want to show some statistics about how many hours a user reported on a day. I know I can do it using a query like this:

Report
|> Ash.Query.filter(user_id: 1)
|> Ash.sum(:duration)

However, i was thinking it would be neat if I could abstract this into a nice Statistics module like this:

defmodule Day do
  # does NOT use postgresql, instead uses ETS
  attributes do
    attribute :date, :date
  end

  relationships do
    belogns_to :user, User
    has_any :reports, Report do
      no_attributes? true
      filter expr(date == parent(date)) # < this throws an error (see below)
    end
  end
end

Using the parent in the expression throws this error:

iex(38)> d |> Ash.load(:reports)
** (Ash.Error.Unknown) 
Bread Crumbs:
  > Exception raised in: Timed.Tracking.Report.read
  > Exception raised in: Timed.Statistics.Day.read

Unknown Error

* ** (KeyError) key :__ash_bindings__ not found in: %Ash.DataLayer.Ets.Query{
  resource: Timed.Statistics.Day,
  filter: #Ash.Filter<id in ["b2176029-0419-4ae6-bde1-9eb774041d01"]>,
  limit: nil,
  sort: [],
  tenant: nil,
  domain: nil,
  distinct: nil,
  distinct_sort: [],
  context: %{
    private: %{tenant: nil},
    action: nil,
    data_layer: %{no_inner_join?: true}
  },
  calculations: [],
  aggregates: [],
  relationships: %{},
  offset: 0
}
  (ash_sql 0.2.61) lib/query.ex:83: AshSql.Query.set_context/4
  (ash 3.4.68) lib/ash/query/query.ex:3213: Ash.Query.data_layer_query/2
  (ash 3.4.68) lib/ash/actions/read/read.ex:596: anonymous fn/8 in Ash.Actions.Read.do_read/5
  (ash 3.4.68) lib/ash/actions/read/read.ex:960: Ash.Actions.Read.maybe_in_transaction/3
  (ash 3.4.68) lib/ash/actions/read/read.ex:315: Ash.Actions.Read.do_run/3
  (ash 3.4.68) lib/ash/actions/read/read.ex:82: anonymous fn/3 in Ash.Actions.Read.run/3
  (ash 3.4.68) lib/ash/actions/read/read.ex:81: Ash.Actions.Read.run/3
  (ash 3.4.68) lib/ash/actions/read/relationships.ex:442: anonymous fn/3 in Ash.Actions.Read.Relationships.do_fetch_related_records/5
  (ash 3.4.68) lib/ash/actions/read/relationships.ex:79: Ash.Actions.Read.Relationships.fetch_related_records/5
  (ash 3.4.68) lib/ash/actions/read/relationships.ex:24: Ash.Actions.Read.Relationships.load/4
  (ash 3.4.68) lib/ash/actions/read/read.ex:344: Ash.Actions.Read.do_run/3
  (ash 3.4.68) lib/ash/actions/read/read.ex:82: anonymous fn/3 in Ash.Actions.Read.run/3
  (ash 3.4.68) lib/ash/actions/read/read.ex:81: Ash.Actions.Read.run/3
  (ash 3.4.68) lib/ash.ex:1910: Ash.load/3
  (ash 3.4.68) lib/ash.ex:1864: Ash.load/3
  (elixir 1.18.3) src/elixir.erl:386: :elixir.eval_external_handler/3
  (stdlib 6.2.1) erl_eval.erl:919: :erl_eval.do_apply/7
  (elixir 1.18.3) src/elixir.erl:364: :elixir.eval_forms/4
  (elixir 1.18.3) lib/module/parallel_checker.ex:120: Module.ParallelChecker.verify/1
  (iex 1.18.3) lib/iex/evaluator.ex:336: IEx.Evaluator.eval_and_inspect/3
    (ash_sql 0.2.61) lib/query.ex:83: AshSql.Query.set_context/4
    (ash 3.4.68) lib/ash/query/query.ex:3213: Ash.Query.data_layer_query/2
    (ash 3.4.68) lib/ash/actions/read/read.ex:596: anonymous fn/8 in Ash.Actions.Read.do_read/5
    (ash 3.4.68) lib/ash/actions/read/read.ex:960: Ash.Actions.Read.maybe_in_transaction/3
    (ash 3.4.68) lib/ash/actions/read/read.ex:315: Ash.Actions.Read.do_run/3
    (ash 3.4.68) lib/ash/actions/read/read.ex:82: anonymous fn/3 in Ash.Actions.Read.run/3
    (ash 3.4.68) lib/ash/actions/read/read.ex:81: Ash.Actions.Read.run/3
    (ash 3.4.68) lib/ash/actions/read/relationships.ex:442: anonymous fn/3 in Ash.Actions.Read.Relationships.do_fetch_related_records/5
    (ash 3.4.68) lib/ash/actions/read/relationships.ex:79: Ash.Actions.Read.Relationships.fetch_related_records/5
    (ash 3.4.68) lib/ash/actions/read/relationships.ex:24: Ash.Actions.Read.Relationships.load/4
    (ash 3.4.68) lib/ash/actions/read/read.ex:344: Ash.Actions.Read.do_run/3
    (ash 3.4.68) lib/ash/actions/read/read.ex:82: anonymous fn/3 in Ash.Actions.Read.run/3
    (ash 3.4.68) lib/ash/actions/read/read.ex:81: Ash.Actions.Read.run/3
    (ash 3.4.68) lib/ash.ex:1910: Ash.load/3
    (ash 3.4.68) lib/ash.ex:1864: Ash.load/3
    iex:38: (file)

If I don’t access the parent, doing something like filter expr(user_id == 2) it works fine. Is this not possible?

2

So, we do have the ability to join across data layers, but the parent/1 expression I think is not something that that logic prepares for. You can use manual relationships or calculations to yield those records. Alternatively,

source_attribute :date
destination_attribute: :date

in your particular example.

1 Like
3

Ok I will try that. You suggested a calculation as an alternative but I can’t figure out how that would work?

4 
defmodule GetReports do
  use Ash.Resource.Calculation

  def load(_, _, _), do: [:stuff, you: :need]

  def calculate(records, _, _) do
    # calculate reports here.
  end
end

calculate :reports, {:array, :struct}, GetReports do
  constraints [items: [instance_of: Report]]
end
1 Like
5

Another quick question:

If I create a resource which doesn’t have a datalayer, as an example to do some computations which are nicer to express using the resource syntax, are these garbage collected?

6

They aren’t persisted anywhere so it’s effectively the same as just a pure function that returns structs. So they’d be garbage collected like any other Elixir value.

1 Like
7

Do you think this is something that would be reasonably easy for me to contribute?

I think it would be really nice to be able to refer to the parent resource in no_attributes? relationships in non-datalayer backed resources.

The main reason I’m talking about this is that I have to build up a bunch of statistics based on Reports. A Report belongs to a user and has a certain day but now I want to do things like “what is the total duration of all the reports on a given day”. Which is something that really has nothing to do with the User resource so I don’t want to put it there. Instead I want something like this (I’m leaving out some non relevant things):

defmodule DailyReport do
  attributes do
    attribute :date, :date
    attribute :user_id, :integer
  end

  relationships do
    has_many :reports, Report do
      no_attributes? true
      filter expr(user_id == parent(user_id) && date == parent(date))
    end
  end

  aggregates
    sum :total_duration, :reports, :duration
  end
end

This is much easier than having to write a manual relationship and is super expressive.

edit: I also tried it with the ETS datalayer and it’s not possible to use parent either.

8

I don’t think it would be an easy or even mildly easy contribution, no :slightly_frowning_face:

Did you explore the calculation route?

defmodule MyApp.MyDomain.MyResource.Calculations.DailyReport do
  use Ash.Calculation

  def calculate(records, _, _) do
    Enum.map(records, fn record -> 
      DailyReport
      |> Ash.Query.filter(user_id == ^record.user_id and date == ^record.date)
      |> Ash.read_first!()
    end)
  end
end

That does one query per record to your daily report resource, but it could be optimized as well.

Then its just

calculate :daily_report, :struct, MyApp.MyDomain.MyResource.Calculations.DailyReport do
  constraints instance_of: DailyReport
end