thiagomajesk

thiagomajesk

Hi everyone, I was toying around with macros today and I reached a stagnation point. After trying multiple approaches I started to think that what I’m doing might not be possible, so I’m hoping for some guidance and/ or alternatives.

I’m trying to implement a module that will help me define some counter caches in a table. Here’s the general idea… I have a schema called reactions that stores various kinds of interactions a user might provide for a post:

schema "reactions" do
    field :feeling, Ecto.Enum, values: [:like, :dislike]
    # embeds_one :counter_caches, Cache
end
post_id user_id feeling
1 1 like
1 2 dislike
1 3 like
2 1 like

I expect that for those enum values, the following fields would be generated in the schema: feeling_like_count and feeling_dislike_count. Here’s what I came up with:

defmodule CounterCache do
  import Ecto.Query

  defmacro __using__(_opts) do
    quote do
      import CounterCache

      Module.register_attribute(__MODULE__, :counter_cache_fields, accumulate: true)
    end
  end

  defmacro counter_cache_field(field, opts \\ []) do
    {group, opts} = Keyword.pop(opts, :group)
    {suffix, _opts} = Keyword.pop(opts, :suffix, "count")

    quote do
      name =
        "#{unquote(group)}_#{unquote(field)}_#{unquote(suffix)}"
        |> String.trim_leading("_")
        |> String.to_atom()

      Module.put_attribute(__MODULE__, :counter_cache_fields, {unquote(group), name})
      Ecto.Schema.field(name, :integer, default: 0)
    end
  end

  defmacro counter_cache_field_enum(module, field) do
    quote do
      values = Ecto.Enum.values(unquote(module), unquote(field))
      Enum.each(values, &counter_cache_field(&1, group: unquote(field)))
    end
  end
end

So, the part that I’m stuck at is generating and exposing the query that retrieves the counter cache fields. I expect the query to be something like this:

select
  count(1) filter (where feeling = 'like') as likes
  count(1) filter (where feeling = 'dislike') as dislikes
from reactions
post_id likes dislikes
1 2 1
2 1 0

Here’s what I’ve managed to do so far with this ancillary function:

def __query__(module, fields) do
  quote bind_quoted: [module: module, fields: fields] do
    Enum.reduce(fields, from(module), fn 
      {nil, field}, query -> 
        select(query, [m], filter(count(1), not is_nil(field(m, ^field))))
      {field, value}, query ->  
        select(query, [m], filter(count(1), not is_nil(field(m, ^field) and field(m, ^field) == ^value)))
     end)
  end
end

I was hoping to be able to call a function where I’d pass the source module (where the query will fetch the information) and receive a query that I can use later to update the embed that holds the cached values in the posts table.

Repo.all(Post.counter_cache_query(Reactions))
#=> [
#=>  %{post_id: 1, likes: 2, dislikes: 1}, 
#=>  %{post_id: 2, likes: 1, dislikes: 0}
#=> ]

I had various problems trying to implement this function. I started defining it inside the __using__ macro, and had @counter_cache_fields be empty by the time I tried to generate the query. Also, tried to call the attribute outside the definition and received an error telling me that it cannot be invoked outside of the module.

So, I’m certainly missing something here, I also remembered that Ecto does something similar: ecto/lib/ecto/schema.ex at b69d1085cfd491a859f1be36463afcf4838e4891 · elixir-ecto/ecto · GitHub with the @changeset_fields attribute; so I’m not sure exactly what’s the problem. Is what I’m trying to achieve even possible?

Showing Posts 1 to 10

eksperimental

eksperimental

Not only outside the module, neither inside after the module has been compiled.
That is why you need a helper like Ecto does in the example defining __changeset__/0

Btw, where is the function that uses the stored values in @counter_cache_fields?

ityonemo

ityonemo

If you register the attribute with persist: true you can get at those attributes after compilation using <Module>.__info__(:attributes)

eksperimental

eksperimental

thank you @ityonemo TIL,
I guess I was assuming all attributes were acceded with @, but I can see that was a wrong assumption.

thiagomajesk

thiagomajesk OP

Hi @eksperimental, I’m not sure I understood what you mean, but the attribute @counter_cache_fields could be passed to __query__/2 as the fields param, for instance. I purposefully didn’t include the function because I didn’t know where exactly to put it, but if you care to leave an example I’d appreciate that.

Interesting, I read the docs but it wasn’t immediately clear what this option did tbh. I’ll test that trick but I’d like to understand first if that’s really necessary (since Ecto doesn’t use this, I figure perhaps there’s another way).

eksperimental

eksperimental

Please share a gist with your code.

thiagomajesk

thiagomajesk OP

The code is in the post, that’s how far I got with it… The CounterCache module is the one I intend to use inside schemas to expose the query function I mentioned (which is where we’re at).

eksperimental

eksperimental

how do you call __query__(module, fields)
If you can share the module where you call use ?

eksperimental

eksperimental

Just by reviewing your code (I haven’t run any code yet), i can tell a few things.:

  • Enum.each/2 is used for side effects where you don’t expect any value to be returned. It always return :ok. So the second clause of your counter_cache_field macro will always return :ok. You probably are looking for for and make sure it is valid the desired AST what you return.
  1. def __query__(module, fields) do is a function and you are returning a quoted expression. Are you sure about this? I don’t know how you are calling this. Usually you create these helpers and call them within the macro you are building.

  2. I fail to see how you read the stored counter_cache_fields attributes.
    That is why I am asking you for the code

eksperimental

eksperimental

My advice is: Create a module that stores and read the attribute and does something similar to what you want to do, but without the Ecto layer.
Once you managed to properly register attributes and access them, port that code to interact with Ecto.

thiagomajesk

thiagomajesk OP

Hey @eksperimental, let me try to improve on the latest comment then…

I’m not calling it yet, I just left the example of the API I want to consume to make the use case a little clearer. It should be something like this: Post.counter_cache_query(Reactions), where counter_cache_query/1 is a function that is defined by CounterCache (perhaps) which is then use-d by a hypothetical Post module… If you care to take a look at my previous examples, you’ll see that I mentioned trying to define this function on the __using__ macro only to find that the attribute was actually empty.

What you mentioned makes complete sense, but if you run the code I think it works (and now I’m curious about it as well, I’d have to make some more tests to confirm).

Like I mentioned previously, this is just “pseudo-code”, to make the spec clearer. I was defining counter_cache_query and using this function as a helper in my tests (hence the quoted expression).

This is exactly the part I’m seeking help to accomplish :sweat_smile:. What I’ve tried before was something like this if I recall properly (tried so many different approaches I won’t remember everything right now):

defmacro __using__(_) do
  quote do
    # ...
    defmacro counter_cache_query(module) do
        # CounterCache.__query__(unquote(module), @counter_cache_fields)
    end
  end
end

I think there’s a gap in my macro knowledge that I’m failing to transmit properly, and I believe that even though this makes complete sense to you, I’m not sure what you mean by that. However, I can tell you this: If you test the code, you’ll see that registering the module attribute and defining the Ecto fields in the schema works properly (more or less), so the difficulty is actually accessing the @counter_cache_fields module attribute. A simple example to visualize it would be something like this (except the function should be injected inside the module that uses it):

defmodule Module do
  @counter_cache_fields [{:feeling, :dislike_count}, {:feeling, :like_count}]

  def counter_cache_query(module) do
    Enum.reduce(@counter_cache_fields, from(module), fn ->
      # generate query using field definitions inside attribute
    end
  end
end

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews