How to store additional metadata returned by the custom EEx Engine

Hi there, me again. I need the community opinion on the topic combining metaprogramming and Phoenix.

Background

Drab.Live uses the custom EEx Engine to process the template. At the end, it returns {:safe, iodata}, just like the standard Phoenix engine does. But, my engine collects also additional metadata, like where which assign lives. This is needed for the living assign capability, when you poke the assign, it needs to know where to put it.

For the proof of concept I’ve choosen the easiest way: the metadata is kept in the DETS file, operated by Drab.Live.Cache. It works, but I need a smarter way of doing it, as it sometimes causes compilation errors (which may be easly resolved by deleting the cache file, but it is unacceptable).

Phoenix.Template.Engine behaviour

The following is the idea of how the thing works. I have an engine, Drab.Live.Engine, which you need to add to the config.exs to be a default engine for a .drab extension. Engine implements Phoenix.Template.Engine behaviour, which covers compile/2 returning compiled template.

This is an overview of how it works (not actual code):

  @behaviour Phoenix.Template.Engine
  @impl true
  def compile(path, _name) do
    {safe, meta} = path 
      |> File.read!() 
      |> EEx.compile_string(engine: Drab.Live.EExEngine, file: path, line: 1)
    Drab.Live.Cache.save(path, meta)
    safe
  end

The above runs at the compile time, and results with the render functions for the template, stored in the MyAppWeb.MyView. As a side effect, it updates Drab.Live.Cache with the metadata for each template file, which are used in the runtime.

How to do it better?

1) Mimic Phoenix and store all the stuff in the view.
defmodule MyAppWeb.MyView do
  use MyAppWeb.Web, :view
  use Drab.Live.View
end

With this approach, Drab.Live.View would inject functions with metadata directly into the your view. Very convenient, but requires double compilation. And plenty of work to mimic Phoenix.View. Also, you will be forced to put use Drab.Live.View to your View or globally to web.ex, which changes the API (which is OK as Drab is still <1.0.0).

2) Create individual modules per compiled template

Drab.Live.Cache.save knows the template file path. It could create a module like MyAppWeb.Drab.Live.Web.Templates.Users.Form for web/templates/users/form.html.drab, etc. It should be done individually, for each compiled template.

I like this approach, as it is simpler for me, and less invading for the user of the library (no need for additional use ... in the web.ex.

The module would have only one function, returning metadata for the given template.

I’ve made some experiments, and looks like getting the compiled binary from defmodule and storing it in the MyAppWeb.Drab.Live.Web.Templates.Users.Form.beam does the trick.

The downside of this approach is a number of artificial modules.

3) Generate one module, having one function per template

As above, but group the functions returning metadata for a template in the one entity. That would be nice, but as far as I know, there is no way to add a function to the existing module. And Phoenix calls compile for each template separately.

4) No metaprogramming, inject metadata to the html

Drab is websocket based, so we have a connectivity to the browser. Actually, it already injects some data to the generated html (see __drab global var in the JS console). It is easy and tempting to just do it, but this is a wasting of resources. The same data travels from the server to the client, and back again.

5) OTHER(???)
1 Like

What about emitting {iodata, metadata} from the compiled template functions and then have the encoder properly unwrap it and only keep the iodata part, similar to how the default html engine unwraps the {:safe, _} tuple.

You haven’t really said at what point to you need that metadata, if only after the template is evaluated this should work just fine.

4 Likes

Hi Michał,
You mean implementing Phoenix.HTML.Safe.to_iodata for some custom data type, returned from the compiler? That sounds like a brilliant idea, and would save a lot of effort. I am going to give it a try.

This metadata is needed in the runtime, so this should do the trick! Thanks a million.

3 Likes

@michalmuskala, thanks again for the idea. I’ve learned a lot today. Unfortunately, the experiment showed that it won’t be so easy.

Primo, I would need to develop my own version of Phoenix.HTML, and for the obvious reasons I don’t want to. Implementing Phoenix.HTML.Safe.to_iodata and building the own encoder for drab safe iodata is not enough, the bunch of widely used functions (the quickest example is raw/1) depends on phoenix safe htmls.

Secundo, creating the own encoder for html, which is not exactly named Phoenix.Template.HTML, provides to the interesting effects for the html files which are not covered with my custom engine (like the normal html.eex files). They are not {:safe, ..} anymore. That’s because for non-Phoenix.Template.HTML encoders Phoenix falls back to the standard engine. I was close to create an issue for this, but now I think it is intentional.
Again, I could force users to use only my encoder to drabbed files, setting the encoder and engine to something like .htmld.drab, leaving .html.* to the standard encoder, but… I obviously don’t want to :slight_smile:

In summary, I believe it’s not worth to go that way. I am building an extension to Phoenix, not the second Phoenix :slight_smile:

1 Like