Do EEx templates compile to iolists or binaries?

Continuing the discussion from Performance - Best practices?

If they are iolists, then why do function_from_file in eex return binaries? https://github.com/idi-ot/eex_test

iex(1)> EExTest.fortune_html [a: "a", b: "b"]
"<!DOCTYPE html>\n<html>\n  <head>\n    <title>Fortunes</title>\n  </head>\n  <body>\n    <table>\n      <tr><th>id</th><th>message</th></tr>\n      \n        <tr><td>a</td><td>a</td></tr>\n      \n        <tr><td>b</td><td>b</td></tr>\n      \n    </table>\n  </body>\n</html>\n"

I would’ve expected something like this

iex(3)> EExTest.custom_fortune_html [a: "a", b: "b"]
["<!DOCTYPE html>\n<html>\n  <head>\n    <title>Fortunes</title>\n  </head>\n  <body>\n    <table>\n      <tr><th>id</th><th>message</th></tr>\n",
 ["<tr><td>", "a", "</td><td>", "a", "</td></tr>", "<tr><td>", "b", "</td><td>",
  "b", "</td></tr>"], "    </table>\n  </body>\n</html>\n"]

Some tests

# render small list

iex(6)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([a: "a", b: "b"]) end) end)
{480068, :ok}

iex(7)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([a: "a", b: "b"]) end) end)
{279135, :ok}
# render empty list

iex(10)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([]) end) end)
{144888, :ok}

iex(11)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([]) end) end)
{74700, :ok}
# render bigger list

iex(12)> data = Enum.map(1..1000, fn i -> {i, i} end)

iex(14)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.custom_fortune_html(data) end) end)
{277821, :ok}

iex(15)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.fortune_html(data) end) end)
{1078125, :ok}
1 Like

I think it depends on the specific engine you use. I think the SmartEngine concats a big binary blob (should be an iolist in my opinion, but someone else could always make a new engine to do so). EEx itself is just the parser stuff.

2 Likes

Oh, I see. I meant the default one. I wonder if the benchmarked apps use any other engine that emits iolists? That’d be a considerable improvement in IO throughput, I think.

Is there any engine that compiles templates to iolists?

1 Like

Honestly I’ve not actually looked… ^.^;

1 Like
iex(2)> EEx.eval_string "foo <%= bar %>", [bar: "baz"], [engine: IOListEngine]
[[[] | "foo "] | "baz"]

Yeah, it doesn’t seem too hard to write one. That’s a naive implementation where I just replaced empty strings with empty lists and string concatenation with list building.

2 Likes

SmartEngine concats a big binary blob

Yes, that’s what it does. https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/eex/lib/eex/engine.ex

But phoenix uses Phoenix.HTML.Engine which does use iolists. https://github.com/phoenixframework/phoenix_html/blob/master/lib/phoenix_html/engine.ex

4 Likes

Ah good to know, I ‘thought’ it did but was not certain. :slight_smile:

1 Like

But they are still slower …

# render small list

iex(1)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([a: "a", b: "b"]) end) end)
{288843, :ok}
iex(2)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([a: "a", b: "b"]) end) end)
{480979, :ok}
iex(3)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.phoenix_fortune_html([a: "a", b: "b"]) end) end)
{384202, :ok}

# render empty list

iex(4)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([]) end) end)
{142473, :ok}
iex(5)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.phoenix_fortune_html([]) end) end)
{120085, :ok}
iex(6)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([]) end) end)
{74349, :ok}

# render bigger list

iex(7)> data = Enum.map(1..1000, fn i -> {i, i} end)

iex(8)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.custom_fortune_html(data) end) end)
{293852, :ok}
iex(9)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.fortune_html(data) end) end)
{1111719, :ok}
iex(10)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.phoenix_fortune_html(data) end) end)
{385075, :ok}
1 Like

Are you doing everything it does? They do things like auto-escaping of text so no html injections and such, lookups for values in different places, etc… etc…

1 Like

That happens at compile time, doesn’t it?

Reading through phoenix_html again, I see that some of the checks are done at runtime though, yeah.

1 Like

How would it? Say some user input is in a variable that is put into the template, it needs to escape that, thus a runtime call, it has all kinds of checks all over (because, for example, it shouldn’t escape that user variable if it is wrapped in a :raw 2-tuple).

1 Like

I wonder now if there would be any benefit from an io list engine, which would work similarly to phoenix html engine, but skip all this escaping stuff. For example, for generating m3u8 playlists.

2 Likes

You should make a simple IOEngine for that and publish it. :slight_smile:

4 Likes