Fast EEx, iolist option for EEx engine

suggestion
#1

Phoenix templates are fast because they work with io_lists. The standard EEx engine however does not work with io_lists. I think it would be great if there was a standard library option to use io_lists with standard EEx templates.

I think it would be trivial to implement. Most of what needs doing is replacing a <> b with [a | b].
Compare the standard EEx engine

with the phoenix engine

I see two options:

  • add an option to most of the functions in the EEx module.
    Perhaps something like iolist: true
  • Create a new engine in core called FastEEx.
6 Likes
#2

Interesting proposal!

If I had to guess, I suspect the core team would prefer a new engine to a new option. I suspect they would say it’s already extensible since it allows you to specify the engine, so why introduce another option.

One challenge I expect you’ll get is, why does this need to be in core? Why not in a library? José has mentioned in the past that he wants elixir to be extendable, so that everything doesn’t have to be in core.

From the tips I posted in the section pin, I think answering the following could bolster your argument:

  • Will it be generally useful to a significant proportion of users? In what situations will people choose to use this engine vs the default?
  • Will it substantially improve the ergonomics of an existing feature?
  • How will it impact learning?

These I would considered answered as follows (assuming new engine):

  • Is it something impossible or difficult to do without adding the feature? No
  • Does it fit with the existing feature set? Yes
  • How complex will the implementation be? Simple
  • How might it impact the rest of the code base? Separate engine, so you have to opt into using it. Shouldn’t effect existing code bases.
  • Does it fit with the existing goals or ethos of the project? Yes
4 Likes
#3

This one interesting to answer I think both many and few. Almost any EEx usage could be replaced, but in most case your templates are not the performance bottleneck.
However in most phoenix applications the templates where not the performance bottleneck.

As something that can be discovered later I guess the answer is no.

It definitely could be a library, but it would be such a small one not worth the effort (almost)

#4

What are some situations where you can’t use IO lists and how would a user know when they can and can’t use them?

I can give a couple examples where I use EEx that are perhaps less common:

  • Translation library with interpolated values. For example, you might put this in a template: t("some.translation.key"), which is an EEx function. How well would IO lists work if there are nested template calls?
  • In code generators to create files and also display messages. Here’s an example.
#5

:wave:

I wrote a IOListEngine once after making a similar topic in Do EEx templates compile to iolists or binaries? but never published it since I no longer needed this functionality. The implementation was rather naive but quite fast.

I can clean it up and publish now if there is any interest in an additional eex engine.

3 Likes
#6

What are some situations where you can’t use IO lists and how would a user know when they can and can’t use them?

I don’t think there are such situations …

#7

I’d be interested. I would also happily clean it up and publish to hex. I think that the maintenance overhead would not be very high

3 Likes
Secure files transfers servers in Elixir ?
#8

Couldn’t find the old one, so wrote it again. Would need to add some more tests.

From the benchmark:

Name                                         ips        average  deviation         median         99th %
io_list_engine_benchshort_template       10.91 M      0.0917 μs   ±276.29%      0.0800 μs        0.27 μs
smart_engine_benchshort_template          1.76 M        0.57 μs   ±657.03%        0.40 μs        2.20 μs

Also to_maybe_precompiled_iodata/1 sometimes emits warnings. It was ok with me, but should probably be fixed if it’s to become a public package.

4 Likes
#9

The difference on those benchmarks is quite noteworthy. although I wonder if it is slower to use iolists in some places. i.e. writing to a socket

#10

Writing to a socket is more efficient with iolists in terms of memory, but I’m not sure about time. I would be very surprised if it was slower, though, since every web server seems to be using iolists when constructing responses.

I can write some simple echo benchmarks with gen_tcp …

#11

I’m sure in aggregate it’s still faster, thats why servers do it. I just wondered if it might reduce the difference from 10x

#12

Definitive answer would require testing, but I suspect that it does not reduce the difference–I recall reading that the implementations of the Erlang IO functions use writev

1 Like
#13

I know have a HTMLEngine that I thing would be a good addition so some more core libraries. I guess that is a slightly different proposal to this one. Am going to think on it a bit but I definitely believe a general purpose html engine would be valuable. Not least because it has security implications. perhaps I need a project called eex_html.

1 Like
#14

Just to be clear, the main difference from phoenix_html's html engine is the absence of functions relying on plug’s conn? Or is there something else as well?

#15

It’s worth mentioning EEx is a general purpose embedded language. I would be surprised if an HTML specific variant were permitted in core.

#16

Yes removing the conn specific functions. I need to double check but I think some functions, possibly form helpers rely on ecto changesets as well.

I am trying to finish off I started here. https://github.com/phoenixframework/phoenix_html/issues/210

No I am the person willing to do the work.

1 Like
#17

They do not. There is a protocol (Phoenix.HTML.FormData) sitting between them. :slight_smile:

4 Likes