How does Phoenix benefit from hardware caching?

Hmm, maybe this timely tweet about seeing cache misses will be helpful.

Hardware caching could simply mean that the iodata is generally laid out in contiguous memory, meaning that the L3 and other CPU caches will readily have that information on hand because the COU generally pre fetches chunks of contiguous RAM when it accesses an address.

this would be distinct from an object graph way of representing the template information which may contain pointers all over the place.

In this interpretation of hardware caching it’s simply a property that makes any given request fast, it isn’t about caching across requests necessarily.

If you want different caching then you could do something that I do. For example I have a few sub-templates in my main templates that are costly to render (conditionals and large data chunk), I could easily have those stored in ETS for a given input and just clear it out every once in a while (in this case this one chunk is the same for everyone so I’d only need to store it in a cache once and just rebuild it when its data changes). In the general case template rendering to iolists are so fast that it is going to be hard to beat (in the general case a caching system lookup would be slower than running render).

Hardware caching could simply mean that the iodata is generally laid out in contiguous memory, meaning that the L3 and other CPU caches will readily have that information on hand because the COU generally pre fetches chunks of contiguous RAM when it accesses an address.

Interesting. I’ll have to see if I can tell whether the addresses given to writev are contiguous in memory.

It’s complicated by the fact that the size of the specific strings changes how the BEAM treats them. They have to be bigger than 512 bytes (in my testing) to both 1) be refcounted, not copied across processes (64 byte minimum) and 2) not be combined when calling writev (based on “ERL_ONHEAP_BIN_LIMIT”, from what I’ve read)

Maybe I’m just Doing It Wrong™ , but I can’t tell that these addresses are contiguous in memory. Some of the addresses I see using dtrace on a Phoenix app:

(0 bytes): 0x0000000000000000
(0 bytes): 0x00000000b0ac4478
(104 bytes): 0x0000000019f43cae
(106 bytes): 0x0000000019f80d48
(1140 bytes): 0x000000001a003088
(125 bytes): 0x0000000019f45797
(125 bytes): 0x0000000019f45817
(125 bytes): 0x0000000019f45897
(17 bytes): 0x0000000019f420e8
(22 bytes): 0x0000000019f43c98
(22 bytes): 0x0000000019f80af8
(241 bytes): 0x0000000019f45917
(29 bytes): 0x0000000019ec02d0
(3 bytes): 0x0000000019f45794
(3 bytes): 0x0000000019f45814
(3 bytes): 0x0000000019f45894
(35 bytes): 0x0000000019f80b38
(460 bytes): 0x0000000019f455c8
(666 bytes): 0x000000001a003528

I figured that if they’re contiguous, I should be able to take one address, add the number of bytes written from there, and find the start of another address in the list. Eg:

“19f45814” |> Integer.parse(16) |> elem(0) |> Kernel.+(3 * 8) |> Integer.to_string(16)

I don’t find anything that way, but he fact that the first couple are listed as writing “0 bytes” probably means the bytes are rounded, so maybe this is the wrong way to go about it.

It’s a bit more complex. There is a cache hierarchy, L1, L2, L3, and then you have the main memory. In between the CPU and the main memory you have the TLB coprocesor in the CPU to translate virtual addresses to physical addresses.

L1 memory is very small, and there are 2 caches, one for instructions and one for data.

As benwilson512 mentioned, when there is a cache miss, the CPU will ask data to the next cache level, up to the HDD, if it needs to. When the data is found, it is copied to all the other cache levels, but it’s done in chunks. It is called cache lines.

So, if you want to create a program that is cache friendly, you would do it in this way, creating data that is contiguous, so if there is a cache miss somewhere, you will retrieve as much useful information as possible.

Now, back to phoenix, you may have usual code, or you may have precompiled template code. The difference is that the first one executes at run time, while the second one can execute at compile time and store the results so they ready at run time and there is almost no execution of any of your code.

You should remember how erlang and elixir work. If you pass some data, the data is copied around. This is done to isolate the data from the execution.

If you are really really interested in having full control of your data layout, then I would recommend you to code in C or C++. These are the 2 only high level languages that allow you to specify how the data will sit in the memory, and then use all sort of techniques to speed up execution. For example, you can pass references and pointers, so that there are no copies involved anywhere.

The whole point of elixir and erlang is not to have a language where you can tweak everything to squeeze any CPU cycle. The point is to have a highly concurrent and reliable system.

Rust I’ve been playing with lately, fantastic C API support, and it has a very good control of memory (though still not to C/C++ extent, however the next version should fix that). This gets me an idea for a mix module to support rust compilation more easily multi-platform than C NIF’s… Hmm…

Have you seen GitHub - rusterlium/rustler: Safe Rust bridge for creating Erlang NIF functions · GitHub ?

Ooo an interop lib, awesome! I may have to play with that…

Aaaaand it seems unmaintained, does not work with the current version of Erlang/Elixir, is suggesting using the old multirust system instead of the modern rustup (as it in actually requires multirust installed, which cannot happen anymore as multirust removes itself if installed now while saying to migrate to rustup…), etc