hst337

hst337

Access optimizations from Hissssst's compiler

Funny thing

is that both map.key and map[:key] are examples of bad decisions in the language design:

The main problem is that the difference between these two ways of accessing a value in map is not just in a

  1. map.key works even when map is an atom !!!
    This is introduced with the ability of language to call functions without parentheses. And this ability actually introduces runtime penalty! So, if we take a look at the erlang term format, we’ll see that map.key is actually compiled to
case map do
  map when is_atom(map) ->
    apply(map, :key, [])
  %{^key => value} ->
    value
  ... ->
    raise KeyError
end

Which handles this extremely rare situation where we actually make a call without parentheses (by my analysis in all of the Ecto, Phoenix, Plug and EctoSQL it happens only once!!!) in the most common construct in the whole language (well, after the function call)! And it actually introduces performance penalties

  1. While map.key is working with structs, map[:key] is not. And it is because Access has is a behaviour instead of a protocol (which was a good decision in the beginning, but just needed some polishing in the compiler).

  2. map[:key] is polymorphic, since it also works with keywords and everything Access-able, while map.key works with structs and maps.

  3. map.key is the same as map.key() and vice versa, parentheses syntax means nothing here. Live with it

The right way to do this

Just use pattern-matching. It is the most powerful and efficient construct in the language. For nested data use Pathex. And if you already have a codebase where everything is dotted, just use optimizing compiler called Tria.

First 10 of 25 Posts Switch mode

cmo

cmo

Come on mate. You can’t tell people to “just” use your experimental compiler instead of vanilla Elixir. That is not the “right way” to deal with a code base where the dots are used.

I think we’re safer just to pattern match ourselves or borrow some code from Membrane.

hst337

hst337 OP

Why do you think so?

cmo

cmo

Risk. Given the choice between a) a minor performance penalty; b) writing code using pattern matchine style; or c) replacing the language compiler with one written by one bloke whose claims about the Elixir compiler and their own replacement are at times disproven and is not battled hardened, you’d be hard pressed to find many people advising option c. The time investment and risk is unlikely to be worth the cost.

hst337

hst337 OP

Which makes key access 2-3 times slower

That’s cool, but it would require refactoring a lot of existing code, which sometimes is not that obvious.

I’d like to hear some real examples about my claims about compilation which were disproven.

That’s true, but it is true for every new technology.

Why is that? I mean, compiler is not 100% ready yet, but is in the state which shows that my ideas around performance optimization are working. It just requires some polishing. As soon as it will be ready, there will be no risks, my aim is to provide as-much-as-possible compatibility with existing compiler.

And I think that it is totally fine to advertise not-yet-ready solution, because it will provide some battle-testing by early adopters (one of whom is a member of already mentioned Membrane project), some useful feedback, and might even help me get some money for my work

sodapopcan

sodapopcan

I guess since it’s a compiler so long as it compiles there is no risk, or does it produce code that could hit runtime exceptions?

That’s an interesting point, I never thought about that! I do feel you would almost have to be trying to write code that would ever hit this scenario, but it sure would be a subtle bug: You happened to pass in an atom instead of a map that also happened to be an erlang module that also happens to have a zero-arity function with the same name as the key you’re trying to access.

dimitarvp

dimitarvp

And here I am, never using map[:key] or map.key at all… :003:

hst337

hst337 OP

Every compiler does.

sodapopcan

sodapopcan

Sorry, terribly worded. I guess I’m asking how different it is from Elixir aside from the optimizations listed. I can always just look at the code.

hst337

hst337 OP

There are several types of things that can be different

  1. Unexpected. I am human, I make mistakes, so I tried to invest a lot of time into making debugging of the compiler as simple as possible.

    • I have made different verbosity levels
    • I made compiler tracing, per-function or per-feature
    • I made automatic github issue submission
    • I’ve made a way for a regular user to peek at what code is generated, but in regular Elixir
  2. Expected. And I’ve divided those into four parts:
    2.1 Compatible. And I am trying to make most of the optimizations compatible as much as possible. So that is compatibility in terms of side-effects ordering, in terms of exceptions, and of course in terms of results returned by functions
    2.2 Incompatible, but invisible. This means that my compiler generates different code from what vanilla compiler does, but nobody actually relies on this behaviour or this behaviour changes in Elixir from time to time. For example, my compiler slightly changes module compilation order, but it doesn’t affect anything, unless the user is relying on unspecified compiler behaviour (like reading .beam files during compilation to affect the result).
    2.3 Incompatible, but with workarounds. This is runtime recompilation with appup or relup (recompile in iex is compatible). I will provide a different interface to these problems, in case anybody will ever be using runtime recompilation in prod with Elixir.
    2.4 Incompatible, but which require Elixir patching. And there’s the only one of them, it is exception formatting. Tria regroups functions into modules, but not in a way a developer originally wrote. So function Module.function/2 can actually become TriaGenerated.Module__function/2. And stacktrace would have this ugly signature instead of the original one. I am just going to introduce the patch to the Elixir and make sure that it is accepted in the upstream, which will allow exceptions formatting.

Plus, there is also this tiny case of Incompatible, but useful when my compiler detects errors that regular Elixir compiler wouldn’t. This is extremely rare and can be found only in cases like

x = %{1 => 2}
case x do
  %{^y => ^y} -> :ok # This clause will never match
  _ -> :error
end
benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey folks I moved these posts out of the other thread because we’re now squarely into a discussion about the readiness or not of @hst337’s compiler.

Thanks!

Where Next?

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91561 914
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New

We're in Beta

About us Mission Statement