Calculus - New data types, private, immutable fields and smart constructors

Your Libraries & OS Mentoring Libraries
, , , ,
1

Hello guys! I implemented another way to create new data types (which are not structs or records) in Elixir. Please check out readme file if you are interested, everything is described in detail with examples:

Thanks!

7 Likes
2

Oh hey, this looks awesome!

Somebody can say ā€œhey, this is not a constructor function, itā€™s a syntactic sugar for value, literal Elixir termā€. But anyway, this is expression and value of this expression is Elixir struct of URI type. For simplicity Iā€™ll call this thing as default constructor . And this default constructor is always public. Indeed, you can write in any place or your program something like this:

I so agree, it is, by definition, A Constructor of the type.

This is mostly caused by elixir being deficient in its typing system, which comes from it modeling after erlang instead of trying to fix that bug in erlang.

And then Full Lambda Calculus! Church Encoding! Lol. As Iā€™m reading through Iā€™m curious when church encoding of integers will come up, or the various combinators. ^.^

This looks like a data driven DSEL I saw in Elixir, though function wrapping.

iex> s0.(:pop, nil)
** (RuntimeError) For value of the type Stack got unsupported METHOD=:pop with SECURITY_KEY=nil

Heh, cute, Iā€™m guessing you are using a make_ref() for the security key as they are unique references in the beam world? Iā€™ve not looked at the code yet.

Lambda types are inferior in performance to classical data types like records or structs. I

Yep definitely figured that at the start.

  • Ī»-type constructors and setters ~ 2 times slower then default constructors and setters for structs
  • Ī»-type getters ~ 6 - 12 times slower then pattern matching on structs (but this is still pretty nice performance)

Not ā€˜asā€™ bad as expected though, thatā€™s within the realm of usability considering structs are raw beam codeā€¦

You can run benchmarks with mix bench command in terminal

Ooo, my favorite things! ^.^

At the moment we canā€™t properly use Elixir protocols with values of Ī»-types (because of the same reason). I have couple ideas about it and maybe will fix it.

I have a couple of ideas for it to work with my ProtocolEx library though, hmmā€¦ If only Tuple Calls werenā€™t removed from the BEAM! That still bugs me to no endā€¦ >.>

Internal state of value of Ī»-type is vulnerable for reading (not writing!) through Function.info/2 core function. At the moment I donā€™t know how to fix it:

Eh, there are ways to but it would slow down the implementation more so not sure it is worth itā€¦

Itā€™s possible to read internal state using this function, but itā€™s still impossible to create new corrupted value of Ī»-type based on this internal state. So all immutable and private data is still really immutable and all values of Ī»-type are still valid and safe.

Ehhhh, actually you canā€¦ ^.^;
Iā€™m leaving work right now or Iā€™d show you how you can break this. :wink:
Ping me tomorrow if you are curious, but in essence you can create almost any internal beam type with :erlang.binary_to_term, including picking it apart with the inverse.

This is a fun looking project though, no church encoding, just lambda encapsulation though, lol. ^.^

6 Likes
3

Thanks for review :slight_smile:

No, it is just 64 random bytes converted to atom and inlined to all methods and private expressions in compile-time:

I think pattern matching on inlined atom is the most performant way how I can reach desired behaviour

Iā€™ll check it out :slight_smile:

Oh, this is probably real if you know binary format of erlang terms) And analysis of beam bytecode of the module is real thing as well.
But anyway, this is pretty hard if we compare it with usage of default constructor for pattern mathcing or updating internal data :slight_smile:

Yes, but I got inspiration from Church Encoding - itā€™s soo good thing. Gives feeling that you have the POWER hahaha

2 Likes
4

I really do not recommend this as BEAM have limit of atoms (1_048_576 by default) and this could cause problems in large scaling.

defmacro __using__(_) do
  quote location: :keep do
    import Calculus, only: [defcalculus: 2]
  end
end

I think itā€™s bad practice to add extra use if only import is called.

Consider using this:

defmacro calculus(opts) do
  quote bind_quoted: [opts: opts], location: :keep do
    {opts[:state], opts[:return]}
  end
end

instead of:

defmacrop calculus(state: state, return: return) do
  quote location: :keep do
    {unquote(state), unquote(return)}
  end
end

defmacrop calculus(return: return, state: state) do
  quote location: :keep do
    {unquote(state), unquote(return)}
 end
end

Generic code could be imported instead.

Here is my proposition how it could look like:

defmodule Example do
  use Calculus, state: user(id: id, name: name, balance: balance)

  defrecordp user([:id, :name, :balance]), default_return: {:literal, :ok}

  # no need to change state
  calculus get_name([name: name], return: name)
  calculus set_name([], arg: new_name, state: new_name)
end

defmodule Example2 do
  use Calculus, state: sample(sum: sum), default_return: :state

  defrecordp sample([:sum])

  calculus add([sum: sum], arg: integer, when: is_integer(integer), state: sum + integer)
  calculus get([sum: sum], return: sum)
  calculus increment([sum: sum], state: sum + 1)
end

In my example use Calculus should do:

  1. Set @before_compile module attribute
  2. Register accumulate module attribute
  3. Import generic functions
  4. Generate security key

calculus macro accepts 2 arguments:

  1. Bindings (inspired by Ecto.Query API)
  2. Options:
    a) arg: passed extra argument (we could optionally add args option here as well)
    b) return: returns binding or literal
    c) state: changes state
    d) when: defines a guard(s)

Finally @before_compile callback would finalize all defined calculuses

Please let me know what do you think about it.

3 Likes
5

This is the same cost of a single atom per type as with structs or records, so probably not a problem :slight_smile:

5 Likes
6

Iā€™d say just use make_ref, it already makes a beam-wide unique value and itā€™s fast. :slight_smile:
Although youā€™ll need to make it in the construct call then, it canā€™t be put in a module (as itā€™s unique, itā€™s not reloadable without term_to_binary/binary_to_termā€™ing it).

Itā€™s basically like Elixirā€™s Protocolā€™s, except it works with matchspecs, in addition to a variety of other features such as controlling ordering, various types of fallbacks, can even write compile-time tests to ensure that the implementations implement it properly. :slight_smile:

Very true. ^.^

You know, a full proper unique type that could not be broken apart or anything would be a NIF resource. :wink:

Hehe, raw lambda work is so much fun, itā€™s like a puzzle on how to encode so many things. ^.^

Eh, but itā€™s only once per ā€˜moduleā€™ so not really an issue. A make_ref ā€˜perā€™ constructor wrapper would work very well though!

1 Like
7

I donā€™t think that make_ref function call is faster then inlined atom literal, pattern matching on atom literals is extremely fast thing :slight_smile:

Anyway, I donā€™t know how I can use make_ref in runtime for encapsulation, because I need thing which is known by value of Ī»-type and which is known by method (just function in module, this eval private expression). There are actually 2 checks:

8

Can I write NIFs in Haskell? :grinning:

9

As are REFā€™s, they are just essentially just boxed integers, so itā€™s a single extra unboxing cost (which will likely to be optimized out anyway because identical pointers).

It would be saved ā€˜intoā€™ the closure itself. :slight_smile:

As long as Haskellā€™s GC doesnā€™t get in the way and Haskell can write C-style dynamic libraries with C-style calls.

1 Like
10

Way out of my depth here as Iā€™ve only been learning the basics of Haskell however I asked myself the same question about writing NIFs and I through a bit of googling I found an interesting package for writing Erlang nodes in Haskell. Iā€™m a bit too early on to really try it out but if you are looking at trying to bring the two together it might be worth looking into as an alternative to NIFs http://hackage.haskell.org/package/hinterface

1 Like
11

But these code examples are not the same things. Your example is less performant because there is call of access protocol in runtime. And itā€™s also less safe because something like calsulus(state: state, returnnnnn: return) will survive compilation. And the worst thing will happen in runtime - access protocol will return implicit nil for this value, and consequences of this are unpredictable, depends on other code - sometimes it can behave correct, sometimes can behave incorrect, sometimes can raise exceptions.

So my code is just ā€œinline implementationā€ of named arguments (which not exist in Elixir by default). I did another library for this, but donā€™t want extra dependency just for one expression:

About other your suggestions about syntax sugar - itā€™s really sugar, question of preferences in design. Maybe it make sense for someone, maybe not, I donā€™t know. I thought about DSL with type, private, public, immutable and method keywords - to avoid boilerplate and auto-generate at least getters, maybe some setters. But finally just decided to make interface as much simple and explicit as possible, with smallest possible amount of abstractions. If I, or someone else needs more high-level DSL - he can build it on top of my library pretty easy, because interface is straightforward.

2 Likes
12

Right, as you said itā€™s just code example - not prod version. You can simply use &Keyword.fetch!/2 instead. Also itā€™s easy for just 2 arguments - if you take look at my proposition with 4 arguments you will end up with way too much code.

Simple for you does not needs to mean simple for others and for sure same goes to my old code. For example your defcalculus macro takes more than 100 lines - thatā€™s way too much (take a look at credo library). Iā€™m not someone pro who decides what things are good or bad practices. I simply take a look at other libraries. When Iā€™m writing code I think if I will understand it even after x years. Can you quickly say what this code is doing?

case left do
  [{:when, ctx0, [e | es]}] -> [{:when, ctx0, [e, key | es]}]
  [e] -> [e, key]
end

For sure itā€™s really easy example and most probably you would know it. Now here goes beginner and see: e and es which says exactly nothing. Itā€™s just code style, sugar, but in matter especially when you are working in team.

defmacrop construct(state) do
  quote location: :keep do
    fn :new, @security_key ->
      calculus(state: unquote(state), return: :ok)
    end
    |> eval(:new)
  end
end

If I understand correctly this macro could be simply imported and there is no need to add it inside other macro. For sure one small macro in bigger macro is not a problem, but itā€™s not the only one you have in it. Even splitting it into few helper functions would make it much more readable.

Simply look at end of file:

        end
      end
    end
  end
end

it explains everything. :smile:

1 Like
13

This isnā€™t only problem. AFAIK functions arenā€™t GCed either, so it can blow up anyway as this extensively use Church structures.

14

Functions are statically compiled, they are ā€œGCā€'d when the module is replaced or removed. A Closure on the beam is just a function pointer, like &Blah.bloop/2 and an environment mapping (positional internally). There are no GC issues there.

2 Likes
15

This is very interesting! :slight_smile:

2 Likes
16

Here you can find one more example of usage, abstract Maybe type which implements Functor, Applicative and Monad behaviours

2 Likes
17

Lol, nice building on it. ^.^

1 Like