Elixir for targeting code generation?

Chat / Discussions
,
1

I have a setup for code generation that I love, but is now moribund. I will describe the aspects that are great and I would like to know what elixir might offer for solving this problem.

My current solution uses the Dart programming language. With dart there are now a few implementations of the language - one for web, one for flutter and another vm for console work. My stuff is all console vm. I say my setup is moribund because all my code is written in Dart 1 and the recent move to Dart 2 has killed it for me. While the language is still awesome IMHO, my m.o. for using it no longer works because version two of the language moved to an ahead of time compilation approach.

I made a decision early on to model items I want to generate code for in the Dart language itself - rather than in json, xml or some data language. My thinking was so often I want to transform models that it did not make sense to even deal with json to dart every time I want to make a change. Plus, by modeling my data in the language I get the intellisense. So, if I’m targeting c++ I model classes, templates, members, statics, consts etc and then at a higher levels of abstraction implementation files, header files, build scripts - etc. The libraries supporting those models, once written don’t change too frequently. The issue with Dart 2 is startup time went from under 1 second to over 8 seconds which killed the workflow. So prior to 2 I could generate 20 files in less than a couple seconds and now its over 10 seconds. Similar issue for testing. The language has support for snapshots which is great for scripts run repeatedly, but does not help my situation because I create my models in the language directly and therefore every change is a “recompile”. Currently there is no real support for “incremental” compilation with the snapshots (e.g. no way to just compile the one script that has changed and patch it in to preexisting snapshot(s)).

Some features I love in current (Dart 1) setup is:

  • Fast turnaround since language was in scripting space - make change, save and less than two seconds all files updated
  • Great support for string interpolation “class ${className} …”. I’ve used template libraries in other languages and I’m ok with them but have no strong attraction to them and don’t mind piecing together all text in code. String interpolation makes it easy.
  • They have this nice syntax called cascades that turns any set of chained method calls into a kind of fluid API automatically. So - a small sample:
        class_('change_tracker')
        ..descr = '''
  Tracks current/previous values of the given type of data. For some
  algorithms it is useful to be able to examine/perform logic on
  current value and compare or evalutate how it has changed since
  previous value.'''
        ..template = [ 'typename T' ]
        ..customBlocks = [clsPublic]
        ..members = [
          member('current')..type = 'T'..access = ro,
          member('previous')..type = 'T'..access = ro,
        ],
  • Excellent libraries for xml, json, yaml etc. One use case is to read xml from protocol schema definitions and generate library support.
  • Excellent error identification (type related) from the IDE before even running the code.

So, I don’t know much about elixir yet and am willing to learn new paradigms if they are good for this problem. What are your thoughts on elixir for this problem?

2

Elixir has code generation built into the language in the form of macros. Elixir supports incremental builds as well. Also, if you do want to pull in a file to generate code, macros can be used for that and you can signal to the elixir compiler that you’re doing so, so that it knows to recompile when that file changes. Pipes (|>) are used for fluid API’s in elixir, but you likely won’t need them if you want to build a DSL.

Dart probably has stricter typing than Elixir. You can use Dialyzer in elixir, but it uses success typing.

1 Like
3

Elixir is a compiled language. Once you have downloaded dependencies it can take a while to compile everything the first time. Once you’ve got the thing built, however, updates happen fairly quickly.

I spend a lot of time with applications in the Elixir REPL, iex where I will make a small change to a file, reload the module in the REPL and try the change out there.

In Elixir, your string interpolation would probably look like like “class #{class_name}”

The built in templating in Elixir comes from the EEx module. The nice thing about EEx is that it takes the template and compiles it into a function (into executable code) so running the template is fast.

Never having seen Dart before I’m not sure what this code snippet means – what you are trying to demonstrate with it.

I know from first hand experience that Elixir parses JSON well. I’ve not used XML or Yaml from Elixir. You might look here: GitHub - h4cc/awesome-elixir: A curated list of amazingly awesome Elixir and Erlang libraries, resources and shiny things. Updates: and GitHub - h4cc/awesome-elixir: A curated list of amazingly awesome Elixir and Erlang libraries, resources and shiny things. Updates:

Elixir is a dynamically typed language. There is support from a tool called Dialyzer to help type-check your code, but you have to run that tool as a separate step. Given your remarks about your desired turnaround time on code changes, I suspect you would find those unsatisfactory.

2 Likes
4

The REPL quick turnaround sounds good and might work for my use-case.

The dart example is a declarative piece of code that defines a C++ class (class_() with a description ( descr ), a C++ template declaration (template<T>), two members (current and previous) both with access read-only ro. Think of it like a DSL, but it is really just cascading calls to functions. What I like is it is kind-of simple like json but vs-code can guide me with intellisense since the methods and properties (class_(...), descr, template, member(...)) all are really function calls that the IDE knows about.

Sounds like I would definitely want to use the Dialyzer, but just not necessarily when using the libraries to define what to generate. Rather use it when writing the supporting objects (class, class member, template, etc).

5

Sensitive readers should look away now and read no further:

#pragma once

#include "<%= fileBaseName %>_Row_fwd.h"
<% baseMetas.each do |m| %>
#include "<%= m.fileBaseName %>_Row.h"
<% end %>
#include "Buffer_fwd.h"
#include "Model.h"

#define RowIncludeHeaders 1
#include "<%= fileBaseName %>_Row_adds.h"
#undef RowIncludeHeaders


<%= openNameSpaceStr %>

using namespace DbApp::Model; using DbApp::NullValReader; using DbApp::NullValWriter; using DbApp::Buffer;


#define RowIncludeDefs 1
#include "<%= fileBaseName %>_Row_adds.h"
#undef RowIncludeDefs


#if IsClientBuild
int32_t	ReadFromResult( Buffer * dbr, RowBaseVec * v );
int32_t	ReadFromResult( Buffer * dbr, RowVecT * v );
int32_t	ReadFromResult( Buffer * dbr );
#endif

#pragma mark Meta

enum PropIdE {
	<%= fields[0].constantName %> = 0, 
<% fieldsTail.each do |f| %>
	<%= f.constantName %>, 
<% end %>
	kNumFields
};

...

Yes, that’s embedded Ruby, embedded in C++ :wink:

1 Like
6

The Elixir tools in VS Code run it automatically on file save, so its results are always available, basically instantly, in the same panel as compiler errors & warnings.

1 Like
7

@sribe

Beautiful. In your previous post it looks like typical template engine fare. Is it similar in elixir?
Based on that post you understand the goal. Are there any additional features in elixir that are really attractive for this space/problem?

8

Elixir has EEx, which is embedded elixir. It allows you to embed elixir into a template file.

class <%= @class_name %> {

}
9

Elixir would be approximately equal to Ruby in terms of convenience for this task, but would execute much faster. That’s assuming keeping the source defs in YAML files as I do now, if you were really ambitious you could use macros to put together a DSL…

1 Like