ChatGPT generated code for eval_string alternative

yolo007wizard

Was talking with ChatGPT, is the following code a valid/good replacement for Code.eval_string?

def create_function(name, args, code) do
  ast = Macro.escape(code, [:strict])
  code = Macro.to_string(ast)
  {name, Code.string_to_quoted(code)}
end

This code would run in a template engine of sorts to load function schema/definitions from a json file.

defmodule Engine do
  def start(schema_files) do
    functions =
      schema_files
      |> Enum.map(&parse_schema_file/1)
      |> Enum.map(&create_function/1)
      |> Map.new

    # Store the functions in the `functions` map
    Process.put(:functions, functions)
  end
end

ChatGPT explanation:

In this example, the Macro.escape/2 function is used to compile the code string into an AST. The :strict option is passed as the second argument to Macro.escape to ensure that the code is compiled in strict mode, which means that all variables must be declared before they are used. The Macro.to_string/1 function is then used to convert the AST back into a string of Elixir code. Finally, the Code.string_to_quoted/1 function is used to parse the code string and return a tuple containing the function name and the function itself.

This approach has the advantage of being more efficient than Code.eval_string, since the code is only compiled once and then stored as a quoted expression, rather than being compiled every time it is called. It also has the advantage of being safer than Code.eval_string, since it does not allow arbitrary code to be executed. However, it is a bit more complex to use and requires a deeper understanding of Elixir’s macro system.

5 comments

#chatgpt #eval_string

0 1220 5

2022-12-15 03:50:02 UTC

First Post!

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

No, this simply does not work. Code.string_to_quoted/1 does not do what it says it does. GPT confuses parsing to AST with compilation, and the Process.put bit is pure fiction.

Post #1

Most Liked

al2o3cr

As is pretty common with ChatGPT, some parts of this are outright hallucinations:

there is no strict option to Macro.escape
Macro.escape does not compile strings into ASTs; if you pass it a string it will return it unchanged
Code.string_to_quoted (no !) does not return an AST, it returns either {:ok, ast} or an error tuple
the result of Code.string_to_quoted has not been “compiled”, it has been parsed into an AST ready to be compiled

But some parts are mostly correct. There is a relationship between Code.eval_string and Code.string_to_quoted! + Code.eval_quoted because they both make use of the same underlying compiler machinery.

Code.eval_string has three steps:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L412-L427

prepare an env with env_for_eval
convert the supplied string to forms with :elixir.string_to_quoted!
evaluate the forms by calling :elixir.eval_forms

Code.string_to_quoted! does only the middle step:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L1000-L1005

while Code.eval_quoted does the first and last steps:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L827-L832

A minor detail: :elixir.eval_quoted does a tiny bit of argument reformatting before calling :elixir.eval_forms

https://github.com/elixir-lang/elixir/blob/dbc92e4cec969191f0d205ef9dfe2e41ac3be610/lib/elixir/src/elixir.erl#L279-L286

As to your original question, the above analysis should make clear that Code.string_to_quoted! at boot + Code.eval_quoted invoked over and over will be faster than Code.eval_string invoked over and over since it does the string_to_quoted! part once instead of over and over (and otherwise does the same thing).

An even faster alternative would be:

at boot, parse all the AST once with Code.string_to_quoted!
at boot, take all those AST fragments and stitch them together into a module definition
call the functions over and over with apply, at normal speed for compiled code

IMO a bigger question is “what happens when one of these functions is configured with a fatal error, like closing a block with ned?”

Code.eval_string will crash with an exception when you try to run the function
Code.string_to_quoted! called at boot-time will crash with an exception and prevent application startup
Code.string_to_quoted (note no !) will return an error tuple explaining the situation

Your application’s specific needs should help you decide which of these is desirable / acceptable.

Post #2