ChatGPT generated code for eval_string alternative

yolo007wizard

Was talking with ChatGPT, is the following code a valid/good replacement for Code.eval_string?

def create_function(name, args, code) do
  ast = Macro.escape(code, [:strict])
  code = Macro.to_string(ast)
  {name, Code.string_to_quoted(code)}
end

This code would run in a template engine of sorts to load function schema/definitions from a json file.

defmodule Engine do
  def start(schema_files) do
    functions =
      schema_files
      |> Enum.map(&parse_schema_file/1)
      |> Enum.map(&create_function/1)
      |> Map.new

    # Store the functions in the `functions` map
    Process.put(:functions, functions)
  end
end

ChatGPT explanation:

In this example, the Macro.escape/2 function is used to compile the code string into an AST. The :strict option is passed as the second argument to Macro.escape to ensure that the code is compiled in strict mode, which means that all variables must be declared before they are used. The Macro.to_string/1 function is then used to convert the AST back into a string of Elixir code. Finally, the Code.string_to_quoted/1 function is used to parse the code string and return a tuple containing the function name and the function itself.

This approach has the advantage of being more efficient than Code.eval_string, since the code is only compiled once and then stored as a quoted expression, rather than being compiled every time it is called. It also has the advantage of being safer than Code.eval_string, since it does not allow arbitrary code to be executed. However, it is a bit more complex to use and requires a deeper understanding of Elixir’s macro system.

5 comments

#chatgpt #eval_string

0 1217 5

2022-12-15 03:50:02 UTC

First 5 of 5 Posts

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

No, this simply does not work. Code.string_to_quoted/1 does not do what it says it does. GPT confuses parsing to AST with compilation, and the Process.put bit is pure fiction.

Post #1

al2o3cr

As is pretty common with ChatGPT, some parts of this are outright hallucinations:

there is no strict option to Macro.escape
Macro.escape does not compile strings into ASTs; if you pass it a string it will return it unchanged
Code.string_to_quoted (no !) does not return an AST, it returns either {:ok, ast} or an error tuple
the result of Code.string_to_quoted has not been “compiled”, it has been parsed into an AST ready to be compiled

But some parts are mostly correct. There is a relationship between Code.eval_string and Code.string_to_quoted! + Code.eval_quoted because they both make use of the same underlying compiler machinery.

Code.eval_string has three steps:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L412-L427

prepare an env with env_for_eval
convert the supplied string to forms with :elixir.string_to_quoted!
evaluate the forms by calling :elixir.eval_forms

Code.string_to_quoted! does only the middle step:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L1000-L1005

while Code.eval_quoted does the first and last steps:

https://github.com/elixir-lang/elixir/blob/0909940b04a3e22c9ea4fedafa2aac349717011c/lib/elixir/lib/code.ex#L827-L832

A minor detail: :elixir.eval_quoted does a tiny bit of argument reformatting before calling :elixir.eval_forms

https://github.com/elixir-lang/elixir/blob/dbc92e4cec969191f0d205ef9dfe2e41ac3be610/lib/elixir/src/elixir.erl#L279-L286

As to your original question, the above analysis should make clear that Code.string_to_quoted! at boot + Code.eval_quoted invoked over and over will be faster than Code.eval_string invoked over and over since it does the string_to_quoted! part once instead of over and over (and otherwise does the same thing).

An even faster alternative would be:

at boot, parse all the AST once with Code.string_to_quoted!
at boot, take all those AST fragments and stitch them together into a module definition
call the functions over and over with apply, at normal speed for compiled code

IMO a bigger question is “what happens when one of these functions is configured with a fatal error, like closing a block with ned?”

Code.eval_string will crash with an exception when you try to run the function
Code.string_to_quoted! called at boot-time will crash with an exception and prevent application startup
Code.string_to_quoted (note no !) will return an error tuple explaining the situation

Your application’s specific needs should help you decide which of these is desirable / acceptable.

Post #2

yolo007wizard

Thanks for those answers!

As a general design idea, I am thinking about how to dynamically load/run code on a general purpose engine thats easy to scale up.

user writes a function in phoenix frontend (using a custom SDK)
function code is stored in spec file as string or file is saved on shared disk somewhere
engine dynamically loads spec file or imports disk file and runs code

ChatGPT also suggested Code.require_file/2 or Code.require_file/3 for dynamic imports

Post #3

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Is your goal to play with ChatGPT, or is your goal to do this dynamic function thing? Because I don’t think interacting with ChatGPT is going to help very much here. It is far more competent than it has any right to be, but it’s still pretty incompetent, particularly if you want to do something off the beaten path.

And accepting and running user defined functions is definitely off the beaten path. If you want a user to write functions, I would strongly suggest that you pick say LUA and run that in GitHub - rvirding/luerl: Lua in Erlang · GitHub and not try to compile actual elixir code from stuff users provide, because compiling running user code as elixir code is a security nightmare.

Post #4