Knowledge of generated code from compilation tracers

mhanberg

Expert LSP Core Team

I use the compilation tracers for Next LS to power its symbol/reference database used for definition/references/hover/workspace symbols etc.

Is there a way (or could there be a way) to know if a function/macro reference is from generated code?

Example

Consider the following code

%GenLSP.Structures.Hover{
  contents: %GenLSP.Structures.MarkupContent{
    kind: GenLSP.Enumerations.MarkupKind.markdown(),
    value: """
    # #{module}
      ^ hover on this location

    #{docs}
    """
  },
  range: %Range{
    start: %Position{line: startl - 1, character: startc - 1},
    end: %Position{line: endl - 1, character: endc - 1}
  }
}

With the tracers as they are today, if you were to hover on the above location, you would see the documentation for Kernel.to_string/1 as seen here

Now this particular example is because string interpolation syntax is compiled into something similar to <<"# ", to_string(module)::binary>> (I think the particulars of this is incorrect, but it is the gist of it).

Now, I wouldn’t consider this a huge problem, except that the range for this “reference” overlaps with the text of the variable binding module, which will interfere with using definition/references on that.

Question

So my question to end this is, is there a way the code from the trace is actually present in the source code?

cc @josevalim

Thanks!

PS: The mailing list details itself as the place to open proposals, and GitHub issues are for actual bugs, so I think Elixir Forum is still the place to ask clarifying questions like this. Please let me know if that understanding is incorrect

4 comments

#compiler-tracing #elixir-compiler

2 600 4

2023-09-17 21:01:12 UTC

First 4 of 4 Posts

josevalim

Creator of Elixir

What you could try to do is to set columns: true in the parser options and then see if there is a column in the metadata. IIRC, generated code does not have column information attached to it.

Post #1

mhanberg

Expert LSP Core Team

That doesn’t seem to be the case from what I’ve observed.

Just to confirm, these are the traces generated from the following piece of code

Code

defmodule TracerDemo do
  defmacrop some_macro(list) do
    quote do
      Enum.map(unquote(list), fn str -> String.upcase(str) end)
    end
  end

  def hello(arg) do
    "hello #{arg}!"
    some_macro(["hello", arg, "!"])
  end
end

Traces

Compiling 1 file (.ex)
[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> :start

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:imported_macro,
 [
   end_of_expression: [newlines: 2, line: 6, column: 6],
   do: [line: 2, column: 30],
   end: [line: 6, column: 3],
   line: 2,
   column: 3
 ], Kernel, :defmacrop, 2}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 2], :elixir_def, :store_definition, 3}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:imported_macro,
 [do: [line: 8, column: 18], end: [line: 11, column: 3], line: 8, column: 3],
 Kernel, :def, 2}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 8], :elixir_def, :store_definition, 3}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 1], :elixir_utils, :noop, 0}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_macro, [closing: [line: 9, column: 17], line: 9, column: 12], Kernel,
 :to_string, 1}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 9], String.Chars, :to_string, 1}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:local_macro, [closing: [line: 10, column: 35], line: 10, column: 5],
 :some_macro, 1}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:alias_reference,
 [
   line: 10,
   counter: {TracerDemo, 4},
   last: [line: 4, column: 7],
   column: 7,
   alias: false
 ], Enum}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 10, closing: [line: 4, column: 63], column: 12], Enum,
 :map, 2}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:alias_reference,
 [
   line: 10,
   counter: {TracerDemo, 4},
   last: [line: 4, column: 41],
   column: 41,
   alias: false
 ], String}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:remote_function, [line: 10, closing: [line: 4, column: 58], column: 48],
 String, :upcase, 1}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> {:on_module,
 <<70, 79, 82, 49, 0, 0, 8, 176, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 1, 19,
   0, 0, 0, 27, 17, 69, 108, 105, 120, 105, 114, 46, 84, 114, 97, 99, 101, 114,
   68, 101, 109, 111, 8, 95, 95, 105, 110, 102, ...>>, :none}

[lib/tracer_demo/tracer.ex:3: TracerDemo.Tracer.trace/2]
it #=> :stop

You can note that the Kernel.to_string trace from the string interpolation is emitted as {:remote_macro, [closing: [line: 9, column: 17], line: 9, column: 12], Kernel, :to_string, 1} and the function and alias traces from the macro are emitted with column information as well.

Full code located here.

Post #2

josevalim

Creator of Elixir

Thank you, apparently it was on my mind to implement this, but I never did. I just pushed a commit to main that addresses the Enum case.

For the string one, there is now a special key called from_interpolation. Note from_brackets is also used when foo[bar] becomes Access.get. The full list is here: Macro — Elixir v1.20.2

Post #3