Getting basic information of a elixir project from GitHub

Questions / Help
1

Hi, Please consider you have a project in GitHub and before running and getting it with mix deps task you want to get some basic information like what is the version and name of this library from mix.exs file.

So regex is not a good way because some people put version as global variable like @version, hence I need to run whole the mix.exs and get the version and name from it.

What is your suggesting to get mix.exs as string from GitHub and compile it? For example, and get the name and version from it or another information before installing it.

I do not know Code.eval_string(code) is useful for me and safe? All the examples in the document is about a function or operation, but not about a full module:

{:module, MishkaDeveloperTools.MixProject,
 <<70, 79, 82, 49, 0, 0, 11, 156, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 1, 5,
   0, 0, 0, 25, 38, 69, 108, 105, 120, 105, 114, 46, 77, 105, 115, 104, 107, 97,
   68, 101, 118, 101, 108, 111, 112, 101, 114, ...>>, {:package, 0}}

After evaluating the module, I do not know how to get version for example:

MishkaDeveloperTools.MixProject.project()

Thank you in advance

2

Code.eval_string is not safe. I’d suggest Code.string_to_quoted

1 Like
3

Thank you, it converts AST, right? So after that, how can I run this module with a random name because it is possible there is a module same name.

Or use something Macro.postwalk ?! To get version and name? But without running this module how can get version for example, the developer load it with some names as global variable like @version

4

You can just travese AST with functions provided in macro. You should look for field version in project funcion. If it contains, @version, than look for @version attirbute. If it contains something else, than there’s no safe way to fetch a version without executing code defined in this module

1 Like
5

Hi, I want to get some value from a module which is AST, and why I want to do this you can see this post Getting basic information of an elixir project from GitHub.

for example I have something like this:

code = """
defmodule MishkaDeveloperTools.MixProject do
  use Mix.Project
  @version "0.0.7"

  def project do
    [
      app: :mishka_developer_tools,
      version: @version,
      elixir: "~> 1.13",
      name: "Mishka developer tools",
      elixirc_paths: elixirc_paths(Mix.env()),
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      description: description(),
      package: package(),
      homepage_url: "https://github.com/mishka-group",
      source_url: "https://github.com/mishka-group/mishka_developer_tools",
    ]
  end
end
"""

for example, I want to get the values of version and elixir and app, so I used Macro.postwalk and I do not know it is the right way to do this:

{:ok, ast} = Code.string_to_quoted(code)
ast
|> Macro.postwalk(fn
......
end)

When I print this, I can see something like this:

{:defmodule, [line: 1],
 [
   {:__aliases__, [line: 1], [:MishkaDeveloperTools, :MixProject]},
   [
     do: {:__block__, [],
      [
        {:use, [line: 2], [{:__aliases__, [line: 2], [:Mix, :Project]}]},
        {:@, [line: 3], [{:version, [line: 3], ["0.0.7"]}]},
        {:def, [line: 5],
         [
           {:project, [line: 5], nil},
           [
             do: [
               app: :mishka_developer_tools,
               version: {:@, [line: 8], [{:version, [line: 8], nil}]},
               elixir: "~> 1.13",
               name: "Mishka developer tools",
               elixirc_paths: {:elixirc_paths, [line: 11],
                [
                  {{:., [line: 11], [{:__aliases__, [line: 11], [:Mix]}, :env]},
                   [line: 11], []}
                ]},
               start_permanent: {:==, [line: 12],
                [
                  {{:., [line: 12], [{:__aliases__, [line: 12], [:Mix]}, :env]},
                   [line: 12], []},
                  :prod
                ]},
               deps: {:deps, [line: 13], []},
               description: {:description, [line: 14], []},
               package: {:package, [line: 15], []},
               homepage_url: "https://github.com/mishka-group",
               source_url: "https://github.com/mishka-group/mishka_developer_tools"
             ]
           ]
         ]}
      ]}
   ]
 ]}

After that, I tried to get project function like this:

Macro.postwalk(ast, fn
  {:defmodule, _line ,[_aliases,[do: {:__block__, _, [
    {:use, [line: 2], [{:__aliases__, [line: 2], [:Mix, :Project]}]},
    {:@, [line: 3], [{:version, [line: 3], ["0.0.7"]}]},
    {:def, [line: 5], block}
  ]}]]} -> IO.inspect block
  v -> v
end)

It is hard code and I do not know what is my user mix.exs file is? So I tried to delete some lanes like:

{:use, [line: 2], [{:__aliases__, [line: 2], [:Mix, :Project]}]},
{:@, [line: 3], [{:version, [line: 3], ["0.0.7"]}]},

but the pattern is not matched, and always the v pattern is run

Macro.postwalk(ast, fn
  {:defmodule, _line ,[_aliases,[do: {:__block__, _, [{:def, [line: 5], block}]}]]} -> IO.inspect block
  v -> v
end)

I think I don’t have enough understanding about the AST, how can I do it and get another lane in my source because if I have something like this: version: @version, I need to back and get @version "0.0.7" for example

Thank you in advance

6

I could be able to get values with this:

{:ok, ast} = Code.string_to_quoted(code)
Macro.postwalk(ast, fn
  {:def, _line, [{:project, _, _}, [do: fun]]} -> {Keyword.get(fun, :app), Keyword.get(fun, :version)}
  v -> v
end)

But how can access just this tuple? {Keyword.get(fun, :app), Keyword.get(fun, :version)} and skip the others?

7

The AST of a keyword list is not a keyword list by itself, so you cannot use Keyword to access things. AST is an abstraction of written code, and not of the data represented by that code. You’ll need more manual approaches for filtering out the information you need.

A rather naive appraoch to getting to the information you seek would be like this:

{_ast, acc} =
  Macro.postwalk(ast, %{version: nil, attributes: %{}}, fn
    {:@, _, [{name, _, value}]} = ast, acc when is_atom(name) and not is_nil(value) ->
      {ast, put_in(acc.attributes[name], value)}

    {:version, {:@, _, [{name, _, nil}]}} = ast, acc ->
      {ast, Map.put(acc, :version, {:attribute, name})}

    {:version, value} = ast, acc ->
      {ast, Map.put(acc, :version, value)}

    ast, acc ->
      {ast, acc}
  end)

acc
# %{attributes: %{version: ["0.0.7"]}, version: {:attribute, :version}}

However you’d want to be more cautious as currently any later keyword list with a :version key would overwrite the returned result.

What you’re doing here is very brittle, as you’re trying to interpret code without running it, which can easily break by someone not sticking to conventions closely.

1 Like
8

Thank you, but do you have any suggestion? Because I need to get some basic information about the dep they want to install and after that I pass this operation to Mix deps.get

9

No. There’s no safe way to do that for all possible mix.exs files. Mix itself doesn’t use versions for git dependencies at all, it uses branches or git sha’s.

1 Like
10

Dear @LostKobrakai, do you have any tutorial especially in elixir to learn work with AST, I am thinking if I had no access to internet I could not solve this problem! I have read Metaprogramming Elixir: Write Less Code, Get More Done (and Have Fun!) by Chris McCord book

11

https://elixirschool.com/en/lessons/advanced/metaprogramming/
https://hexdocs.pm/elixir/Macro.html

1 Like