How to produce executable code using yecc?

mfrasca

maybe it’s not the way to go, but this was my initial guess.

I have been writing a parser for a reduced query dialect that I have partially inherited and much expanded, allowing users write things like:

plant where accession.code='2018.0047'

it’s not ready, but the missing intermediate steps are clear, except the final one: how do I have the result executed?

I am targeting as result the quote representation of the equivalent Ecto.Query.from query. for the above example the equivalent as far as I am concerned would be:
from(p in "plant", select: [:id], join: a in "accession", on: a.id==p.accession_id, where: a.code=="2018.0047")

I have been looking into the structures returned by the __schema__ functions, and all looks quite doable, I mean I know how to extract the table name from the modules, and owner and related modules and keys from the association given its name, so let’s assume that my parser does return this value:

{:from, [context: Elixir, import: Ecto.Query],
 [
   {:in, [context: Elixir, import: Kernel], [{:p, [], Elixir}, "plant"]},
   [
     select: [:id],
     join: {:in, [context: Elixir, import: Kernel],
      [{:a, [], Elixir}, "accession"]},
     on: {:==, [context: Elixir, import: Kernel],
      [
        {{:., [], [{:a, [], Elixir}, :id]}, [], []},
        {{:., [], [{:p, [], Elixir}, :accession_id]}, [], []}
      ]},
     where: {:==, [context: Elixir, import: Kernel],
      [{{:., [], [{:a, [], Elixir}, :code]}, [], []}, "2018.0047"]}
   ]
 ]}

how do I get Ecto to execute it?

18 comments

#ecto-query #yecc

0 1802 18

2019-07-03 13:01:53 UTC

Showing Posts 1 to 10

Show Best Posts
Show All Posts (oldest first)
Show All Posts (newest first)

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

The result of quote is not something you execute. Instead of building Elixir AST and then trying to get it compiled, you’d be much better off just dynamically constructing the ecto query data structure itself.

Post #1

mfrasca

I understand your individual words, but I don’t see what they mean. “the ecto query data structure”?

you mean there’s a proper way to produce Elixir code from a yecc parser?

(added this as a question on StackOverflow)

Post #2

lpil

Creator of Gleam

There’s no quick way to transform an abstract syntax tree made ith yecc into something executable.

One possible option would be to write a function that converts the yecc AST into Elixir AST (the data format returned by quote) that does what you want, and then using that to create an Elixir function at compile time.

Post #3

kip

ex_cldr Core Team

Its been a while since I dived into the Ecto source, but my recollection is that at the lowest level, Ecto does actually interpret an elixir (or very close to elixir) AST as it builds the SQL query. Therefore it may be possible to dive into what AST is produced by the Ecto DSL and mirror that. Not a trivial exercise and there’s no guarantee that the AST used by Ecto will stay the same, but I suspect its a reasonable approach to take if that’s what you really need to do.

Post #4

mfrasca

my parser does return values according to the data format returned by quote.
“compile time” is too early. the queries are user input, built dynamically.

Post #5

mfrasca

you don’t remember a few keywords to look for in the code. something to grep?

Post #6

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Then build a function that takes the output of yecc and walks it, calling the relevant Ecto.Query functions along the way to build an ecto query data structure. Trying to “execute” the AST is not something you should do dynamically based on user input.

Post #7

mfrasca

it’s myself, defining the output of yecc. and I’m trying to understand which would be the most obvious choice for this output. I thought that producing an AST, reproducing the quote of a from query was an obvious thing to do. I counted on the ability to unquote my output, and I wonder if the limitation “(CompileError) unquote called outside quote)” is circumventable.

I fail to see any no go in executing an AST based on user input in this case. there’s a yecc grammar validating it mostly. then if it fails, it fails, where’s the problem?

what do you mean by “an Ecto query data structure”?

Post #8

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Elixir code is compiled not interpreted. Dynamically compiling code all the time is not going to perform well, and may lead to unbound memory growth since you’d be generating compilation artifacts all the time.

By an ecto query data structure I mean a %Ecto.Query{} struct, which is what you get when you call the functions:

SomeSchema |> where(foo: ^whatever)

Post #9

mfrasca

ok. this is a no-go, for Elixir.

thank you.

I need to let users insert complex queries, either guided, or typing, editing what was guidedly produced. then I need to execute them. in Python, I have been using pyparsing and ply, and the result is an object which implements the two functions count and select. so I can tell the user how many records would match the query, and which they are. that’s what I’m trying to do in Elixir as well. this need is a very hard requirement.

Post #10

Load more posts (8 remaining)