Support both sqlite and postgres in ash application

I’m evaluating migrating an existing application to ash and it’s looking pretty good so far.

However, I got stuck at this: My app currently supports both sqlite and postgres, configured via a compile time setting. (I need sqlite support because my app is also bundled on android, postgres is used on servers)

I cannot figure out how to make that work with ash.
The problem is in the resources. I tried this:

  if App.Config.use_sqlite?() do
    use Ash.Resource,
      domain: App,
      data_layer: AshSqlite.DataLayer
  else
    use Ash.Resource,
      domain: App,
      data_layer: AshPostgres.DataLayer
  end

  # same conditional for the `postgres` and `sqlite` expressions

  actions do
  ...

However, I get a compile error:

    error: undefined function actions/1 (there is no such import)
    │
 12 │   actions do

This is very weird, since when I manually comment out the appropriate use it compiles just fine.

What is happening here? Why does this conditional compilation logic not work with ash?

It used to work just fine when I did that in my ecto repo to configure the appropriate adapter…

:thinking: try this instead:

use mix ash.gen.base_resource MyApp.Resource and then modify it to look something like this:

  # use it in the resources like this
  use MyApp.Resource

  defmodule MyApp.Resource do
    defmacro __using__(opts) do
       data_layer = 
         if App.Config.use_sqlite?() do
           AshSqlite.DataLayer
         else
           AshPostgres.DataLayer
         end

       opts = Keyword.put_new(opts, :data_layer, data_layer)

        quote do
          use Ash.Resource, unquote(opts)
        end
      end
    end
1 Like

Thank you!
This got me one step closer.
However, now I’m getting a similar error a bit later here:

  if App.Config.use_sqlite?() do
    sqlite do
       ...
    end
  else
    postgres do
       ....
    end
  end
    error: undefined function sqlite/1 (there is no such import)
    │
  9 │     sqlite do
    │     ^

This is with use_sqlite?() == false, with true I get the error on postgres do.

Maybe I need some more macro magic? I just don’t understand how this unreachable code path can cause a compile error, that function doesn’t need to be defined in that branch!

I got it working by doing:

  if App.Config.use_sqlite?() do
    import AshSqlite.DataLayer

    sqlite do
      # ...
    end
  else
    import AshPostgres.DataLayer

    postgres do
      # ...
    end
  end

A bit weird, since the use MyApp.Resource should already have imported those definitions, at least for the branch that is compiled.

Anyway, that seems to work and isn’t too bad..

Now ideally I could also get rid of the duplication inside the postgres and the sqlite block.
They will have to be kept in sync and as far as I can tell the options that each support is pretty much identical.

I’m not that familiar with macros, but I’m guessing this could be done via a macro?

Ok, this was easier than anticipated.
I created a macro like this:

  defmacro sqlite_or_postgres(do: code) do
    quote do
      if unquote(App.Config.use_sqlite?()) do
        import AshSqlite.DataLayer

        sqlite do
          unquote(code)
        end
      else
        import AshPostgres.DataLayer

        postgres do
          unquote(code)
        end
      end
    end
  end

And can use it like this:

sqlite_or_postgres do
    table "things"
    repo App.Repo
    # ...
  end

I’m sure this will shoot me in the foot at some point when I need to use a feature that only exist in postgres, but for now it seems to work nicely :crossed_fingers:

Simpler would be this:

 defmodule MyApp.Resource do
    defmacro __using__(opts) do
       {data_layer, extension_to_add} = 
         if App.Config.use_sqlite?() do
           {AshSqlite.DataLayer, AshPostgres.DataLayer}
         else
           {AshPostgres.DataLayer, AshSqlite.DataLayer}
         end

      {table, opts} = Keyword.pop(opts, :table)

       opts = 
         opts
         |> Keyword.put_new(:data_layer, data_layer)
         |> Keyword.update(:extensions, [extension_to_add], &[extension_to_add | &1])

        quote do
          use Ash.Resource, unquote(opts)
          
           postgres do
              table unquote(table)
              repo YourRepo
           end

           sqlite do
             table unquote(table)
             repo YourRepo
           end
        end
      end
    end

You can use the postgres and sqlite extensions multiple times, and can add it as an extension without making it the data layer so that when you have special sqlite or postgres specific things in a resource, you can just do postgres do ... and not include table.

1 Like

Thanks, the extension_to_add part might be useful!

However, I’ve also got references blocks, identity_index_names and custom_indexes etc. in those sqlite_or_postgres blocks.

So with your proposed approach, it seems like I’d have to define all of these inside the use MyApp.Resource, table: ..., etc. That doesn’t seem like a good approach?

Yeah, that’s a good point :thinking:. You could do it your way, and then with the addition of adding both extensions, you could do custom things in postgres and sqlite blocks respectively.

Great, then I’ll go with that. Seems the most flexible and DRY approach.
Thank you very much for the help!