Where to put basic Ecto Queries

I have grown up thinking in terms of layers. First, MyApp.Data.Schemas.ModelOne/ModelTwo etc for the data layer. Then, it sounds like your gut is telling you that you need to add some organization for the data access part, but that it doesn’t need to be too crazy. So a simple module in MyApp.Data with the access functions would possibly suffice. If you find that grows too large, then you can break it out to MyApp.Data.Domain1/Domain2 etc.

alias MyApp.Data.Schemas.User

defmodule MyApp.Data.User do
  def get_by_name(username) do
    # query here
  end
  # etc...
end

But definitely keep it in the controller if it isn’t much, and then break out the layering organization as it gets more complex.

I’m still coming from the OO world though, and I’m not sure that if this kind of organization is done in the Elixir/FP world. (I like it though :smile:)

EDIT: FWIW, here is a link to my ib_gib app’s data.ex and data folder. If you notice, this is a separate app from the phoenix app. Phoenix is not your application is a nice elixirconf video that explains some basics of layering, but again it depends on your requirements.

2 Likes

I think most complex systems end up that way, like I have my Helpers that I put the low-level stuff in, but I also have modules for ‘systems’ of data processing that use those other Helpers to do higher level things, and those in turn are used by controllers and channels.

1 Like

Thanks for the feedback guys! I’m also coming from and OO world and have seen the use of good layers in a decent sized application. I know for small things it’s best to just put things in the controller, but for the system I’m porting right now there’s a lot of pieces to it beyond the HTTP API so, most of the Ecto query stuff is going to get used by other pieces of the system.

Another reason I’ve had in the past for having good layering and centralizing database access is that as an app scales the DB can become a bottleneck and you have to be careful about what queries you are running. If you have queries being run in lots of different places, it can make it hard to keep track of what columns are being used heavily and which ones need indexes.

Thanks for the link to the talk, I’ll check that out. I totally agree with the premise.

I have a good idea of how I’m going to organize things now. Thanks!

1 Like

I’ve been using a naming convention so that all the query and save logic is in a separate module named xRepo. For example there is the Policy module, and the PolicyRepo module. If the Policy is an aggregate the PolicyRepo can be take care of querying and saving the associated struct as well.

I think it’s important to avoid the bloat that is caused in Rails type models where the save and query all end up in the same place as regular biz rules.

1 Like

Nice idea. Just out of curiosity, in your xRepo module do you reference the applications Repo directly, or is that a parameter, or are you just returning queries from the xRepo module?

Check out this topic Should we Separate Ecto From Phoenix in Umbrella App? some ideas are shared here

3 Likes

It baffles me that there’s no clear consensus on how to handle this. I assumed I could define these functions in my models/schemas like arel scopes in Rails, but was told that is ‘bad practice’. The ‘best practice’ seems to be undefined so far?

2 Likes

I agree, I was thinking there would have been an established way of doing it, too. That’s one reason I was happy to hear Chris McCord talking about this for Phoenix 1.3. I figured the way they handled it there will probably become the most common pattern, if not the standard. But he hasn’t really given any details about how they are planning to implement moving these queries out of the controllers for Phoenix 1.3, just said that they were going to do it.

1 Like

I have separated out my Phoenix project into an umbrella application. The Phoenix portion no longer contains any dependency on Ecto. The separate app now handles all things related to the database. I have modules that you could think of like a service that does any data manipulation. The Phoenix portion calls these service modules and have no concept of a schema or model. This really cleaned up my code for readability and has allowed me to refactor code more easily and create new features faster. I’d recommend breaking it out for those reasons if not just for a learning experience.

4 Likes

Currently, I reference Ecto.Repo directly in the xRepo module. However, you bring up a good point. It would be worth considering passing it in if it’s likely to change or helpful for testing purposes.

Hi,

I’m having the same problem! What did you ended up using? I tried to put the basic queries in a module and import it on the web.ex. But that didnt seem to work.

Thanks!

What error message are you getting? What is your example code? Etc…? :slight_smile:

model.ex

defmodule MyApp.Model do
use MyApp.Web, :model
import MyApp.Models.Helpers
end

helpers.ex

defmodule MyApp.Models.Helpers do
import Ecto.Query
def get_all_shared(query) do
from q in query,
select: q
end
end

iex
MyApp.Model.get_all_shared says

** (CompileError) iex:13: undefined function get_all_shared/1
(elixir) expanding macro: Kernel.|>/2
iex:13: (file)

Thanks! Not sure what I’m missing. I should be able to just call get_all_shared since I imported it

You can call it from within the module that you imported it to. From what I gather you are trying to call MyApp.Model.get_all_shared() from within iex, which does not exist. If you want to re-export it then you have to defdelegate it as well (or make a __using__ macro, but don’t) so it will be re-exported with that module. :slight_smile:

Hey yea! I thought I should be able to call MyApp.Model.get_all_shared() since I did an iex -S mix. shouldnt that work? thanks!

The code you posted above does not have a MyApp.Model.get_all_shared/0 though, so if you are supposed to have it then you did not show code that defines it. :slight_smile:
This is your code from above:

defmodule MyApp.Model do
  use MyApp.Web, :model
  import MyApp.Models.Helpers
end

defmodule MyApp.Models.Helpers do
  import Ecto.Query
  def get_all_shared(query) do
    from q in query,
    select: q
  end
end

You defined a get_all_shared/2 inside of MyApp.Models.Helpers but you have defined no functions at all whatsoever inside of MyApp.Model. The only thing MyApp.Model has defined in it is use MyApp.Web, :model, which defines some helpers for functions inside of the MyApp.Model module, and it has import MyApp.Models.Helpers, which brings the functions of import MyApp.Models.Helpers into the scope of this module for functions inside of this module to more easily call, but since there are no functions in this module the import MyApp.Models.Helpers line and use MyApp.Web, :model line are both basically no-ops.

1 Like

thanks! sorry im still very new to this! I thought I can call get_all_shared from MyApp.Model since i already imported it without defining a function inside of it. =p my bad :stuck_out_tongue: thanks!

import just brings another modules public functions into the ‘local’ scope. :slight_smile:
To re-export to the outer scope you need to defdelegate each function that you want to re-export. :slight_smile:

3 Likes

thanks! havent heard of defdelegate before will check it out. elixir/phoenix community rocks! :stuck_out_tongue:

Is it better to do the defdelegate in this case or just put the shared basic queries in Repo?