I’m trying the understand standard ways people organize code in their phoenix applications.
If I have a User module that defines an Ecto schema, and then I have a UserController that lists users by first name, I’m going to have some kind of Ecto query to do that. I’m not wanting to put that into my Controller since that really doesn’t seem like the right task for a controller, and I wouldn’t be able to reuse it in other parts of the application. But I’m not sure where people normally put this sort of stuff. Doesn’t seem like it should go in the User module either.
I’m sure I can come up with a place, but was wondering if there a pattern for this in Elixir applications?
Personally i dont see any overall problems with putting queries in your controller.
However when you do want to reuse queries I take the composable queries ( https://blog.drewolson.org/composable-queries-ecto ) approach.
My main reason for not wanting to put queries in the controller is that I’ve built a lot of decent sized web applications and APIs and I’ve found putting query type logic in the controller to be a bad idea because you often end up needing it somewhere else in the application.
And along those lines, Chris McCord and others at Elixir Conf were talking about Phoenix 1.3 moving away from putting queries in the controller. Which I would agree with, but I’m not sure the way that is normally done in Elixir. Seems like the consensus is not to put it in the module defining the schema.
I like the aproache to put the queries into other module along side the models. But I only do that for queries that are strategics, doing normal all day queries I see no warm to put at controller. I must confess that I don’t like the controller query approach. But It is the easier to be done.
I have grown up thinking in terms of layers. First, MyApp.Data.Schemas.ModelOne/ModelTwo etc for the data layer. Then, it sounds like your gut is telling you that you need to add some organization for the data access part, but that it doesn’t need to be too crazy. So a simple module in MyApp.Data with the access functions would possibly suffice. If you find that grows too large, then you can break it out to MyApp.Data.Domain1/Domain2 etc.
alias MyApp.Data.Schemas.User
defmodule MyApp.Data.User do
def get_by_name(username) do
# query here
end
# etc...
end
But definitely keep it in the controller if it isn’t much, and then break out the layering organization as it gets more complex.
I’m still coming from the OO world though, and I’m not sure that if this kind of organization is done in the Elixir/FP world. (I like it though )
EDIT: FWIW, here is a link to my ib_gib app’s data.ex and data folder. If you notice, this is a separate app from the phoenix app. Phoenix is not your application is a nice elixirconf video that explains some basics of layering, but again it depends on your requirements.
I think most complex systems end up that way, like I have my Helpers that I put the low-level stuff in, but I also have modules for ‘systems’ of data processing that use those other Helpers to do higher level things, and those in turn are used by controllers and channels.
Thanks for the feedback guys! I’m also coming from and OO world and have seen the use of good layers in a decent sized application. I know for small things it’s best to just put things in the controller, but for the system I’m porting right now there’s a lot of pieces to it beyond the HTTP API so, most of the Ecto query stuff is going to get used by other pieces of the system.
Another reason I’ve had in the past for having good layering and centralizing database access is that as an app scales the DB can become a bottleneck and you have to be careful about what queries you are running. If you have queries being run in lots of different places, it can make it hard to keep track of what columns are being used heavily and which ones need indexes.
Thanks for the link to the talk, I’ll check that out. I totally agree with the premise.
I have a good idea of how I’m going to organize things now. Thanks!
I’ve been using a naming convention so that all the query and save logic is in a separate module named xRepo. For example there is the Policy module, and the PolicyRepo module. If the Policy is an aggregate the PolicyRepo can be take care of querying and saving the associated struct as well.
I think it’s important to avoid the bloat that is caused in Rails type models where the save and query all end up in the same place as regular biz rules.
Nice idea. Just out of curiosity, in your xRepo module do you reference the applications Repo directly, or is that a parameter, or are you just returning queries from the xRepo module?
It baffles me that there’s no clear consensus on how to handle this. I assumed I could define these functions in my models/schemas like arel scopes in Rails, but was told that is ‘bad practice’. The ‘best practice’ seems to be undefined so far?
I agree, I was thinking there would have been an established way of doing it, too. That’s one reason I was happy to hear Chris McCord talking about this for Phoenix 1.3. I figured the way they handled it there will probably become the most common pattern, if not the standard. But he hasn’t really given any details about how they are planning to implement moving these queries out of the controllers for Phoenix 1.3, just said that they were going to do it.
I have separated out my Phoenix project into an umbrella application. The Phoenix portion no longer contains any dependency on Ecto. The separate app now handles all things related to the database. I have modules that you could think of like a service that does any data manipulation. The Phoenix portion calls these service modules and have no concept of a schema or model. This really cleaned up my code for readability and has allowed me to refactor code more easily and create new features faster. I’d recommend breaking it out for those reasons if not just for a learning experience.
Currently, I reference Ecto.Repo directly in the xRepo module. However, you bring up a good point. It would be worth considering passing it in if it’s likely to change or helpful for testing purposes.
I’m having the same problem! What did you ended up using? I tried to put the basic queries in a module and import it on the web.ex. But that didnt seem to work.
You can call it from within the module that you imported it to. From what I gather you are trying to call MyApp.Model.get_all_shared() from within iex, which does not exist. If you want to re-export it then you have to defdelegate it as well (or make a __using__ macro, but don’t) so it will be re-exported with that module.
)
