Well, just because I really don’t need all the db connection, queries and commands logic. So it’s more of a architectural/conceptual problem IMO.
I agree, but that’s not the point at all.
This is a good argument, maybe an umbrella project can make it a little bit easier. IDK
And I’m one of them! But I guess Ecto and Changeset are big enough libraries to live on their own.
They do take too far sometimes, but I think this is not one of this times. Elixir standard lib still growing and I don’t have too much problems with that because they have good reasons for it and they explain them to us.
I hadn’t thought of the educational angle, that’s a very good point. It’s natural you raised it, especially because of your background. By splitting the package, one directs new users into better patterns, which is in fact a plus I hadn’t considered. Following the right patterns is easier when the framework design encourages them.
Regarding splitting the database interface into it’s own app, that’s another discussion. Even if I think DB access should occur in the same application, it should never happen directly in the frontend, and there’s an educational statement to be made regarding not importing Ecto inside a controller.
Yes, it’s a very different use case, and I don’t support people using DB queries in their controllers, that’s a horrible pattern. I just don’t know if I feel the need of splitting Ecto because of that amount of unused code.
I think the main difference here is that I think of dependencies “at the module level”: I don’t care about the package. To me the natural unit of code reuse is the module, and if the package has some modules I won’t use, fine. I mostly care about the code I actually use. This is a somewhat lazy position, I know…
Can you publish two hex packages from an umbrella project? I have to look into it.
I’m in favor of big standard libraries, and adding property testing is a clear-cut case of when growing the standard library is correct: it needs to be added to the standard library is it is to be used to test the standard library And in any case, StreamData’s architecture is flexible enough so that it can b extended in future versions.
As José said, you can pass a Plug.Conn for basic forms, but it’s also possible to do exactly what you’re describing by implementing the Phoenix.FormData protocol which is all Phoenix.Ecto does for changesets. So to be clear, we have always supported using forms without ecto/changesets as it has always been protocol driven. I agree a separate changeset lib, or changeset-like lib that could be pulled in with off-the-shelf Phoenix.FormData implementation would be handy, but I also think it’s a non-trivial effort on the Ecto team’s part so the only thing left to do is do all the work
In the web.ex macro’s that are pulled in to things like controllers and views and such, you could add a macro call that scans the scope for aliases or calls to any module that implements Ecto’s Repo or Queries then error out saying don’t use those in the front-end.
Would not catch ‘all’ cases (apply and such) but would catch 99% of them. ^.^
I wonder if we are tackling this from the wrong direction? Instead of trying to take the schema and changesets away from Ecto, we should take the repository and adapters away from Ecto, as the repository is the one that clearly builds on top of the former.
I think this is a capital idea, akin to the ActiveModel/ActiveRecord split––but the result would be useful in many more contexts.
In such a split the embedded_schema terminology becomes even more of a misnomer. In my mind this always should have been just schema, with the string name being optional unless using a persistence repo that depends upon it––at least, for top-level embedded_schemas.
Ecto.Schema with backing types and changesets really is the core of Ecto, and this is one arena where I think a committed, slow-moving, core supported library is more beneficial than a proliferation of 3rd party options that move fast (see the ruby virtus/dry-rb space) because by definition converting information into, out of, and between custom-typed data structures is always going to what defines boundaries between systems, and it’d be nice if every Elixir lib could rely on the same abstraction instead of rolling their own.
The Absinthe.Schema is a good example of this; it seems silly to have to define your own custom Ecto and Absinthe types and schema for every single domain entity in your system.
Not to say that inventing an Ecto.Schema API that is sufficiently generic and extensible enough such that both Ecto.Adapters and Absinthe could target it, while still being useful on its own with just changesets, would be trivial, of course. But man, what a joy it would be to use!
The name string is the smallest difference between the two - the biggest difference is that schema implies lifetime tracking of the struct through the __meta__ field, while embedded_schema does not.
It has been long on my list to build a generic data manipulation/conformance library akin to changesets, but more flexible (primary based on data structures instead of modules and extensible through protocols). I also had a couple rough attempts, but nothing I would be really satisfied with.
I’m adding my voice to the perspective of splitting the storage abstraction and the changesets. As-is, they are too easy to misuse by junior developers, because they feel like they are this thing you’re supposed to create before you do an Ecto query of sorts; they’re not being obviously leveraged as their own separate tool, which they SHOULD, because they are incredibly powerful.
Honestly I’m even OK with them being split and still being API-coupled to Ecto; because then the work can start in earnest in defining a better API and a tool that deserves a name more representative of its value than “changeset”.
@michalmuskala do you have links to code samples and/or musings of some of your attempts?
I’ll be releasing a library this weekend that does just this. I’m just deciding on a name for the data structure it manages: José didn’t like record.
Dave
@Trevoke No, unfortunately, I don’t have anything concrete to show. But I know @wojtekmach was experimenting with something last week, maybe he’ll have something.
@Dave_Thomas that sounds great! I agree, though, that record is not a great name. We already have records in Elixir with a very particular meaning, overloading the term can lead to confusion. Knowing my great naming skills, I’d probably call it data
@michalmuskala (and anyone else interested) here’s the branch: https://github.com/wojtekmach/ecto/tree/wm-split. So far I didn’t encounter any major blockers, just a couple of minor places where there’s unnecessary coupling and I plan to address it on master before any split happens. There are more things to consider, however:
do we have 2 packages ecto, ecto_repo or ecto, ecto_data & ecto_repo (ecto being just a “meta” package)
how to change docs & guides
prepare some git-fu to split it up in a way that preserves history (need to dig up my notes where I split our project into an umbrella)
I’m not actively working on the split but I intend to eventually prepare a formal proposal for this.
You could have a look at Data Division ( https://hex.pm/packages/dd [https://hex.pm/packages/dd] ).
It’s a work in progress, but it does support end-to-end ecto-in-one-app and phoenix-in-another.
I was going to announce it fully, but now that I’m using it for real, I keep seeing little tweaks to make the ecto integration a little easier, so I’m holding back.
I also have a problem with errors: it looks like Phoenix doesn’t see any error list I set, but I’m sure that is tractable.
Dave
And in any case, StreamData’s architecture is flexible enough so that it can b extended in future versions.
