Hey y’all, a while ago I had a fun idea for a project which should open possibilities for exploring data in some fun ways, so I got cracking. Since then, I’ve run in to the limits of my understanding of Phoenix and Ecto, so I’m hoping I can entice some folks to help me out…
I’ll take care of all the front-end/UI aspects of this project; I’m looking for help sorting out how to use Ecto and Phoenix correctly!
This has been my go-to project to get my hands dirty with a powerful back-end language: I’m a front-end developer according to my resume but like to play with Real Languages* to know what else is out there. I’m already familiar with Elixir’s syntax as we’ve been using it at work for years.
Project description:
I’m trying to keep track of the songs which a band plays in their live concerts, and then pull out statistics based on the dataset created from each live concert. I’m excited to look for insights and other fun patterns in this data in a variety of ways, including graph representations of the linked-list-based data. I think the site could also be helpful for fans of the bands for which we can collect data!
At the moment the code is in a private GitHub repo; I’m happy to invite collaborators.
State of Affairs
I have some basic CRUD routes set up and have put together a basic interface for it
Linked lists seemed like an appropriate structure for individual sets; I used Arbor to get there (but am not convinced I’m using it right)
I have one significant bug to squash, related to how I’ve used the linked lists
Dependencies are quite out of date! (using Phoenix 1.3.0 yikes) I have a hunch that there are simpler ways to use relationships than how I’ve done it…
Also have a number of features to add which I’m stuck on how it would be implemented on the backend (I’m fine to tackle the UX stuff myself)
Please let me know if this sounds interesting at all to you!
Can’t promise I’ll jump in – too busy. But I can give you a TODO list I inferred from quickly scanning your repo.
Prune outdated branches which I think means all of them besides master – they are all older than it. Don’t hoard branches! If they have such a good idea coded in them, then work tirelessly to merge those to master or delete them 1-2 months after you created them. Be ruthless. We the programmers love hoarding ideas “for later” and 90% of the time we never ever implement them.
Migrate to Phoenix 1.4.x and its project structure ASAP. The closer to the current ways of doing things your project is, the more likely it is for you to attract potential contributors. Do this alone, before inviting anyone else.
Do your damnedest to upgrade to Ecto 3.1.x. It can save you a bit of work if you combine that with the upgrade to Phoenix 1.4.x.
Add Phoenix contexts – basically intermediaries between anything frontend-related (not only Web UI; also API endpoints for the future) and Ecto. Nothing frontend-related should have a clue how is the data fetched or what is its persistent shape. Add a lot of helpers like get_user(id), get_song_with_performances(id) etc.
Modify your controller tests not to assert on textual content. One example in the case of create endpoints: I’d assert on an HTTP 200 response code only (without searching for text in the response content) and check if the new record has been added to the DB.
Add very nitpicky data validation both in Ecto schemas and as DB constraints. Anything that must be limited and can be expressed in SQL and Ecto validators, you should add it. Then you can easily make Ecto aware of the DB-specific constraints by using check_constraint(:column_name, name: "name_of_constraint_in_the_database", message: "your desired error message") in your Ecto schema file’s changeset(...) function(s). This is to avoid runtime exceptions when a DB constraint is violated. Ecto will add an error to your changeset instead.
If you have a business logic module (haven’t inspected your project that closely) you can try and add property tests to it for bonus points. I once coded my heart and soul 3 weeks for a hobby project, checked everything I could rigorously and at least 5 times… and upon starting to add property tests, I found 3 bugs in the first hour. We the humans make mistakes, can’t do much about it.
And finally, if you still have a will to live at this point, try and make an umbrella project out of your monolith project – if you anticipate it could ever grow beyond a core + a web UI. If not, then better don’t, the added initial complexity might be too much and discourage you.
Any suggestions/best practices for the Phoenix connects? Makes sense to me as far as decoupling goes; are there best practices/standards yet?
I think I could use a re-introduction to changesets. The changes that will be made to the data all seem pretty small to me, but I have some convoluted update logic in a few places…
I can definitely see aneed for an umbrella project!
RE: Phoenix contexts, they are usually scoped like so: YourApp.Accounts or YourApp.Orders, and then you add semantically connected functions to them (manipulating more than one schema / model). For example the Accounts context might have functions like get_user but also reset_password and lock_account. And Orders could have new_order and then add_line_to_order and apply_discount_coupon_to_order. Etc. to infinity.
Note that the usual way that Phoenix generates models (when it detects you have contexts or if you specify a context) is YourApp.Accounts.User / YourApp.Orders.Order / YourApp.Orders.LineItem. I however disagree with that way of doing things and I leave the contexts alone in a separate directory, to make it clear that those context modules are just intermediaries that mediate access to the underlying storage system (usually the DB).
To me the clearer naming is YourApp.Schema.User / YourApp.Schema.Order / YourApp.Schema.LineItem – flat structure with a schema namespace, to make it clear those modules are directly bound to the storage mechanism of your app. (And that’s how I do my own apps.)
I don’t want to influence you though. One thing that many Elixir programmers don’t like to hear is, but I am gonna say it anyway, is…
“Do what feels right to you.”
Phoenix is not gonna punish you unless you move its essential files to arcane places – like the router, endpoint, and a few others. Everything else is up to you and you can structure your project as you see fit.
