Comparison to Django / Django Rest Framework?

I’m wrapping up a Django/DRF based intranet-only CRUD application. I chose Django because I’m familiar with its ORM, architecture, deployment and of course quirks. I chose DRF because I it’s one of the easiest ways in Django-land to present a CRUD REST API, and I wanted to use JS-based forms rather than the native Django ones. Performance doesn’t really enter into it as the data volume is low and user count is in the single or double digits :wink:

I am relatively happy with the result. Using a mostly declarative language, I get easily-generated REST APIs with CRUD functionality, validation, nested relations*, custom queries and so on. This is because DRF comes with built-in tight-coupling with the Django ORM if you want to use it, and saves you having to redefine the fields types and validation all over again.

* However, getting nested relations updates to work was a major PITA. DRF doesn’t support those out of the box, since a safe general behaviour can’t be easily defined, so the instruction is to subclass the Serializer and do whatever you want in the update and create methods.

Unfortunately, I found the internals of DRF are really convoluted and I’ve been banging my head on various edge cases for a long time, which has left a bitter taste in my mouth. I’m done now, and reasonably happy with the end-user functionality (it works!), but for the next one I would like to consider other options.

I haven’t really used Ecto or Phoenix for a CRUD application, but I’ve read a lot about them, tried out sample projects, and I’m monitoring the progress of the projects.

From this superficial experience, I would like to point out a few things:

  • Ecto’s built-in fields are limited, but you can augment those yourself. There are already libraries out there with common fields. You are a bit dependent on Postgrex because adding custom fields to Postgrex seems a bit more involved.
  • Echo Changesets (and schema less change sets?) should provide most, if not all, of the functionality required to do app-level conversion and validation from an external JSON document to a native Elixir struct, then updating or creating the entries in the DB.
  • However it’s not clear to me if there is a need for another library/framework to more easily plug everything together, including converting validation errors to HTTP 400 with nice error messages etc.
  • Phoenix doesn’t provide authentication/signup support so an external library must be selected.
  • Phoenix/Plug can handle file upload, but leaves the details up to you. An external library must be selected to store and serve files.
  • Phoenix doesn’t have any email functionality, so an external library must be selected (though the guides do suggest Bamboo for this.
  • Django’s migrations automagically work most of the time, saving a huge amount of time. Ecto has migrations but you have to write them on your own.
  • Django’s built-in admin is a very nice bonus while developing, while with Phoenix an external library must be selected.
  • Django lacks a javascript/static assets pipeline, only providing you with a “collectstatic” command. You must select an external library or roll your own. Phoenix has brunch and has enough hooks to let you use your own.
  • Django is locked in the “request/response” cycle. There is some ongoing work with Django Channels to surpass that though. Phoenix has first-class supports for websockets, including a javascript library.
  • Django does a lot of work introspecting models in order to augment parts of the dev experience, with sometimes good and sometimes bad results.
  • Django has pervasive autoreload, but no live-reload. Phoenix has autoreload and live reload.

(Have I missed something?)

I’m not going anywhere with this, just wanted to point these things out to start a conversation. I don’t see one way as particularly better over the other, just understandable differences considering the origins and age of each project. I don’t even mention the Elixir-specific benefits over Python (long running processes, parallelism and so on) because I believe in this forum they are a given.

That said, I believe there is space for an “opinionated Phoenix”, be it a custom generator or a pseudo-app that automatically pre-selects and integrates external libraries to provide some common functionality, perhaps providing a unified API that looks the same but differs on dev vs prod vs test. OTOH, perhaps it’s just an unneeded layer of abstraction?

I am just curious if people who have a previous Django/DRF experience can give their take on this?

2 Likes

Not at all. I have a few custom types I use and I just pass the names in. :slight_smile:

Ecto changesets are validation definition and type conversions only, they do not nor should not perform updating or creating anything in the DB (I use them for a lot of non-Ecto things actually ^.^).

Isn’t that already mostly done? Plus it is easy to define your own custom ones in your project. :slight_smile:

Remember, in Dev mode the normal errors are instead replaced with a full stacktrace with light debugging page, not the errors you expect. You can disable that in dev.exs if you want. :slight_smile:

Why too many ways to do it though. Like I need ActiveDirectory auth, LDAP (not AD), a third-party database (in oracle, that does not follow a reasonable format), Google OAuth, among others (all in the same project!). Everyone’s use-case is different. Thus I use ueberauth, it is very nicely pluggable with a single interface to route through. If anything I’d like to see a refined-ueberauth added to phoenix, a Plug-like thing for authentication.

Besides, libraries are where things should remain, Phoenix should not be heavy for those that do not need excess functionality. :slight_smile:

(Still would be nice to have a phoenixy plug-style auth though, no pages, just the code, ueberauth would be a good base to set it up on.)

Then how else do you know what is added? o.O

I’m not sure how this could be automatically inferred at all…

Do remember, an Ecto Schema is just a local schema representation of the data you need, not the ‘entire’ table (a single schema can even represent multiple tables as I often use them for), so it is not something you could calculate it from for example. So I’m not sure how else you could do it without the migration-like files that ecto uses to define the actual database.

And even those external libraries fall hard since I use a lot of custom types. It is hard to define an ‘admin’ interface that works even with basic projects. Plus most ‘admin’ interfaces are just either raw Database or plain Schema displays with inserting and updating (and even the updating fails a lot), they are not true admin interfaces that actually manage context-specific data. For example, the admin interface I built at work is context-specific, it displays, hmm, well almost no direct database content at all, rather it works with the ‘Context’ interfaces (though mine exist pre-context’s), even dynamically in many areas (yay behaviours!). :slight_smile:

Hmm, that’s an idea, an admin interface that you build up in parts in how it works, a set of behaviours to define how to work and so forth hmm…

/me thinks those interfaces that just display database data are pretty worthless overall (especially when having to deal with VERY messy old oracle databases)

I don’t even use brunch, I just use NPM straight. ^.^

Remember, a SCHEMA is NOT a MODEL! A schema is a ‘view’ into some data structure (it may be a database, might be only part of a table, might be parts of multiple tables, might be something else entirely that is not in the database), so introspection will not do what you expect and hence why you should not. :wink:

If you want to introspect the actual database, then really introspect it with SQL, don’t go through layers that do not map directly to it (like most ‘admin’ interfaces do…).

If only Phoenix could live-reload only pages that are relevant for what was changed. ^.^;

Sounds like you want to make a library, you should get started on it. :slight_smile:

/me last touched django like 15 years ago when it was new and horrible, cannot really talk about how it is now, but it was really really horrible back then

2 Likes

I’m in a similar situation: I have a Django/DRF backend but quirks are stacking up so I’m starting to look for something else, Elixir and Phoenix being my current first choices. As I am still not halfway through “Programming Elixir” I will just list some of the problems I see in my Django/DRF stack and some first impressions I have on the Elixir/Phoenix world:

  • Django and DRF documentation is state of the art: no book is required unless you want to hit some more “advanced level quality” with Two Scoops Of Django. When you search for documentation recommendation on Phoenix you are usually told to buy some book like Programming Phoenix which is not even updated to Phoenix 1.3 yet.
  • I know they say loose coupling is at the base of Django’s philosophy but I just struggle to see how can this be the case: to me it looks a massively monolithic piece of software and the ORM is an hard dependency on almost everything in the Django world (and I’m talking about 3rd party libraries too, not only the main framework).
  • Fat models make my code ugly. I know I could separate business logic in some kind of service but it still feel like I’m working around a design flaw.
  • Sometimes the Django ORM gets in the way: there are times I found myself able to do things using raw SQL but doing many tries to get the ORM DSL replicate the results.
  • The automatic admin can have a bad influence on your design decisions: for example you can’t use inlines inside other inlines (an admin inline is some kind of widget to edit a relation model directly inside the admin view of the model having the relation) so you may think to alter the schema to work around the admin limitation. This is very bad. BTW Phoenix has this https://github.com/smpallen99/ex_admin but I don’t know if it is on par on features with the Django Admin.
  • You have to be really careful on using nested relations with the DRF because it doesn’t optimize queries. You have to manually inject select_related and prefetch_related in the right places to avoid duplicated hits on the database. I had some endpoints with parent/child/grandchild relations doing more than 700 queries before manual optimization.
  • Django’s ORM supports JSONB.
  • Django and Python ecosystems are just huge, I don’t know how to compare this to Elixir/Phoenix.
1 Like

I use django on a daily basis and I can provide some personal feedback.

Django’s migrations automagically work most of the time, saving a huge amount of time. Ecto has migrations but you have to write them on your own.

You don’t actually have to write the migrations yourself. It’s just like with django when you create the model THEN you ask django to create the migration from that model (or add/remove the changes). Phoenix lets you write your schema fields on the command line and it generates an ecto migration file for you. The big difference in both migration systems however is, with ecto, the order of migrations is extremely clear since all migrations are in the same folder and they are run in order. I had nightmares with django’s migrations when they were out of sync and some models depend on each other but django doesn’t know which one to create first, etc.

Django’s built-in admin is a very nice bonus while developing, while with Phoenix an external library must be selected.

I love django’s builtin admin. It was the reason I used django a few years back. However, after using ex_admin with phoenix, I realized how limiting django’s admin was. I’m able to do more functionality with less code. Stuff like controlling how records are saved and what form fields to show based on the logged in user, etc. I realize these things are possible with django’s admin, but they’re much easier with ex_admin.

If you are building apis, I think having django + DRF is a bit of an overkill. You’re using a framework and adding another framework to it to get the functionality you want.

4 Likes

“I haven’t really used Ecto or Phoenix for a CRUD application, but I’ve read a lot about them, tried out sample projects, and I’m monitoring the progress of the projects.”

Well on a framework specific board you are likely to get the sentiment that the framework is good for everything - but for a relatively simple CRUD application I’d take Django just about every time. My use case for Elixir is when you need high concurrency and the other major selling points of Erlang. Just the ecosysytem with python/Django alone is going to make you more productive if you are just doing simple crud for even a medium size much less small user base.

Looking over your observations the other problem you are bumping into is a VERY different philosophy in author approach. Phoenix believes in NOT including things where Django has its “batteries included” philosophy. Coming from a background of working with startups who need to show proof of concept THEN improve architecture I am not a big fan of Phoenix and feel the whole point of a framework is to increase programmer efficiency and not reinvent the wheel each time. As long as I can turn off features I’d rather have basic ones like authentication or at least have a recomended package as an add on (both of which has been pretty much refused the last time I read up on it).

like you I have found DRF to be a bit clunky but also found I can work around its clunkiness. Though I now use C# as my main work horse I found Django to be solid and as you suggested good to get things done. its one of the best thought out frameworks I have worked with ( and prefer their division of concerns into apps much more than contexts). Its kind of the opposite to the node/javascript world (always on the cutting edge and breaking changes) - solid, tried and true and thats where most development actually is.

P.S. things have improved steadily with training for elixir and phoenix. I had the same observation when the same book was recommended over and over again everywhere as the answer (ignoring that one author’s way of teaching will never fit everyone and the author of a framework is not always the best teacher of it) but you will generally find other sources for learning - make sure to join the appropriate mailing lists.

2 Likes

But, does it also update the schema file for me?

In Django, you don’t have any command-line utilities for changing the schema, because the source of truth for the schema are the schema files (models.py in Django), and the only way for requesting changes for the schema is:

  1. edit the models.py file with the change you want.
  2. ask Django to generate the migration script.
  3. ask Django to apply the migrations.

At the end of this process, you have a consistent schema, written by you, in your models.py files; Django takes care of guaranteeing that whatever you have in models.py is what you also have in the DB.

Whereas with Ecto the way of applying an schema change seems to be:

  1. Specify the change you want, as a migration, either by writing the migration script/file manually, or writing the change as command-line instruction for Ecto.
  2. Update the schema file manually, while making sure that your change matches the one you wrote manually for the migration (field names, options, datatypes, etc.)
  3. Ask Ecto to apply the migration.

In Django you only have to perform manually the step 1., but with Ecto you have to perform 1. and 2. manually.

From your answer, I gather Ecto is not capable of generating the migration files the way Django (and other ORMs) do.

Has any of this changed recently? if not, is there any technical/design reason for not having this convenience available in Ecto?

1 Like

I find both Phoenix and Ecto try very hard to only offer functionality that scales. Many of the conveniences they “lack” are hard-learned choices made by smart, experienced people who have built serious applications in other frameworks, only to find themselves frustrated by the limitations of the technology they started off using.

From my personal experience, Django’s model migration strategy in particular is a great example of this (and most of the decisions Django makes at large IMHO, though I won’t dwell on that).

When your models define what your schema stores, you are locked in to a lot of things:

  • You can’t add indexes you know your application’s usage patterns needs without modifying seemingly unrelated app code
  • You can’t add columns only some apps accessing your database need without modifying seemingly unrelated serialization logic
  • You can’t trivially control the order migrations are applied your to database outside of code changes

In short, you are letting one single application dictate your database’s design, which should generally come first. Django seems great following the happy-monolith-path, but both Phoenix and Ecto are really good at making sure they only encourage patterns that give you the room to grow into more complex applications and sophisticated architecture.

2 Likes

I agree with @christhekeele on this point strongly.
I always struggle to know the order of which migrations are run in Django.
Ecto provides all migration files in one directory so you can easily see all migrations. In addition to that, they are run in the same way they are visually ordered with timestamped filenames. This makes it extremely easy to rearrange order.

Migrations are meant to describe the database structure.
Schemas do not necessarily reflect the DB table exactly. I might have more/less fields in the schemas than in the DB table for business-logic purposes. I might also have more than one schema per db table. I could have a User schema and an Admin schema and both link to the same Accounts table with different logic for example.

I get that Django’s ORM provides nice conveniences, but when you need to go off the common road, it’s a bit of a challenge. With Ecto, it’s more involved, but you can accommodate more scenarios more easily.

4 Likes

Well, this has never been a problem for me, because their order is fully defined at the (Django) “app” level. But I do try to avoid migrations that affect multiple (Django) “apps”; luckily they don’t arise that often. I would be interested in a real example of how you ended up in that kind of situation with Django.

As I said, Django also does this for you, but not at the whole project level, but on the application level.

Also, regarding Ecto and you mentioning it puts all the migrations in a single directory (/priv/repo/migrations), now I wonder, how does it organize migrations for multiple repos? Does it put all of them in the same directory? That seems … confusing. (I’m just making assumptions here, as I have not used multiple repos yet, just inferring from that dir. structure.)

I admit I’m not familiar with this approach, and I have yet to use it and see how it compares to approaches in other ORMs/forms/serializers. We’ll see, it’s a different perspective and it does sound interesting :slight_smile:

If this is so, it’s quite inconvenient, since it’s spread in many files, and it’s expressed as diffs against previous versions. Compared to that, it’s much more convenient to just look at the schema directly as reported by the storage engine (postgresql, mysql).

In a given Repo, is there any source of truth for what is the full structure of the entities in the Repo, on the Elixir side? In Django, it’s the full set of models.py files, in Rails/ActiveRecord it’s the schema.rb file, which is regenerated automatically after each migration. With Ecto, what should I look at? The output of mix ecto.dump?

I’m only starting with Elixir, let’s hope I bump into one of those not-so-used roads, with a good reason to be there, so I can learn what this additional flexibility/complexity brings to me.

Thanks for your answers.

1 Like

Honestly, the most common situation is when two models in two different apps have foreign keys to each other. Django often complains about this situation if I don’t makemigrations carefully. Since Django does ordered migrations on at the application level, it’s easy to see only the migrations of that specific app. But if you have foreign keys between apps, this is where it becomes kind of messy. With all that, Django is trying to solve this problem amazingly. It’s just not as clear and obvious as Ecto’s approach (at least for me). And once you go “inter-app” relations, it gets way more complicated.

During the early stages of development, I tend to not really care too much about the structure of schemas/migrations. I can only add the most important fields first and move ahead, adding necessary fields to both the schema and migration files as I see fit. Often, I face the need to re-arrange migrations based on table depends on which.

Correct. mix ecto.dump creates a file with the current structure of the database. It’s the equivalent of rails’ schema.rb.

I like Django. And I like that you only deal with one file, models.py. But for me, it does have its problems and I find Ecto much clearer.