Should we adopt Dave's way of building applications as a series of components? (Dave's talk has now been added!)

Chat / Discussions
, , ,
1

Another point that Dave drives home in his course is that applications really are components, and you should treat them as such. Not as “full fledged applications”. We’ve been doing that umbrella-style before, but umbrellas are confusing and I always felt that even mix phx.new --no-ecto --no-html --no-brunch api generated a ton of fluff. In the same vein that files/modules are free and you can use them to separate api/otp glue/implementation, applications are free and you can - and I think should - use them to separate concerns. The reason therefore that I’m putting this here is that I feel it’s basically the same principle at work.

We’re setting up a new service, that will be responsible for various internal permissions related stuff. The “hard” part so far was whittling down the Phoenix generated code to something clean and manageable:

..../api$ find test lib config
test
test/support
test/support/conn_case.ex
test/test_helper.exs
lib
lib/api.ex
lib/api
lib/api/endpoint.ex
lib/api/router.ex
lib/api/user_policy_controller.ex
lib/api/application.ex
config
config/config.exs
config/deploy.exs
config/dev.exs
config/test.exs

no channels, no contexts, no mixing up of UI and business logic in a single OTP application. The sole controller:

defmodule Api.UserPolicyController do
  @moduledoc """
  This module contains the interface between the API and
  the business logic in the user_policies application..

  """
  use Api, :controller

  def get_for_user(conn, user_id) do
    {:ok, policy_document} = UserPolicies.get_policy_for(user_id)
    json conn, policy_document
  end
end

which calls out to a sibling app (using the “poncho” organization style):

.../user_policies$ find test lib config
test
test/test_helper.exs
lib
lib/user_policy.ex
lib/user_policies.ex
config
config/config.exs

I like this a lot better than the Phoenix default “we are Rails and toss everything together” mess. These are skeleton both currently applications, but it is immediately clear where things go when we start fleshing them out, there’s very nice separation of concerns, a random bypasser can pick an interest and only open that code, and so on. Frankly, with the biggest cost being a top level Makefile to drive CI builds (I’m settling on a pretty standard skeleton for that), I think it’s worth pursuing. At least experimenting with, to see how it feels. My gut says it feels much better, and after ~40 years of coding, I have learned to rely on my gut. YGMV, of course.

(and I’ll shuddup now with my heresies :wink: )

Edit: See post 20 and below for Dave’s talk on this topic and the discussion following the video.

11 Likes
2

I love the idea of building an app as a series of components - it just feels incredibly right to me :023:

Not only does it make it easier to work on individual areas of an app, but significantly easier to swap out component as and when you see fit.

Each component would also not be ‘tied-down’ so much by the rest of the app, so where appropriate a component could even have its own database (a graph DB for a r/ship component perhaps) so each area of your app could be the ‘best’ version of itself as they would be highly independent.

Here are a few related discussions:

2 Likes
3

Someone really should convince the Erlang/OTP peeps to do a s/application/component/ on the codebase :wink:

1 Like
4

I think that is the core point - i.e. don’t get distracted by umbrella projects and context modules. Personally I’m not big on the term “component” but that is neither here nor there as I don’t have a better alternative - but I think that OTP applications are more “service-y” than “component-y”. Component evokes an image of “part of an assembly/aggregate” which focuses more on the static aspects of the system. Service focuses more on “collaboration/coordination” as the main part of implementing the behaviour of the system - which is its primary value.

People find static aspects easier to deal with even when clearly it’s the dynamic aspects that are the key to getting the job done.

even mix phx.new --no-ecto --no-html --no-brunch api generated a ton of fluff.

Likely an indication that Phoenix as a framework isn’t a good match for the problem you are solving.

I like this a lot better than the Phoenix default “we are Rails and toss everything together” mess.

From the Introduction of Programming Phoenix ≥ 1.4:

So it is very clear where Phoenix’s origins are. And while Phoenix is an improvement over Rails, it seems to want to capture/retain the appeal that Rails had as a development platform which in itself establishes certain constraints. So umbrella projects and context modules may be more suitable for developers stuck with a “web-application” mindset - i.e. they aren’t quite ready to adopt thinking in terms of a “service-based system”.

You may have come to a point where you need set up your own problem-focused micro-framework based on Plug or Raxx. The good thing about OSS is that ideas and their implementations are open for scrutiny and available for scavenging.

ElixirDaze 2018 - Build your very own web framework in Elixir by German Velasco

Someone really should convince the Erlang/OTP peeps to do a s/application/component/ on the codebase :wink:

Not in favor.

  • Process ≈ Server
  • Bunch of processes organized in a tree - coordinating via messages not bound by the tree structure

Doesn’t sound anything like a component.

Functional programming is about data transformation - behaviour as response to data.
Processes behave based on inbound data and encapsulated data (state).

Component emphasizes structure without necessarily emphasizing connections.

10 Likes
5

Someone really should convince the Erlang/OTP peeps to do a s/application/component/ on the codebase :wink:

from pragdave - Splitting APIs, Servers, and Implementations in Elixir

You’re right about applications - badly named - we should have called them components.

Joe Armstrong would agree with the sentiments, though I doubt actually replacing it in the codebase would actually be feasible. :wink:

1 Like
6

I guess this is something that very much depends on people’s background and past experiences. The word “component” to me can be stateful and dynamic. In my mind, a component can be started and stopped, be interacted with, manage its own state, interact with other components, etc.

Likewise, the word “application” brings to mind something that is “complete”, not a part of a larger system. This confused me for a while when I started with Elixir, though by now I have pretty much gotten used to it after working with Elixir for a while.

2 Likes
7

You’re probably right. Although I find that distinction less important than how wrong the name “application” is :wink:

Phoenix is Rails without a lot of the bad stuff. I get that. We started out just using Cowboy and Plug, but somehow every little service ended up being slightly different, because there is no style to follow. Which is why I recommended we do it “the phoenix way”, but then with the minimum amount of fluff which means taking a chisel and chipping away things until just the core is left.

In terms of problems, spending 15 minutes after mix phx.new to clean up is probably not even going to make it in my top-50. I do wonder though about the sort of patterns that we as a community propagate. GenServers mixing API, OTP and Implementation; Phoenix apps mixing two or three entirely orthogonal web-y things (templates, SPAs with JSON APIs, websockets) together with business logic and persistence; etcetera. I think we should have louder warning signs over quick examples “this is not how you are supposed to do it” :wink:

3 Likes
8

Why not just make up a new name, like the gem for ruby or jar for java? I know a gem or a jar is just a package and that’s not what an OTP application is, but my point is that we do not need to stick to a name that means something. When you first heard gem (or whatever you call it on your previous language) you did not associate everything that was inside there, someone had to explain you.

I’m not suggesting this name for erlang applications, that would be hard to convince them, but we could call it something like flask and say it is totally equivalent to an OTP application. :slightly_smiling_face:

Anyways, my hopes on this kind of change are very small, since a change like that can have its problems. I really am not bothered that much by the name application. :man_shrugging:

4 Likes
9

We just have to decide:

  • do we want a descriptive name (which is good because it helps to organize stuff defining what should be inside it)
  • or we want to not care about these naming stuff and allow people to do whatever they want inside their flask/howeverwenameit (which is good because MAYBE we would not have these discussions anymore)
10

I much prefer descriptive names - I think they make things feel more natural and intuitive :slight_smile:

Components is an excellent name imo :003:

Sometimes other names work well due to specific reasons, and usually when other benefits override going for a descriptive name (eg the link between Ruby and Ruby Gems still makes you feel Gems are something that belong to or related to Ruby, but also that they are something ‘great’.)

11

Well, me too, but sometimes that’s just too hard to bother with. But as I said, I don’t even mind about calling them applications, whatever. Changing it to components for me would be the same, maybe a little bit better descriptive. :man_shrugging:

Well, then maybe all we have to do is invite some very creative people to come up with ‘great’ names and be happy! :joy:

12

There are two features behind umbrella projects:

  1. Organizing multiple applications in the same repository (you called it poncho but it really is known as the mono-repo pattern)
  2. Allow those applications to run under the same config + deps

The first goal is way more important than the second. If people are using mono-repo, then mission accomplished.

The second feature is a convenience. If it fits your workflow, great, otherwise nuke the umbrella and use path dependencies. This is by design. To the point there are no features specific to umbrellas in Mix on how dependencies are handled. Everything is built on top of path dependencies exactly because we should be allowed to drop umbrellas and fallback to path dependencies.

We had this discussion in the past but in a nutshell, you are supposed to grow out of the built-in generators and upgrade at least to mix phx.new --umbrella. :slight_smile:

It is meant to be a journey. If we started everybody at the end of the road, then most people would end-up confused and wondering: how did I get here? Do we really need all of this?

17 Likes
13

Naming is hard and very subjective. Personally, I’m not sure I like Components as a replacement term for Applications. I think Component is an overloaded term in the front-end space (and programming in general), and I also feel it’s kind of a static term better suited within the context of an Application. Application also doesn’t feel perfectly right, but works better than Components to suggest it’s a more active actor rather than static resource, with the added benefit of familiarity.

I think Service is also good term for this, but maybe a bit narrow. Service in my head also feels more like a resource, or a stateless entity rather than in Elixir’s domain where things can be a lot more than just stateless.

An Elixir Application is sort of different than the typical Applications definition; our go to terms don’t fit very well due to Elixir/Erlang’s multi-faceted nature of being stateless, stateful, composable, and natively interactive in ways Application doesn’t hint at very well. I still think Application is the best term because I don’t really have any internal qualms with the idea and terminology of composable applications.

1 Like
14

:purple_heart::green_heart::blue_heart::yellow_heart: Thanks for that!!! This made Elixir really fun and easy language to learn! But come on! You should not give your secret sauce that easily! :joy:

2 Likes
15

Components sounds entirely wrong to replace Applications because Applications in OTP are singular and the word Components makes me think something that is instanceable many times. If Application were to be renamed then I’d think Daemon would be more accurate (OTP does follow the OS-style conventions after all).

5 Likes
16

To me, an ‘application’ is the whole thing - the finished product - and the application can be made up of many components.

To me a component is literally just that - part of a greater whole.

So for instance I could have a social networking application, which has a registration component, a profile component, a messaging component, a reporting component, etc. Each component could also be made up of smaller components.

I love the idea of building apps as a series of components in this sense :slight_smile:

I also love how Umbrella apps were ahead of their time :003: and how Dave is taking the idea to the next level :023: (I’m just waiting for him to do his next course to fill in all the remaining blanks :lol:).

3 Likes
17

I might make myself some time and submit a PR for mix phx.new --minimal, which would generate even less boilerplate :wink:

(didn’t know about the umbrella option, but I still will stay on the path of path dependencies for now. So far, they seem less confusing to me. And yes, it’s extremely simple to move between both models; one of the reasons I keep liking Elixir is that moving stuff around is so easy)

5 Likes
18

I just had another idea: what if instead of trying to get a name that fits better than application to all applications purposes, we classify the different purposes into separate things? Like using the term library for applications with no process started, and daemon for the applications with supervisors. @OvermindDL1 I really liked daemon BTW! :slight_smile:

Also, the common name for both in my opinion could be a project, the only generic name I see that could fit for both situations. :wink:

19

If following the OS style, Applications would be Daemons, random Supervisors would be, well, still Supervisors, Processes would be, well, also Processes… The only thing that does not match fairly directly is just Application to Daemon, but still, Application is fine enough, so I don’t see the point in renaming anything personally. ^.^;

3 Likes
20

Here is Dave’s talk from Empex where he talks about his “components”. You might want to sit down for this one :grinning:

15 Likes