Can we discuss app architecture and optimisation etc?

Hey all,

I’m wondering if we can discuss questions like:

  • when to make an app ‘just an app’ or an umbrella app?

  • when to make something a plug?

  • when to use/not use atoms (as a noob I see their benefits when writing code but read people warning to avoid some uses due to their persisting in an ‘atom table’.

  • etc - other parts of Phoenix |> Elixir | OTP ecosystem.

in terms of:

being able to separate concerns for:

  1. scaling - horizontally and vertically

  2. stopping/starting processes - eg backup, maintenance

  3. speed eg what does Phoenix take care of and what must we take cafe of eg when to use/not use atoms

  4. maintainability - easy of reading/editing code - perhaps when to use ‘hot swappable code’?

And perhaps in the context of different use cases so we can refer back to them eg how to architect sessions/auth as an app or plug or mix of so signups/signing in could be turned off temporarily for unscheduled maintenance while allowing existing signed in users to continue to use the app, and whatever common/outlier use cases this community has?

2 Likes

There is a hard limit of 1,048,576 and atoms are not garbage collected so you do not want things converted to atoms when dealing with user input or places where you are likely to hit the limit.

3 Likes

Discussion is great but you should know there are great resources that touch on some of these topics:

  • Elixir in Action
  • Designing for Scalability with Erlang/OTP
  • The official docs
  • Elixir conference talks

Yes, true - I have been reading Elixir in Action, The Little Elixir and OTP Guidebook, and other books, as well as reading the official docs, watching Elixir conference talks on Youtube…

but as a noob I am asking that we have a discussion that can possilbly go towards a wiki entry about this subject because I chose to learn Elixir and Phoenix Framework for several reasons; one being speed of dev, and having to read across so many resources is time consuming and sometimes confusing.

If we can aggregate the community’s knowledge on this subject into the wiki it will help a lot of noobs like me stick with this framework and everyone of all skills/knowledge levels develop better apps.

A good rule of thumb for me, although might not cover all use cases, is: use a single app if you’re building libraries, use an umbrella app if you’re building systems.

Umbrella app provides an easy way to logically separate components (or apps) of your project into this neat, self-contained projects. What’s interesting is that you can refer modules in another app without any explicit imports or anything by just specifying it as a dependency of the caller.

But having said that, if you’re just up and running, you’ll probably be better using the single app approach with scaffolding such as what Phoenix does. You can convert one to another in a somewhat straightforward way, although it might need some extra effort. Also, premature componentization might also make you shoot yourself in the foot.

At least for now, if I’m building an Elixir project with Phoenix, I’ll use an umbrella with at least two apps: the domain (often persistence layer) and the web interface (Phoenix).

Do note that most of these are not really my idea; I’ve adopted them from this forum, books, talks, blog posts, and other amazing resources. :slight_smile:

3 Likes
you do not want things converted to atoms when dealing with user input 

the to_existing_atom function can help sometimes

2 Likes

This :

Is orthogonal to this more or less.

but far more linked to this [quote=“RisingFromAshes, post:1, topic:2782”]
maintainability - easy of reading/editing code
[/quote]

In general, make something a plug if it take a conn and return a conn.
Use an umbrella app the first time you have multiple set of concerns.
I need to interact with users => Phoenix
I need to save data in a database => Ecto
I need to do some computation and logic on data between the two => Your own app
They are all related to one project, i need to keep them tied up together => Umbrella.

Thanks, I think I understand your point that coding in Elixir/Phoenix allows us to focus more on code maintainability as the language/framework/OTP handles the scaling etc, but this hasn’t really helped as the general info you’ve provided, I and most noobs already know, and when we read about processes, behaviours, etc, and start to get confident enough to take different approaches, and have delusional goals of building the ‘next big thing’ (which is why I spent weeks researching languages and frameworks before settling on this ecosystem, and before that a few years learning to code other languages, building stuff, and a first startup), and so on.

We want to know as much about how to code for scaling, speed, etc as is possible eg to only use atoms when … because of RAM, how we can scale eg can we have an umbrella app run over multiple FreeBSD servers (and should we?), when to use a plug instead of a function inside a module (because it can take compile time options(?) not just make code easier to read, pipe, etc), and how when should something be an app - because it can be started/shutdown/backed up easily etc - for real world dev ops when the app has grown to be used gloabally.

For example, looking around this forum there are so many scattered Q&As about umbrella apps - asking about practical stuff to use them, their limitations, etc for example, it would be great to have a wiki entry about this stuff - overall architecture - as I’ve tried to explain as a noob.

I really enjoy the clarity/elegance of Elixir and Phoenix, and after learning some Clojure, I’ve come to find that I always have leaned towards functional thinking, but the magic appears to be in the OTP/Beam VM and higher level stuff like GenServer/GenStage, almost as if the messy mutability of OOP has been shifted/‘copied’ to that level from the coding environment and we no longer have to worry about it, but I and others want to build apps that are as popular or more popular than Twitter etc, and to do that, we don’t don’t want to just rely on coding for maintainability but scalability etc too, which means we want to know when to use ‘components’ not just how.

That’s why I asked about those other factors as consideration for when to use a plug etc.

The thing is, there is no difference here between maitainability and scalability. And the way you separate your code is orthogonal to your scalability.

Scalability means that you have to be able to change the way part of your apps talk to each other. Because that is the thing that limit scale. And that means that you want to have a code base that is easy to change.

This is what I saw re: app and umbrella projects, not sure if this adds anything to the discussion but helped me a lot on how to sort some of my projects.

1 Like