shanesveller

shanesveller

This relates to a discussion thread we’ve had at work back in July and I’ve gisted my answer for that here. It contains more of my positive sentiments about Elixir/BEAM, and there’s a decent amount of those, but it’s semi-off-topic.


For the thread at hand: I’ve stopped working with Elixir outside of my day job, and mostly stopped promoting it to others or participating in user communities in my spare time. I no longer feel the drive I once had to help others be successful with the language.

I feel that there are too many papercuts compared to what I personally value in software development, and most of them are inflexible aspects of the experience of working with Elixir. I began learning Elixir in mid-2013 and have worked with it professionally in some capacity or another since 2016, and full-time as my primary focus since late 2018. I still work with it daily and have no plans to change that for the foreseeable future.

Most of what I am feeling jaded about falls under one of these larger topics, and I’m possibly going to ruffle some feathers and/or summon some pedants to tell me how I’m mistaken:

  • I want an efficient, sustainable, and graceful on-ramp to deploying software. Otherwise, what’s the point? I know this part of the domain extremely well, care about it very/too much, and have invested many hours in making it better personally and professionally, but it basically only gets us to the same table-stakes that any other language enjoys. Runtime vs compile-time configuration continues to be a massive attention/energy sink and teaching hurdle. Teaching good instincts for Elixir deployment has not gotten any easier than it was in 2018, IMO.

    More personally, I don’t live in a BEAM-only world and I can’t tolerate niche features and tools that can’t be standardized with the other languages I write/deploy. I don’t value hot upgrades and I assert that most businesses don’t need them, only pursue them from vanity.

  • I increasingly value provably correct software, which for me means an actual type system to go with your good testing practices. Dialyzer is an insufficient tool here because it is so easy to ignore/misinterpret/bypass. It has non-zero value but the ROI on its usage is very poor even as a willing participant. I can count on my hands the number of meaningful bugs it has caught at my professional roles, although they were often extremely nefarious and would never have been caught by tests.

  • It’s hard for me to pin this part down past half-baked sentiments, but I feel that Elixir is a language that is reasonably good for experts but far too flexible, vague and undirected for neophytes. There is not enough guardrails in the core tooling, and few idioms being prescribed from first-party sources. As many people try to claim about C, it’s easily possible to write very clean, well-designed and effective code in Elixir. However, the language gives you plenty of rope to hang yourself with if you are not working with or at that expert-level experience. I’m not arguing for something like Golang, where someone with two weeks of experience and someone with 15 years of development experience write similar code because they have no other choice. I don’t know what I am arguing for here, but I feel a lack. rustc and clippy do absolute wonders here as pedagogical tools.

  • We still don’t have much adoption by mainstream companies and providers - it is a night and day difference when comparing to a language that is used more broadly. Our library ecosystem often feels like this XKCD comic, and our language tooling (i.e. elixir-ls) are maintained by underappreciated and diligent volunteers who are spread way too thin. I don’t think more than a handful of people are paid to work on this space, and I still encounter areas a few times a year where I would have no choice but to write an integration library myself.

    ExAWS is still looking for maintainers last I saw, despite it probably being extremely crucial to many businesses, and AFAICT Elixir is basically the ghost of a dream within FAANG, Dropbox, Spotify, etc… I don’t chase these companies out of me-too attitudes or vanity, but I feel quite powerfully that they generally disregard Elixir as a realistic choice for serious projects and so we are excluded from the kinds of contributions that their additional engineering power could make, to the benefit of us all.

  • There are some use-cases that we are quasi-permanently not suitable for. Some of those happen to be areas of work that I really want to participate in. We are very good for web development and for network daemons, and Nerves is very compelling for embedded, but Elixir is a poor fit for desktop software, CLI applications, game development, machine-learning or other compute-heavy tasks, or FFI coming from other languages.

  • There is some noticeable brain-drain occurring, especially towards Rust and WASM. Losing the active participation of some of the folks who have moved on really set us back, IMO.

  • Fairly minor in the list above, but I think we made deals with devils in courting the amount of Blockchain/Crypto orgs that are present in the community. I personally find that approach unconscionably wasteful ecologically and also almost universally solving the wrong problem technically.

My gist above has more (not particularly refined) thoughts here as well. If I sound like a hater, I had a lot of good to say about Elixir too. I am undoubtedly a pessimist and a cynic, though.

I don’t think libraries of end-user macros or other non-breaking changes are the way out for most of my problems. I also don’t think some ill-defined “craftsmanship” movement about professionalism is what’s missing either. My problems aren’t because the community isn’t comprised of talented folks.

I wish projects like Gleam, Hamler, and whatever Facebook is cooking up the absolute very best, but I don’t think I am likely to stick around long enough to see their dreams fully realized. They would probably resolve a lot of my frustrations if they can deliver on the premise.

My current front-runner as a replacement is Rust. That’s probably fairly obvious from some of my phrasing above.

Showing Posts 94 to 85

lessless

lessless

The people who tend to write code negligently and without much care what gets passed to where don’t survive for long coding Rust, OCaml, Haskell, and the other compile-time typed languages.

I’m talking about empowering customers with new capabilities. That’s on the other level than codebase aesthetics and the absence of errors related to type conversion.
If someone doesn’t have the skills to adhere to the quality bar you do pair-programming, let them have time to learn and practice deliberately until a problem is solved.

They will either raise to the brighter part of the hill or will continue their journey where they don’t have to be under the light constantly.

But it’s a bit too extreme a view for my taste because my practice has shown that forcing people to “behave” with good tooling (or a strict programming language) works.

I stick to the theory that people want to become better at what they’re doing. And either developing software is not important for them or the company doesn’t give them a chance to unveil their potential.

But it’s a bit too extreme a view for my taste because my practice has shown that forcing people to “behave” with good tooling (or a strict programming language) works.

Practice can show only that something is not inherently wrong, not that something is completely right.

Rust’s async used to suck but the community is extremely dedicated and things are improving with an impressive speed.

That’s great to know!

This can’t be generalized.

I’m willing to bet my last penny on that. Give complicated enough domain space, decently sized company, and two-three years timeframe. Team will always win over a group of individual contributors.

That one I fully agree with. Elixir is extremely productive. You can test your idea with just a few lines in iex and prototype a solution in minutes.

I meant long-lived processes that outlive individual I/O requests/response cycles. They add a dimension of time to the app.

Rust’s iteration cycles are very punishing in terms of an up-front cost and this can be quite annoying.

Not only Rust. Again, Rust is a great language. There is no intention to start a flame or language-politics discussion here.

Software is an implementation of a conceptual model and every day we learn more and more about it.
Compare yourself after one year in a project and after two years.
You are more knowledgeable and understand how abstractions should be structured much better than you did before.
But how many teams update their codebases to reflect their updates in their conceptual models?
Many of them prefer to go through hoops of mental-mapping while writing code.

And that’s what makes people neglectable to their professional duties and become disappointed in their software development careers.

Even worse - junior developers are being poisoned into thinking that it’s the only possible way to go.

And then you have to add tools to “force” people into doing things, instead of having them as just a safeguard against a silly mistake.

dimitarvp

dimitarvp

This actually works for a lot of issues. The people who tend to write code negligently and without much care what gets passed to where don’t survive for long coding Rust, OCaml, Haskell, and the other compile-time typed languages.

Even outside that area, enforcing a simple pre-commit GIT hook – which is a tooling solution, not a programming language one – can do wonders for the productivity of the team.

I relate to the argument that not all social / economic / politic issues can be solved by tech. That’s true. But it’s a bit too extreme a view for my taste because my practice has shown that forcing people to “behave” with good tooling (or a strict programming language) works.

Strange example. I’d definitely leave the store and not return there anytime soon. There are acceptable and unacceptable ways of gathering metrics / analytics.

I’ve been in one such company last year. Rust’s async has been improving literally every week, for months. We ended up having one Kubernetes pod with 4 vCPUs and 8GB RAM with a NVMe SSD storage that peaked at about 150,000 parallel requests.

Rust’s async used to suck but the community is extremely dedicated and things are improving with an impressive speed.

I agree it’s not OTP, of course; nobody has beat that so far. But Rust is getting fairly close, trust me.

I’ve seen this but I’ve seen the other extreme as well. This can’t be generalized.

Agreed. Good devs find ways around the problems in their ecosystem and even make them work to their advantage. Example: we might not have compile-time typing guarantees in Elixir but between mocks and property tests you can gear your system with a very heavy anti-bug armor.

Bad devs will screw up no matter what they work with.

That one I fully agree with. Elixir is extremely productive. You can test your idea with just a few lines in iex and prototype a solution in minutes.

Rust’s iteration cycles are very punishing in terms of an up-front cost and this can be quite annoying.

lessless

lessless

A powerful type system is a good thing to have but it’s even of lesser value than tests when it’s coming to proving the logical correctness of the software.
At the end of the day, only good software development practices can save it.

And it’s about working with your teammates on software as a team.

I see this trend of techies trying to solve socio-technical problems with more tech and I think about how pervasive that practice is.
To quote “When Coffee & Kale Compete”.

I feel that way especially with software because we have the analytics and the geeks who are building the software; they’re all about tracking and logging and all these data…I always give the analogy of being a retail shop owner and hiding in the back room and trying to learn from your customers by watching the closed-circuit television.
You could watch [customers] come in, walk around your store, pick up things, put them down, try things on…or you could just walk out and ask them, “Hey, what brought you in here today? What are you looking for? What other places did you try in the past?”

And I believe this is at least a part of the reason why people are migrating from Elixir to Go and then to Rust

There is some noticeable brain-drain occurring, especially towards Rust and WASM.

The hell one would prefer a language with rudimentary concurrency primitives for networking heavy-services? That’s really above my head.

I remember a conversation when crypto guys discussed that their node has a limit of 40 connections because async I/O in Rust sucks. They worked in a company that I hugely respect and which was valued above $500m.

Sure, the trade-off there is that Rust became the de-facto standard language for the new wave of crypto projects and it has all the cryptographic libraries which will be ridiculous to reimplement in Elixir (At least it was probably unfeasible before Nx).

But the point is that a team of developers working with a dynamic type system will deliver a better crafted piece of software that will serve their customers better than a group of individual developers working with a static type system.

TLDR: groups of individual contributors who don’t know how to write software together, as a team seeking shelter in their tools.

That doesn’t have to do anything with language.

disclosure: I do love Rust’s type system and it’s the next language that I want to learn because it will definitely expand my worldview.
But not as nearly as Elixir who added notion of time to my everyday practice.

sasajuric

sasajuric

Author of Elixir In Action

Yeah, I also think it’s similar to how I work in general life, compartmentalizing things when they become too large.

You can use this particular mix task without needing to setup boundaries or add the boundary compiler. I.e. just add the lib as a dependency (I think even runtime: false is enough) and invoke the task on some module.

dimitarvp

dimitarvp

I arrived at the same way of doing work about 8 or so years ago. I wanted my code to read like plain English and I have bent rules to achieve this goal. During this process I also discovered the “junk drawer” module pattern. It’s the way our brain does things anyway – and is reflected in how many of us “organize” our homes as well. :037:

This has also led me to use module/function names like:

  • MyApp.Bussiness.Cart.add
  • MyAppWeb.Views.Util.line_item_container
  • MyApp.Schema.User
  • MyApp.Business.Users.reset_password

etc. And, as said above, utilize defdelegate so I can have one convenient place in my brain to look for functions related to X or Y in the project.

Kudos and really great work on that piece of software. :023: I keep looking for excuses to use it and that might be the one.

sasajuric

sasajuric

Author of Elixir In Action

This is a great (and a very important) point! In my view, a large number of micromodules is basically just another extreme, not particularly better than a small number of megamodules. In both cases I find it hard to see the forest for the trees. Personally I prefer a balanced code, that has a “reasonable” amount of “reasonably sized” modules. This is all very hand-wavy, but I don’t think we can use a precise number (i.e. preferred LOC per module) as a rule. As a very rough guideline, I think that modules > 1000 LOC (not including user-facing documentation) can often be partitioned in a meaningful way. OTOH, a small module with a single public function is likely an overkill. Occasionally straying away from those guidelines is fine, but if the codebase ends up with a lot of mega- or micro-modules, then it’s probably under- or over-designed.

Even more important than the size is to look for cohesion. Things in side the same module should logically strongly belong together, i.e. deal with the same logical concept. To get that I start by stashing functions in a single “junk drawer” module (e.g. top-level context), and then refactor once the module starts feeling bloated. This is to me the essence of the agile design. Don’t start with an overelaborate design upfront, but instead make educated decisions after the fact, reasoning about the existing code that supports today’s features, and identifying groups of behaviour.

The boundary tool recently got a helper mix task called boundary.visualize.funs that can assist with such refactoring. The task produces a dependency graph of functions inside a module, which can help the developer make a decision about a module split. Roughly speaking, a frequent sink, i.e. one or a few functions called by many others, might indicate a potential for a separate abstraction. Likewise, independent verticals (groups of functions that do not call each other) could sometimes be worth splitting (though not always, e.g. I prefer to keep the readers and the mutators of the same “thing” together, and they typically don’t depend on each other).

dimitarvp

dimitarvp

Yep. “Service”, “manager” and “context” in particular are words that have been so abused over the years that these days they mean absolutely nothing to me. Mention them in a dev meeting and you’ll get blank stares from yours truly. :003:

sodapopcan

sodapopcan

As someone who gets headaches seeing huge classes, I don’t feel that same pain with modules full of functions. The really great thing about functional is that if a module gets to unwieldy, you just break it out into another module and delegate to it. While you still have to come up with a name for it, you don’t have to worry about what pattern you’re using to extract it (“service” or “manager” or any of that utter nonsense) and then, just as @dimitarvp mentions, defdelegate to it. The cost of making a naming mistake in this instance (no pun intended) is also very, very low. In fact, my root contexts generally just have a few functions with a bunch of delegates and it’s dead simple to see exactly what the entire context does with a quick scan, function signatures and all.

dimitarvp

dimitarvp

(is a bit bored during Sunday afternoon) … Oh, I know! I’ll necro a thread! :003:

You are comparing two polar opposites (200 files x 20 lines each vs a single file with 4000 lines). In all Phoenix projects I worked on we had a Users context that never went above 400-500 lines of code which is still very manageable with a good IDE (or even for me with Emacs and without any tree-view-on-the-left addons).

Obviously I am not telling you how you should feel about your style – large amount of small files is for me a complete “nope” however. I find it reassuring that something like MyApp.Domain.Users contains every operation I might need to operate on the users in the system. And if the file grows big, I’ll split it apart and put a bunch of defdelegates in the central context to keep the perception that there is a single doorway to the user functionality in place.

That, plus exercising some discipline on what functions are public or private nets me exactly the same benefits you see from your approach, only I have to open much less files during my workday.

Exadra37

Exadra37

No, I don’t have any. Just an Organization in Github and Gitlab that I started with some repos, but that I never finished.

I place the code that is common to more then 1 action in the root of the folder for that resource, like here:

So the todo_context.ex and todo_socket.ex are where common code goes, and one of the few places where I don’t go by the rule of 1 public method per module.

Always remember that code duplication is always better then the wrong abstraction. In doubt, don’t abstract until you understand better the domain you are working on, because later you will be able to do better abstractions to avoid code duplication.

Not on my editor:

Screenshot from 2021-01-13 18-30-42

But, the naming convention used for naming the files is whatever it makes sense for you.

For me the important bit to drive clean code and clean architecture is to keep the folder structure resource -> action -> module.ex

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
AstonJ
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
Null-logic-0
What IDE or editor are you using for Elixir development? Personally, I use Zed, and I really like it, but sometimes I wish there were a ...
New
maennchen
:warning: Security advisory: Decimal DoS vulnerability A vulnerability has been published for decimal where very large exponents can cau...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews