On 'Explicit is better than Implicit'

[Qqwy]

Hello there, everyone,

I just listened to the ‘break it down like a fraction’ episode of Elixir Outlaws on my way back home. It is mostly about the subject ‘explicit vs implicit’. I found it very interesting, since I thought it had a very clear and simple meaning, to find out that in actuality, people have given very different interpretations to what ‘explicit is better than implicit’ actually means.

So that’s why I think it might be worth talking more about this, because I agree with show-hosts Chris Keathley (@keathley) Amos King (@adkron) and Anna Neyzberg that applying a guideline without thinking about the context it was conceived in is a dangerous thing.

---

Personally, I’ve always thought that the main idea of ‘explicit is better than implicit’, which I believe has its origins in the Zen of Python is not about how (if at all) complexity should be abstracted away (which is an interpretation that a lot of the podcast episode focuses on), but rather the (mostly) orthogonal concept of (a) clear naming and (b) making it as clear as possible how the data/state flows through the program.

So the idea that “a user should understand with as little cognitive effort as possible what a call to a function does”. Importantly, not how it does it! (this is what is hopefully abstracted away).
This has been used as an argument in many aspects of the design of the language itself, as well as the standard library:

• The use of defining new infix operators is often discouraged, and José has made the choice to only allow a handful of operators to be (re)defined, rather than allowing arbitrary new ones to be used.
• Pipelines prefix module names (for non-local functions), which make it more clear what the subject is (expected to be) w.r.t. a chain of member methods that work on ‘whatever the last member method returned’ in Ruby.
• The separation of functions that create/transform datastructures from functions that perform an effect. A prime example here would be creating an Ecto query or changeset vs. running it on a particular Repo.

So I am very eager to hear what other people think about this:

1. Have you seen ‘explicit is better than implicit’ been used in the wild as justification for something where you think it is not applicable?
2. What is your interpretation of ‘explicit is better than implicit’? Do you try to apply it to your code in one way or another, or not?

[JEG2]

I shared a couple of thoughts on Twitter:

https://twitter.com/JEG2/status/1121561197780844544

and there were some replies:

https://twitter.com/ChrisKeathley/status/1121737994657050624

[Phillipp]

Personally, I love to write rather explicit code (and more code) than implicit code with hidden side effects.

Over the last 1,5 years, I had to work on a few client projects in javascript/nodejs and one thing was the same on all of them:

I wrote rather explicit code where the “data path” is easy to see and reason about. Then a couple weeks in, the javascript/nodejs developers at the client started to rewrite everything just to end up with the minimum amount of code but with maximum complexity and implicity. Especially things like “factories” and “currying” were their favorite solutions to everything.

The functionality to the outside was the same, the performance of the application wasn’t improved but IMO the code smelt like a dead fish.

For me, explicit vs. implicit is mostly about code design and readability. When I look at code, I want to understand what it does without following a handful of helper functions and modelling of the data structure in my head first.

[josevalim]

My take on the “explicit better than implicit” is that we want to make the complexity explicit. This makes its interpretation a bit more personal, because what is complex for some is not necessarily complex for others, but I believe it aligns well with FP. Mutability is troublesome? So let’s make it explicitly opt-in. Communication is hard? Let’s be explicit with messages! Oh, you have side-effects? Let’s keep them in their own corner.

In particular, explicit does not mean dropping things like “convention over configuration”. In fact, I would argue that making everything verbose makes it less explicit, because you don’t know what to focus on (i.e. if everything is explicit, then nothing is explicit).

PS: I didn’t listen to the episode yet.

[OvermindDL1]

@josevalim

In particular, explicit does not mean dropping things like “convention over configuration”. In fact, I would argue that making everything verbose makes it less explicit, because you don’t know what to focus on (i.e. if everything is explicit, then nothing is explicit).

This is so true!

Go is a prime example of this, the sheer huge amount of code just to do simple algorithms obscures the work actually being done.

It’s a fine balance. I’m finding this thread quite interesting to read so far to see everyone’s ideas on it.

[dimitarvp]

To me, “explicitness” must be married to “readable for humans”. I’m fully okay with some implicit variable somewhere – say, an Agent storing a base URL to a 3rd party API, or Ecto’s Repo config (or almost any config really). If that implicit variable changes rarely then the reduced human cognitive cost of calling functions with less arity is indispensable.

[adkron]

Implicit vs explicit is about the functionality of an API and communication. The discussion that we were having on the show was born out of some developers avoiding putting parts of the code into functions and the argument is that things aren’t explicit once they were in another function.

I find this to be a fallacious argument. Placing code into a function doesn’t make it implicit. In fact, it often creates a more explicit API when paired with an intention revealing name. Naming something in a way that communicates the intention and what it does is one secret to a self-documenting API. In the end, we want code that communicates its intention in an explicit way. Often the intention is lost when the reader has to wade through conditionals and Enum calls to figure out the intention of the code.

[peerreynders]

@adkron

The discussion that we were having on the show was born out of some developers avoiding putting parts of the code into functions and the argument is that things aren’t explicit once they were in another function.

This sounds like my regular rant about the habit of many JavaScript developers of using inline anonymous functions instead of factoring the functionality into a well-named function.

However that is not the impression I got from the podcast.

To me the argument seemed to be that the Python-style preference of Explicit Is Better Than Implicit is wrongheaded:

#Explicit
def read_csv(filename):
    # code for reading a csv

def read_json(filename):
    # code for reading a json

#Implicit
def read(filename):
    # code for reading a csv or json
    # depending on the file extension

i.e. read(filename) is implicit but also more loosely coupled; read_x(filename) is more explicit but a) more tightly coupled and b) forces the complexity of determining the file type onto the user code.

[PragTob]

(didn’t listen to podcast as podcast really isn’t my kinda medium)

Thanks for bringing the discussion here, never thought there would be so many different opinions/views on this

My major interpretation of explicit vs. implicit is to a degree how much I can easily see what’s going on and control it.

My favorite example is ActiveRecord vs. Ecto.

In ActiveRecord I define all the validations and callbacks in the model. Somewhere else I then call save and all of these are magically executed (with all the weird conditions applied in the model). I don’t see what is executed and in what order.
In Ecto on the other hand I specifically choose a changeset that I want this to be run through, I can easily see what is done and in what order. If I don’t want something to happen I can easily remove it or explicitly call another changeset. Makes my intentions very clear.

Similar thing about automatic preloading of associations vs. not or the way in which you need a PhD in ActiveRecord to know what triggers a database case vs. modeling it more explicitly through use of a Repo in Ecto.

As it still rings in my head from an early Ruby Rogues episode where @JEG2 said “magic is always bad” - the more experience I have, the truer that statement rings - kudos James

[keathley]

@PragTob

(didn’t listen to podcast as podcast really isn’t my kinda medium)

Oh that’s fine lots of people don’t listen to podcasts. No big deal.

@PragTob

it still rings in my head from an early Ruby Rogues episode

Hey wait a minute!

[Rich_Morin]

@peerreynders

This sounds like my regular rant about the habit of many JavaScript developers of using inline anonymous functions instead of factoring the functionality into a well-named function.

This! Anonymous functions complect the function code with the call, turning two (or more) simpler expressions into a single, complicated one. I don’t like this in general and I particularly dislike it when I’m creating a pipeline.

So, I almost always stuff my anonymous functions into local variables and then use them by name. However, I’m still struggling with the definition of a “well-named function”. My current (idiosyncratic) idiom looks like:

|> Enum.map(map_fn)

This works fine for many simple situations, but it can get a bit ugly when there are naming collisions:

|> Enum.map(map_fn1)
|> Enum.map(map_fn2)

[PragTob]

@keathley

Hey wait a minute!

Let me explain,

I love the concept of podcasts and believe they generally can be a great resource. However, I’m a very visual learner and person - so if I “just” listen to something, no matter how interesting, I feel like I should do something with my eyes and am otherwise “bored”.

I tried coding while listening to podcasts, that works as long as I do easy features or refactorings I’ve done a hundred times already. One bug or architecture decision though and see you back here in ~30 minutes with no idea what anyone said on the podcast.

These days the only time I really listen to podcasts is when I go by bus (I get sick when I read in a bus). I kind of want to listen to podcasts more (especially the outlaws as I heard many great things about it) so I’m looking for opportunities like going jogging again, longer chores at home, inhaling or ???

[adkron]

I’ve been trying to figure out how to inhale podcasts since I quit driving 100 miles each way to work. Even then I would listen to the same podcast multiple times because my brain would wander in the middle of it. I sometimes miss that drive because of the thinking time, but seven years of it and I had to quit.

[keathley]

Since a few people haven’t listened to the podcast I thought I would try to explain some of the main points. I’m not sure I can enumerate everything here with as much nuance as we had on the show so I’d still recommend listening. But hopefully I can capture the main points.

I made the case on the show that a lot of the apis that people like in elixir tend to be highly implicit apis. I also made the case that explicit apis tend to be worse apis then implicit apis. That was my contrarian opinion. So now I’ll try to justify it.

When I’m talking about implicit apis what I’m really talking about is encapsulation. A key part of good api design is about hiding large amounts of complexity behind a small interface. If this is done well then the end user doesn’t ever notice the underlying complexity which is really as good as removing complexity altogether. The interface should be small for similar reasons. If you need to chain together multiple api calls in order to use a module then this is a more complicated api then an api that accomplishes the same goal through a single api call. A good example of this would be if every time you wanted to execute a query through ecto you also needed to check out the database connection yourself, parse and compile the query yourself, send the query to Postgres yourself, etc. I refer to APIs like this as “maximally explicit APIs”. They tend to be cumbersome to use and more error prone. But ftmp ecto hides all of this complexity and it’s a better api because of that.

But all of those details are “implicit” to us, the end users of Ecto. We don’t see how any of that stuff is managed from an api perspective. Sure there are escape hatches into that level of the system. But generally you don’t have to wander down there. At our layer of the stack the call to Repo is explicit, the query is explicit, and everything else is encapsulated.

Obviously those details are handled somewhere. If you work on a database pooling solution then you might have to care about them. But at our layer we’re able to ignore them and instead be more explicit about our domain logic.

The trend I see is that people have taken advice like “explicit is greater than implicit” and are carrying it a little too far. If you build a “maximally explicit” api then you quickly start to limit your ability to encapsulate complexity. If you take explicitness to it’s limit then encapsulation becomes impossible.

I think when people say they want explicit APIs over implicit APIs what they really are saying is “don’t surprise me”. If you say to ecto “run this query” you expect it to run the query. You don’t expect it to run the query and also buy you a toaster. The rails example you mention @PragTob is a great example of a surprising api. It’s almost like it was designed to create surprise through indirection.

I think what we ought to be heading for are APIs that encapsulate large amounts of complexity - which means APIs that hide implicit details - and have few if any surprises. It’s a tricky balance. But I think that’s the correct strategy.

[otijhuis]

What I find interesting is that most of this discussion about being explicit is about explicit actions/implementation details. I find being explicit very important but I look at it from different perspective.

When I search for the meaning of explicit I get the following:

stated clearly and in detail, leaving no room for confusion or doubt.

To me it’s all about being explicit about the meaning of the code. The part leaving no room for confusion or doubt is very important. This ties in to “don’t surprise me”. What the code says it will do, and what it actually does when you call it, should match. And it only should use the appropriate actions to get to that outcome. No surprises. How much detail you need depends on the context.

I would say this:

stated clearly and with enough detail as to leave no room for confusion or doubt about the outcome in the current context.

Here’s a simple javascript example

if (person.sex === 1 and person.children.length > 0) { ...do something... }

From a technical standpoint this is very explicit. I know what it’s checking. But I have no clue what the meaning is from a business logic point of view.

if (person.isFemale() and person.hasChildren()) { ...do something... }

This looks much better but it might only have made it more readable. What if this code is specifically about female parents. Is this clear from the if statement? You can assume it’s correct but maybe someone forgot a check. There’s still room for doubt.

if (person.isFemaleParent()) { ...do something... }

To me this code is very explicit, yet it hides the details. It tells you exactly what the code cares about from a business perspective. There could be a bug in isFemaleParent but you know precisely what it’s supposed to check and there’s no confusion or doubt about the meaning.

In case of an API, be explicit about what it returns or what the outcome is. This can be an API contract. The consumer still wants you to be explicit, just about the outcome/result, not the implementation details. For example, with a GraphQL API you’re being explicit about what the consumer can ask for.

So I agree with your opinion about hiding complexity. I just believe being explicit should focus on the meaning/outcome of the code (in context), not the actual implementation details.

[nathanl]

A good example of this would be if every time you wanted to execute a query through ecto you also needed to check out the database connection yourself, parse and compile the query yourself, send the query to Postgres yourself, etc. I refer to APIs like this as “maximally explicit APIs”. They tend to be cumbersome to use and more error prone. But ftmp ecto hides all of this complexity and it’s a better api because of that.

Agreed. I think that you should make something implicit when you can promise that callers won’t have to care about it. Ecto can keep this promise in all cases I’ve personally encountered, so I think this implicitness is good.

As another example, consider a Rails controller action from the guides:

class ClientsController < ApplicationController
  def new
  end
end

Note that the empty method from the example above would work just fine because Rails will by default render the new.html.erb view unless the action says otherwise.

I don’t like this because 1) maybe 5% of the time I’ll need to make it render something different and 2) 100% of the time I need to know what file it’s going to render. To me it makes most sense if I always see an explicit render; that makes it obvious which file I should work on for the template and where I can specify to use a different one if necessary. But someone could argue that omitting the “standard” render lets you focus on the unique aspects of this controller, and you can be explicit if you prefer to. I’d say this implicitness is iffy.

In ActiveRecord I define all the validations and callbacks in the model. Somewhere else I then call save and all of these are magically executed (with all the weird conditions applied in the model). I don’t see what is executed and in what order.

In this case, buggy interactions between callbacks are easy to create and not fun to debug. Especially when you have callbacks that touch associations, associations that auto-load, etc. The promise is that you won’t have to care about how the callbacks run, but that promise gets broken. I’d say this implicitness is bad.

[dimitarvp]

@otijhuis

stated clearly and with enough detail as to leave no room for confusion or doubt about the outcome in the current context.

In functional languages “explicit” also means “…and don’t have my functions depend on external state if it can be at all helped, please”. Obviously when you work with files, console I/O, network etc. then it’s unavoidable. But “explicit” in the context of this thread carries the extra context of “functions should be idempotent and self-sufficient”.

Which also has its limits. As I mentioned in a previous comment, you wouldn’t want to specify your DB credentials on every call; in such cases having implicit state (the DB credentials, adapter, pooling options etc.) which is persisted and loaded from a non-obvious agent is still convenient. And as long as the programmers in the team don’t forget about these special cases – and keep them to a minimum – then a good balance between minimum WTFs/minute and convenience is achieved.

[otijhuis]

don’t have my functions depend on external state if it can be at all helped, please”.

I totally agree and I’d include that in the leave no room for confusion or doubt part. I wouldn’t say that explicit has anything to do with being idempotent though. But if you mean that a function shouldn’t behave differently when external state changes, I agree. And if I can keep a function pure, I definitely will.

You’re right about things like DB credentials and such. That’s why I like the repo pattern in Ecto for instance. Your configuration might live elsewhere but you’re still being explicit enough about which repo is affected and you can name the repo appropriately. There are always trade-offs, that’s why I added in the current context to my definition. For the people working in that context it has to be clear and easy to reason about.