[style] Parametrising generic functionality: behaviours vs protocols

All of that makes sense. Protocols are literally behaviours with custom dispatching rules, so your impression that they impose some rules in favor of some features is correct.

You are also correct that Flow could use protocols and the reason we didn’t use them is we are not ready to expose the Window materialization API.

That’s the only part I don’t agree with. If you are already using a protocol then you already have a behaviour. Protocols do not allow default implementations because it generating coupling between protocols and implementations. We already have the coupling at the specification level, which is the point of protocols, but moving it to the implementation level means that changing the default implementation of a protocol will affect all implementations that depend on it.

The __using__+defoverridable trick can also be really harmful for augmenting the protocol. For example, imagine the protocol has 2 functions today. In the future, you add a third function but you also provide a default implementation for it. Now implementations have no idea there is a third function! Maybe they would like to implement it but the default implementation means no warning is emitted. Situation can be even worse on libraries pre-1.0: what if you decide to rename a function? With the default implementation, they won’t even know something was renamed and developers will be unaware the contract even changed, wondering for hours why their implementation isn’t being called.

While we cannot solve the first problem (the one the developer will not get a warning if new functions are added to the protocols), we can solve the second one. Instead of:

# Inside quote
def add(a, b) do
  a + b
end

defoverridable add: 2

# Inside implementation
def add(a, b) do
  # ...
end

We should have:

@overridable true
def add(a, b) do
  a + b
end

# Inside implementation
@override true
def add(a, b) do
  # ...
end

This way if you override something that was not marked as overridable or something that does not exist, you will get a warning. It also improves code readability, because anyone reading the code will see the whole purpose of add/2 is to override a callback. Previously readers of the code could be left wondering why there is an add/2 function in the first place.

I want to send a proposal for the new overridable system after Elixir v1.4 is out. So you need to be aware of the limitations of the current system today.

Thanks for your reply. Just to clarify then, the @overridable/@override system is not in place yet, but will be submitted in a future proposal?

That’s the only part I don’t agree with. If you are already using a protocol then you already have a behaviour. Protocols do not allow default implementations because it generating coupling between protocols and implementations. We already have the coupling at the specification level, which is the point of protocols, but moving it to the implementation level means that changing the default implementation of a protocol will affect all implementations that depend on it.

The __using__+defoverridable trick can also be really harmful for augmenting the protocol. For example, imagine the protocol has 2 functions today. In the future, you add a third function but you also provide a default implementation for it. Now implementations have no idea there is a third function! Maybe they would like to implement it but the default implementation means no warning is emitted. Situation can be even worse on libraries pre-1.0: what if you decide to rename a function? With the default implementation, they won’t even know something was renamed and developers will be unaware the contract even changed, wondering for hours why their implementation isn’t being called.

I completely understand the rationale there, but like I said, this is a port from an OOP system where there is coupling at the implementation level, and a lot of the functionality depends on it. The idea is that it’s up to the developer of the library to make future changes to default implementations such that they preserve backwards compatibility or at least log some warnings to let developers know if something is wrong.

Further, this is the point I was making before as regards the “feeling” of protocols vs behaviours—protocols are more loosely coupled and just have to conform to the interface, while behaviours can have default implementations and feel more coupled. What I “need” are “parametrised behaviours”, callback functions tightly coupled to an implementation but which can also receive a few custom parameters. Hence my initial solution of using protocol dispatch since it’s already been worked on, tested and tried, but hiding that away somewhat from the user to discourage them from thinking of the module as a protocol implementation, since as you said, that implies much looser coupling than what is actually expected.

I see two possible other solutions:

1. Ask the developers to explicitly opt into default implementations of certain functions, perhaps as parameters to the __using__ macro.
2. Split up the large interfaces into many smaller protocols with one or two functions each, being able to opt into default implementations using @derive.

Number 1 seems somewhat verbose, while number 2 may be something I consider in the “refactoring” phase once the system is actually working.

I realise a lot of the friction here arises from trying to port a large system architected with OOP in mind to Elixir which uses different idioms, but I believe that it’s easier to first move it over into an Elixir codebase and only then make large structural refactorings. I could not find any other examples of large OOP codebases being ported over to something like Erlang or Elixir, usually people opt for a full rewrite from ground-up.

Yes, it will submitted soon™.

Other than that, it seems you are well aware of the trade-offs involved, which was the goal of my original reply. I would only add that you can also provide default implementations for protocols, if you would like to, exactly as you do for behaviours (but they are also discouraged for exactly the same reasons).

I always find porting such systems fun, exactly because of the challenges in converting from OO to FP.

Heh, porting my C++ ECS to Erlang years ago is still one of my most difficult conversions, I was never able to get the performance to anything that I’d consider acceptable (though the purity that I had to sacrifice in C++ for performance made it oh so clean ^.^).