More optics in Funx: when you need multiple foci, use a traversal.
Includes a Livebook
I’ve never been super clear on the concept of functional programming and this is the first time I’ve heard it linked with “declarative” code, which is also something I’m frankly not super sold on as a useful category. But in the interest of hopefully gaining some clarity on both, my question is why you frequently seem to target pattern matching as the main “alternative” to these more functional tools? My understanding is that pattern matching itself is touted as a fundamental part of the Elixir’s functional tool kit? Again, not claiming any firm grasp of any of these categories so maybe I’m reading too much into that example.
But pattern matching is not the tool I would reach for if I wanted to solve the problem in your hypothetical:
Our domain requires that a transaction’s item price match its payment amount. We don’t need a single focus: we need both the item and the payment.
This is a boundary problem: we want to prevent an invalid transaction from being processed.
Wouldn’t we optimally want to prevent such a transaction from existing? Pattern matching works great as a tool for controlling execution, but I don’t think of it being a good candidate for validation for this reason. As in your example, it might be that no function head would match the invalid case, but not due to any intent to validate the data being passed, but because Elixir’s unique architecture enables us to forego explicit handling for the majority of error cases (“let it crash“). If anything it seems like your approach is more about defensiveness (which might be justified in the domain) than anything else. Wouldn’t the proper comparison be using a tool like Ecto to handle validating the data inputs (which would be modeled/tested etc)? What is not “declarative” or “functional” about that approach (which is a standard pattern)?
1 Like
The term “declarative” is just very overloaded and has several popular meanings which are only vaguely related. One of those meanings is that of “referential transparency”, essentially pure functions, which of course has a lot to do with functional programming.
My own preferred meaning (and we have discussed this before) is that of code which rebuilds itself from scratch on each invocation rather than attempting to “patch” itself with small state changes. This is actually a programming technique more than anything which is why I often write “declarative style”. This technique is helpful for preventing bugs and tools that enable it (like React) are very powerful as long as you know that’s what they’re for. Unfortunately many do not.
The technique is strangely hard to put into words, but it has a great “know it when you see it” quality, so here are some articles with didactic examples that I found helpful.
All three of these are about frontend, but the technique is very broadly useful. Actually it was the description in section 6.3 of the FoundationDB paper that finally made it click for me. A distributed OLTP database is about as far from frontend as you can get and yet they present a real-world case of the exact same technique preventing bugs!
1 Like
Appreciate your being game about repeating the discussion but I think the reason it hasn’t clicked for me is because I think about these things in terms of the backend where I’m not sure what to make of this idea:
So my question here remains the same, what are the advantages to the kind of tools being promoted in OP, vs just having a function that encapsulates the validation logic (so it can be reused, tested etc)?
1 Like
The version of declarative in the OP is one which I would describe as “factoring out functions plus a fancy DSL”. There are times where you want a fancy DSL, usually for aesthetic reasons. Like, it’s kinda nice that Ecto queries look like SQL so they’re recognizable. And as for factoring, we all know how to do that.
As far as I can tell we have entire frameworks built around that declarative (e.g. Ash) and TBH I don’t really get it either, but hopefully someone else can provide the defense you’re looking for.
However, I think you understand “my” declarative more than you realize:
What is letting it crash but rebuilding the program from scratch? 
This is actually exactly the same concept. OTP supervision and React components take the same fundamental idea of “turn it off and on again”. The only real difference is that the latter are more sophisticated, but in fairness React is also decades newer. If you try to reason through “how do I build OTP supervisors but with dozens of processes in unique roles that can retain their complex state” you will eventually reinvent React from first principles.
I’m using “declarative” in contrast to imperative or procedural control flow, not as a synonym for “pure FP.” What I mean is that declarative code describes intent, while procedural code proscribes execution.
2 Likes
OK, that’s the definition I’m familiar with (and so still not sold on). How does using a “Traversal” pattern describe intent better than a plain old validate function?
Yes, agreed. Funx implements the underlying patterns. The DSLs are there to improve the syntax and readability, but behind the scenes they just compile down to Funx constructs.
I tend to talk about “let it crash” more specifically, starting from invariants. You don’t try to fix or defend against broken invariants. If “a transaction must always have an :item” is an invariant, then a missing :item is a crash, not something to recover from.
It’s confusing because the definitions are all related, but as I said, vaguely.
The reason writing code that “defines intent” is useful is that you get to avoid manually defining the codepaths for complex state transitions. Instead you rebuild the state from scratch. This is good because keeping disparate pieces of state in sync by hand is bug-prone and laborious.
The OP is not a good pedagogical example of this because the code in question isn’t really stateful at all.
Your definition is interesting but strikes me as fairly esoteric, both in terms of use and remote from connotations of “declarative.” I think I have a vague sense of the connection you’re getting at, but I’ve also heard “self-healing” and “recoverable” as descriptions of that kind of design either of which seem like a much better term?
A Traversal is an optic that returns a list of foci. It’s just a pattern for combining Lens and Prism.
bind Traversal.to_list_maybe(Processor.cc_payment_trav) states “only let through a transaction that has both an item and a credit card payment.”
guard PaymentMustMatchPrice further narrows that to transactions where the item price and payment amount match.
In the pipeline, everything below is skipped if either the traversal or the guard does not match.
The :raise option is where we state that this is an invariant. It either succeeds or raises an error (crashes).
If you’re thinking “couldn’t this just be a single validation?”, yes.
I think we’re talking past each other a bit on terminology. My use of “declarative” isn’t meant to imply self-healing or recoverable behavior. The examples here fail fast by design. By “declarative” I only mean that the code is describing intent and constraints, not prescribing control flow.
1 Like
I did not invent this meaning; if you check the three articles I linked you will see it is actually in common use (one is literally the official React blog).
FWIW my favorite term to describe how React works is that you write “rewindable” code, but nobody will recognize that if I use it.
Healing and recovery are nice but they imply errors. This is true of OTP crashes but is not true of the general technique. You want to rebuild state not only in the case of a crash but in the case of any state change. Crashing is actually a special case.
If you have spare time and you like databases study FoundationDB and this will become very clear. The whole idea is that they use the same recovery path for state changes and crashes (and they do call it Recovery, and it is a bit of a misnomer for this reason). Also Hobbes uses the same design btw
Another related use of “declarative” is in the context of SQL, where declarative code is useful for a very specific reason (data independence) which is not commonly understood by most devs unless you’re into databases. Incidentally the declarative-ness of SQL has nothing to do with query optimization, which is what most people seem to think. That article is also a fun treatment of the concept of “declarative”. It really is a bit of a rorschach term lol.
Optics are a generic concept. I’m using a common pain point in the examples as a concrete way to show how they work, not as a claim that our only choice in solving this problem is between pattern matching in the function head and the solution I’m showing.
The cause of the confusion (and this long confused me as well) is that in simple cases imperative code looks an awful lot like code which describes intent.
def validate_length(string, length), do: String.length(string) < length
def validate_not_empty(string), do: string != ""
def validate(string) do
validate_length(string) and validate_not_empty(string)
end
Like, this code describes its intent pretty well. Is it declarative?
There are actually cases that accentuate the difference well, but they are generally centered around complex state transitions.
No, my reply was to @garrison who is using the terminology differently. I don’t really understand either use.
OK, definition has expanded a bit, but nothing I haven’t already heard. What I haven’t heard is how/why this tooling in particular does those things more than other validation patterns.
Yes, sorry, I should have been clear that I understand the concept of lenses and traversals and they seem like an interesting pattern but my question is more directed that the (implicitly positive) rhetorical use of “declarative.” It doesn’t really help me understand why I should consider this pattern over ones that are simply more familiar to me. Again, not specifically a problem in your presentation, I have never found uses of this term very compelling.
Oh, you’re saying you don’t like my use of “declarative” as something implicitly positive.
That’s a fair point. By its nature, declarative code, as I’m using the term, adds at least one level of abstraction, and that indirection has real costs.
Also, under my definition, It’s probably safe to say that only SQL is truly declarative. The rest of us are on a sliding scale.
1 Like
There were many different implementations of optics in Elixir, namely Lens, Focus and now Funx. I’ve made my own some time ago, called Pathex
The main problem with all other optics solutions is performance, especially with Funx/Witchcraft approaches, where results are wrapped into structures, introducing runtime overhead. In *ML family of languages (and similar languages like Rust) these things are optimized at compile-time into branching, while in elixir every such operation introduces a dispatch on the returned structure and initialization of the new structure.
At the same time Pathex always compiles into almost the most efficient code it can, inlining everything it can, concatenating paths, etc.
Check it out: Pathex v2.6.1 — Documentation
2 Likes
I like your work!
And I agree. There are times when we need to be as close to the metal as possible, where eliminating runtime abstraction layers matters.
