Elixir and BPM (business process management)

And I am not going to prove anything for anybody with such shallow interest and prejudices.

Itā€™s more about ā€œtrying to get stuff done in a reasonable time frameā€ and yes, this can often be seen as shallow from an academical point of view. Wish I had the time to give proper attention to many interesting ideas but alas, we have to juggle a lot of balls to live decently these days. :102:

1 Like

Where BPM sorts of solutions can be very powerful is when you have somewhat technical domain experts who canā€™t code, but can model or read a business process diagram. Spreadsheet wizards tend to find this kind of pseudo-coding easy. I walked through an accountant responsible for our sales commissions how to use Process Builder (Salesforce tool) and sent him some documentation to reference. It took him a couple weeks, but he automated a lot of the headache in that process without me having to spend the time to learn the domain in depth to code up a solution. Iā€™ll have to rebuild that piece pretty soon anyway (it is a Salesforce tool after all), but it has provided a lot of value in the mean time, and now I have something to reference when scoping the long-term solution out.

Poorly/cheaply/quickly built applications can run into the issue of requiring a developer resource to make changes to processes, and that can be very problematic for a business. It can get worse when the business hacks together some other solution with spreadsheets and now the source of truth is spread-out (pun intended). As much as I dislike solutions like Salesforce, they do allow for quick, point-and-click changes which is something that Rails/Phoenix approaches which require a developer resource arenā€™t very good at unless the change is designed for.

BPM / declarative business process modeling solutions would be good for communication and frequently changing domains. That being said I think the line where a business should just call up an experienced consultant and have them build it is sooner than typical.

3 Likes

Hm. A few points:

  1. Poor domain specialist in a suit pretending to be an expert can be very problematic for a business as well. It all depends on where do you choose to place your trust.

  2. Domain experts using Excel or executable charts tools can work quite well of course. Itā€™s just that I cannot see any evidence this is quicker or cheaper than hiring a senior programmer for a year and have them sit and talk with your experts so he can (a) solve most of the problems, and (b) devise a DSL syntax with which the domain experts can extend / augment the system. Maybe I am wrong (as I said several times :003:) but I simply cannot find good articles to convince me. I have read a few and they seem to address very niche problems.

  3. I do wonder if hiring a programmer wonā€™t be cheaper in the long run, provided they know what exactly must be done (not always easy). I was consulting for an org that fired all of its domain experts and failed to listen to me when I told them that they should not lay off certain two of them for a while because without their knowledge I would not be able to actually continue working on their systems! They did not bother to answer at all and I guess they assumed I am a cretin because shortly after they fired the mediator through which I was consulting for them (and thus fired me as well). My point here is that I heard their CFO once say ā€œwe have way too many suits in hereā€ which is cynical and ugly but what I extracted from this was that the CFO found them useless. Maybe they would have been deemed useful if they knew how to make executable charts? Maybe.


I agree that point-and-click and executable charts solutions are sometimes necessary. I just question how often. A programmer can very quickly find themselves irrelevant in todayā€™s market and while I am very okay with learning FP and reducing my hiring opportunities a bit, I am absolutely not okay with working on enabling executable charts culture and tech in a company and find myself unemployable in a few short years because I havenā€™t kept up with the market in the meantime.

I definitely agree most of what you said except

If I can automate myself out of a job Iā€™d consider that a win! Now I learn something new and work on some other interesting problem. Thereā€™s always more interesting problems and topics than anyone ever has time for. Sales Commission applications are boring - better to automate that away so thereā€™s time for better things.

Figure it out. Make it right. Make it easy. Train someone else to do it. Find another problem. Repeat.

I also doubt the likelihood that a capable developer/engineer could automate themselves out of a job. Good software architecture is not simply simple. Iā€™d say the self-automated-destruction scenario is more likely to occur from gate-keeping or information hoarding type approaches to a job. Designing oneself to be indispensable when unnecessary is more likely to lead to that scenario of finding oneself unemployed and out of date. It might work in dysfunctional organizations, or just temporarily, but if thereā€™s any justice - not forever.

Of course technology advancement is accelerating, so maybe the real solution is less jobs, more gigs, and systems to make that a viable living.

2 Likes

Oh, 100% with you here. Exactly what I think as well. I was just expressing concern that if I keep my head buried in such niche problems like the Salesforce tools you mentioned, then I might find myself unemployable if I donā€™t keep up. Outside of that, you said it better than I could!

True. A really good programmer can solve a plethora of problems for their entire career. We should not try making ourselves irreplaceable, thatā€™s not what I meant. I mean that we should (1) automate the boring stuff and (2) keep ourselves current and our minds sharp.

Yep. If only there was not so much artificial inhibition of economical and political modelsā€¦

1 Like

I do not know salesforce and itā€™s process builder. I fear salesforce will use a proprietary DSL for modeling flows. BPMN is used by a lot/most specialized BPMSā€™s, making the models portable (no vendor lock-in). Bpmn is not like a spreadsheet, it is a visual programming language (external dsl) with blocks representing usertasks (with a pointer to a form), servicetasks (with a pointer to a service, input and output params), decision elements for processflow like arrows with decisions (salary > 100000) and decisiontasks (containing a pointer to a decisiontable - spreadsheet like, see DMN1.1), events etc. a BPMS is an orchestrator, not a development tool for services, forms (for these you could use a separate low code solution of course) etc. a BPM consultant is just a programmer. And it is not difficult to learn for regular programmers like us. There are people like that Bernd RĆ¼cker that I mentioned who find that the BPM modellers they design are intended for programmers, not ā€œcitizen developersā€. I agree with his reasoning. Non-technical people are not developers. But blocks and arrows + spreadsheets are more understandable for business people so it eases communication, moreover the programmer also gets an easier grasp / overview of (complicated) processes. These DSLā€™s being executable by an interpreter is a pre.
BPM is not only about modeling, it is about improving processes also https://en.wikipedia.org/wiki/Process_mining . During the execution of a process you record all sorts of info that you can analyse automated and use to improve. It is a tool for easing collaboration between users in a company. Users see usertasks to be fulfilled popping up realtime with a timeframe (set in the BPMN usertask element) wherein the task should be done, if they have a role that permits it (defined by the ā€œswimming laneā€ BPMN element wherein the usertask resides) . They can follow the progress of a workflow instance. Etc.

This is definitely true. Iā€™d say vendor lock-in is a pillar of their business model. Iā€™d love to see an Elixir solution take on Salesforce - thereā€™s a lot of slow-moving cruft in their product thatā€™s ripe for disruption.

1 Like

What do you mean with an Elixir solution take on Salesforce?

I guess I mean a comparable platform for the common business software needs Salesforce provides.

Features like user-management, permissions/authorization, admin interfaces, reports, forms, approval-processes, etc. Itā€™s also used to build custom applications or install packages from a marketplace and network users with consultants. Problem is that to develop on the platform youā€™re giving up version control, open-source, and developer experience - a lot of this due to the vendor lock-in tendencies over time. Itā€™s a slow moving Java business platform, and when they eventually get around to integrating crucial features that have been around for a while it tends to be limited due the technical debt incurred from this approach over time.

For a variety of reasons Elixir makes sense as an alternative building block for a product like Salesforce.

  • Salesforce has a fairly robust metadata system to define ā€œobjectsā€ and configure things like page layouts, defaults, field-level permissions, etc. Itā€™s simple enough to store this meta-data in a database, but Elixir has ETS for very capable in-memory caching which would put this kind of capability on steroids, or at least make it significantly more flexible. There are a lot of abstractions built on top of the meta-data system that allow for the point-and-click management of this sort of system, so itā€™s a very core part their success.
  • Salesforce has an add-on marketplace much like Shopify except thereā€™s a lot more overhead to getting your package approved and available. I donā€™t see why add-ons couldnā€™t just be another Elixir application/component/service/what-have-you. Separate database, separate logic, etc. Just point-and-click integration of a new business capability. I was reading recently in this forum that they recommend developing Shopify add-ons in Elixir rather than Ruby for these sorts of reasons. Iā€™m not really a fan of the SaaS approach where your data is being handled by an external service, so I really like the idea of running the software on your own machines.
  • Most of the complicated nature of developing with Salesforce is due to their multi-tenancy model which requires an unnatural bulkification of imperative Java 6-like code and a lot of work-arounds for various limits. Salesforceā€™s multi-tenancy approach could be circumvented entirely by just spinning up a new container/server with the software. The limits are there to prevent service disruption of other customers on a server, so itā€™s understandable, but completely unnecessary in the age of containers. Even in the same multi-tenant environment it would be trivial to manage those sorts of concerns in a functional language with the concurrency support Elixir has.

The difficult part is designing something of this size with all the point-and-click conveniences. The golden-feature is when a somewhat-technical domain expert or administrator type can put together a maintainable business process flow without needing years of development experience. So not at all a trivial undertaking, but Elixir is what Iā€™d pick to build a comparable business application platform given the chance.

Most BPM products are a vehicle to sell consulting and training. Iā€™ve seen them waste millions and millions of dollars. Havenā€™t really seen them work outside domains like Salesforce. I have found myself thinking about an Elixirish way to ā€œsolveā€ this though - what Iā€™d like is a DSL that can yield rules/flows that can be reviewed and marked up by a business expert, but implemented by a programmer.

2 Likes

Check out the camunda web-based modelers on github (BPMN2.0 and DMN1.1). I use those with my own elixir/phoenix backend. Camunda has an opensource backend interpreter also for these. Written in java.
Zalando and Goldman Sachs seem to be enthousiast (as I mentioned before on this forum)

Everything that is more or less an executable flowchart diagram looks to be hijacked for such purposes, even if the tech itself can be helpful.

IMO thatā€™s a relatively good way to go about it ā€“ has to be a language whose macros yield AST, like Elixir or a LISP dialect.

Everything? Isnā€™t that a wild guess?

There is BPMN and DMN already, open standards with many years of research behind them. And understandable for business experts. But you want to reinvent the wheel? No, worse even, develop a proprietary DSL?

2 Likes

Of course it is a wild guess. I dipped my toes in the area long ago and decided I donā€™t like the preliminary red signs, as discussed above. And I already admitted I might be wrong. But I wonā€™t be dipping further to find out ā€“ I got scared away by the first wave of solutions and I am unwilling to try again. Call me biased again :stuck_out_tongue: but it is what it is.

As @jeremyjh stated, tech like this attracts expensive consultants that are dying to do a vendor lock-in in the company.

I have read in Excel-related threads in HackerNews how programmers trained accountants and analysts to use a mini-DSL they devised with great success.

So IMO thereā€™s something else here ā€“ maybe the BPMN tech is too big and maybe intimidating for many (not gonna say all)? And maybe a small proprietary DSL is less confusing to a lot of business people, much like Excel is complex but still feels graspable?

Not sure. But itā€™s one of the possibilities.

That in itself is a business opportunity.

Quote:

Those new to BPMN understandably find it overwhelming. There are flow objects, connecting objects, swim lanes, and artifacts. And thatā€™s just the categories within the notation. Overall, there are over 40 different elements, each with rules about when it can and cannot be used.

(In contrast, our quick list of workflow diagramming elements in the Business Process Analysis course contains the 5 elements that are used by the majority of practicing business analysts.)

Itā€™s invaluable to document and capture your business processes for communication purposes.

I find that it is usually the detail necessary for the automation aspect that can get into the way of making the information the stakeholders find relevant readily available (which often needs to be solved with even more automation). Of course without automation, process model documents can become stale rather quickly as updates can easily be missed.

1 Like

It might be that the elixir language is overwhelming and intimidating for many (not gonna say all) also. I think it is even a lot more intimidating than bpmn. I leave the search for quotes to you.
What exactly is that business opportunity Peer? If you mean the proprietary DSL suggested: that can grow out by business needs to a monster that is much more horrifying than bpmn. It might be wiser to start with a bpmn subset. The open source gui modelers are there already, available on github, menu-items can be removed.
Itā€™s a pity that mostly opponents speak up on this forum, and none of them has much knowledge of what is criticized. I have had private conversations with elixir devs that had another attitude (and then suddely this brightball - a frequent poster here - comments pop up on hacker news), who knows who else here maybe just has no taste to speak up. But it is what it is.

The way I understood that quote was to train people to formalize their business processes by focusing on diagramming (i.e. visualizing) with a BPMN subset without any intention towards automation but simply as a basis for communication, i.e. as artefacts to judge their progress on the path of continuous improvement.

and then suddenly this brightball

That quote was so general it could have just referred to capturing business processes with a standardized notation to improve visibility and discourse - it wasnā€™t clear whether it actually referred to ā€œLow Code BPMā€ and its inherent automation.

A lot of people groan about UML. But UML is just a standardized notation that can be used to visualize valuable information. But it canā€™t guarantee:

  • That the most valuable information is put front and centre and depicted in the most appropriate manner.
  • Everybody will understand it.
  • Some tools (e.g. Rational ClearCase) wonā€™t get people to hate it.
1 Like

Not what I understood, see at least

This is less general: https://news.ycombinator.com/item?id=16288486

I attended a five-day training on IBMā€™s toolset in 2008. Then I watched a project throw money into a burning barrel for a year and deliver next to nothing. Am I an expert? Noā€¦but this is not cheaply acquired skepticism.

1 Like