Elixir and BPM (business process management)

I do not know salesforce and it’s process builder. I fear salesforce will use a proprietary DSL for modeling flows. BPMN is used by a lot/most specialized BPMS’s, making the models portable (no vendor lock-in). Bpmn is not like a spreadsheet, it is a visual programming language (external dsl) with blocks representing usertasks (with a pointer to a form), servicetasks (with a pointer to a service, input and output params), decision elements for processflow like arrows with decisions (salary > 100000) and decisiontasks (containing a pointer to a decisiontable - spreadsheet like, see DMN1.1), events etc. a BPMS is an orchestrator, not a development tool for services, forms (for these you could use a separate low code solution of course) etc. a BPM consultant is just a programmer. And it is not difficult to learn for regular programmers like us. There are people like that Bernd Rücker that I mentioned who find that the BPM modellers they design are intended for programmers, not “citizen developers”. I agree with his reasoning. Non-technical people are not developers. But blocks and arrows + spreadsheets are more understandable for business people so it eases communication, moreover the programmer also gets an easier grasp / overview of (complicated) processes. These DSL’s being executable by an interpreter is a pre.
BPM is not only about modeling, it is about improving processes also https://en.wikipedia.org/wiki/Process_mining . During the execution of a process you record all sorts of info that you can analyse automated and use to improve. It is a tool for easing collaboration between users in a company. Users see usertasks to be fulfilled popping up realtime with a timeframe (set in the BPMN usertask element) wherein the task should be done, if they have a role that permits it (defined by the “swimming lane” BPMN element wherein the usertask resides) . They can follow the progress of a workflow instance. Etc.

This is definitely true. I’d say vendor lock-in is a pillar of their business model. I’d love to see an Elixir solution take on Salesforce - there’s a lot of slow-moving cruft in their product that’s ripe for disruption.

1 Like

What do you mean with an Elixir solution take on Salesforce?

I guess I mean a comparable platform for the common business software needs Salesforce provides.

Features like user-management, permissions/authorization, admin interfaces, reports, forms, approval-processes, etc. It’s also used to build custom applications or install packages from a marketplace and network users with consultants. Problem is that to develop on the platform you’re giving up version control, open-source, and developer experience - a lot of this due to the vendor lock-in tendencies over time. It’s a slow moving Java business platform, and when they eventually get around to integrating crucial features that have been around for a while it tends to be limited due the technical debt incurred from this approach over time.

For a variety of reasons Elixir makes sense as an alternative building block for a product like Salesforce.

  • Salesforce has a fairly robust metadata system to define “objects” and configure things like page layouts, defaults, field-level permissions, etc. It’s simple enough to store this meta-data in a database, but Elixir has ETS for very capable in-memory caching which would put this kind of capability on steroids, or at least make it significantly more flexible. There are a lot of abstractions built on top of the meta-data system that allow for the point-and-click management of this sort of system, so it’s a very core part their success.
  • Salesforce has an add-on marketplace much like Shopify except there’s a lot more overhead to getting your package approved and available. I don’t see why add-ons couldn’t just be another Elixir application/component/service/what-have-you. Separate database, separate logic, etc. Just point-and-click integration of a new business capability. I was reading recently in this forum that they recommend developing Shopify add-ons in Elixir rather than Ruby for these sorts of reasons. I’m not really a fan of the SaaS approach where your data is being handled by an external service, so I really like the idea of running the software on your own machines.
  • Most of the complicated nature of developing with Salesforce is due to their multi-tenancy model which requires an unnatural bulkification of imperative Java 6-like code and a lot of work-arounds for various limits. Salesforce’s multi-tenancy approach could be circumvented entirely by just spinning up a new container/server with the software. The limits are there to prevent service disruption of other customers on a server, so it’s understandable, but completely unnecessary in the age of containers. Even in the same multi-tenant environment it would be trivial to manage those sorts of concerns in a functional language with the concurrency support Elixir has.

The difficult part is designing something of this size with all the point-and-click conveniences. The golden-feature is when a somewhat-technical domain expert or administrator type can put together a maintainable business process flow without needing years of development experience. So not at all a trivial undertaking, but Elixir is what I’d pick to build a comparable business application platform given the chance.

Most BPM products are a vehicle to sell consulting and training. I’ve seen them waste millions and millions of dollars. Haven’t really seen them work outside domains like Salesforce. I have found myself thinking about an Elixirish way to “solve” this though - what I’d like is a DSL that can yield rules/flows that can be reviewed and marked up by a business expert, but implemented by a programmer.

2 Likes

Check out the camunda web-based modelers on github (BPMN2.0 and DMN1.1). I use those with my own elixir/phoenix backend. Camunda has an opensource backend interpreter also for these. Written in java.
Zalando and Goldman Sachs seem to be enthousiast (as I mentioned before on this forum)


Everything that is more or less an executable flowchart diagram looks to be hijacked for such purposes, even if the tech itself can be helpful.

IMO that’s a relatively good way to go about it – has to be a language whose macros yield AST, like Elixir or a LISP dialect.

Everything? Isn’t that a wild guess?

There is BPMN and DMN already, open standards with many years of research behind them. And understandable for business experts. But you want to reinvent the wheel? No, worse even, develop a proprietary DSL?

2 Likes

Of course it is a wild guess. I dipped my toes in the area long ago and decided I don’t like the preliminary red signs, as discussed above. And I already admitted I might be wrong. But I won’t be dipping further to find out – I got scared away by the first wave of solutions and I am unwilling to try again. Call me biased again :stuck_out_tongue: but it is what it is.

As @jeremyjh stated, tech like this attracts expensive consultants that are dying to do a vendor lock-in in the company.

I have read in Excel-related threads in HackerNews how programmers trained accountants and analysts to use a mini-DSL they devised with great success.

So IMO there’s something else here – maybe the BPMN tech is too big and maybe intimidating for many (not gonna say all)? And maybe a small proprietary DSL is less confusing to a lot of business people, much like Excel is complex but still feels graspable?

Not sure. But it’s one of the possibilities.

That in itself is a business opportunity.

Quote:

Those new to BPMN understandably find it overwhelming. There are flow objects, connecting objects, swim lanes, and artifacts. And that’s just the categories within the notation. Overall, there are over 40 different elements, each with rules about when it can and cannot be used.

(In contrast, our quick list of workflow diagramming elements in the Business Process Analysis course contains the 5 elements that are used by the majority of practicing business analysts.)

It’s invaluable to document and capture your business processes for communication purposes.

I find that it is usually the detail necessary for the automation aspect that can get into the way of making the information the stakeholders find relevant readily available (which often needs to be solved with even more automation). Of course without automation, process model documents can become stale rather quickly as updates can easily be missed.

1 Like

It might be that the elixir language is overwhelming and intimidating for many (not gonna say all) also. I think it is even a lot more intimidating than bpmn. I leave the search for quotes to you.
What exactly is that business opportunity Peer? If you mean the proprietary DSL suggested: that can grow out by business needs to a monster that is much more horrifying than bpmn. It might be wiser to start with a bpmn subset. The open source gui modelers are there already, available on github, menu-items can be removed.
It’s a pity that mostly opponents speak up on this forum, and none of them has much knowledge of what is criticized. I have had private conversations with elixir devs that had another attitude (and then suddely this brightball - a frequent poster here - comments pop up on hacker news), who knows who else here maybe just has no taste to speak up. But it is what it is.

The way I understood that quote was to train people to formalize their business processes by focusing on diagramming (i.e. visualizing) with a BPMN subset without any intention towards automation but simply as a basis for communication, i.e. as artefacts to judge their progress on the path of continuous improvement.

and then suddenly this brightball

That quote was so general it could have just referred to capturing business processes with a standardized notation to improve visibility and discourse - it wasn’t clear whether it actually referred to “Low Code BPM” and its inherent automation.

A lot of people groan about UML. But UML is just a standardized notation that can be used to visualize valuable information. But it can’t guarantee:

  • That the most valuable information is put front and centre and depicted in the most appropriate manner.
  • Everybody will understand it.
  • Some tools (e.g. Rational ClearCase) won’t get people to hate it.
1 Like

Not what I understood, see at least

This is less general: https://news.ycombinator.com/item?id=16288486

I attended a five-day training on IBM’s toolset in 2008. Then I watched a project throw money into a burning barrel for a year and deliver next to nothing. Am I an expert? No…but this is not cheaply acquired skepticism.

1 Like

Are there good examples of approaches that instead of using BPMN to generate code to run, code is written and a process diagram is generated from an analysis of the code? I expect the variability in code expression would pose issues. The trade-off is it requires a coder to implement, but it would solve the communication and “stale” problems. I guess in a space like Elixir I’m wary of generated code implementing concurrency well; Salesforce’s declarative tools have issues with scalability and implicit implementation details, it can be hard to anticipate bugs or know when something will fail.

Another idea in this space is event storming. It might be easier to guide our token Excel-technical user through some kind of guided Event Storming session to define the process through events, commands, entity definitions, etc. It could be that this DDD modeling exercise is a simpler set of vocabulary to learn than the BPMN lexicon. The question is can this approach specify with enough granularity to generate capable code?

Fair enough.

To be fair there are just too many other reasons why something like this could go off the rails.

With a brilliant tool and a competent Business Process Analyst (BPA) five days may just be enough time. However all too often some tools are sold as some kind of universal panacea to be wielded by people who are responsible for conducting “the business” but who don’t necessarily have the requisite BPA mindset that the tool is based on. I’m skeptical that a non-BPA could acquire the necessary mindset in five days (apart from being trained on the tool).

Often these type of ventures fail because of “people problems”. In the end the education of the tool user is more important. A competent BPA is an asset to the business; the right tools could make a BPA more effective - given the right circumstances.

Code-first asset generation tends to lead to more obfuscated results. In the end model-first generation should be more likely to succeed. However a successful result still relies on the tool user having a solid understanding of the relevant modeling principles.

1 Like

Oh, I didn’t work on the project that used the tools - we were evaluating it at the same time as another project and were included in the training as more of a courtesy or sales effort. Despite a lot of pressure to use these tools we ended up writing a traditional app and our project worked out fine. But I had regular contact with people from the department that adopted it and heard about their progress or lack of. They did train a bunch of people but most of the work was done by consultants who had more experience but of course that doesn’t mean it wasn’t largely a people problem or management problem. I’m sure someone, somewhere in the world has used that toolset successfully. But I’m deeply skeptical it is an efficient or reliable way to deliver the desired objectives.

1 Like

I think the issue is BPM tools are marketed toward senior management generally involve high touch sales and like most “enterprise” products generally don’t have much love from developers.

1 Like

For clarity: a bpmn modeler does not generate code, each model is persisted in xml. It is this xml that you can interpret in a backend language like elixir. I dare not think of a tool that generates a bpmn model from code. Keep in mind also that these tools are meant for orchestration only (and process mining / management). The bpmn contains pointers to services / forms etc, but no details about those.
Maybe you can try an event storming session with a bpmn editor instead of stickies, on a big screen. The screen would look like https://demo.bpmn.io/new (this is not the complete modeler, the real one contains a properties editor on the right and there may be more, I send the link just to give an idea).

Now that I’ve found this thread I’ll have to give a write up soon.

Short version though: BPM generally aligns with ISO certifications that companies aim for which require documenting business processes. Then you move from being able to take a flow chart and turn it into something that runs. Then you start hooking nodes of that flow chart into other systems to automate away some of the human bits where possible.

The power in it is the ability to start simple and then improve as you find the process being repeated a lot. Anybody can write up a process and drop it in a folder on a network drive to say “this is our on-boarding process” but without a system to enforce it, it probably won’t be.

With BPM in place you start with essentially a flow charted checklist, then you start routing it to different people based on responses in that checklist.

My interest comes because I see a lot of benefit from it for my wife’s small business. It could make her life a lot easier for managing everything from signing up new patients, filing insurance claims, handling employee license renewals, patient discharge, etc. Unfortunately, the systems are pretty complicated as most seem geared toward larger companies…and most are written in Java as well.

It’s very much a glue system that handles talking to both humans and APIs.

Simple examples would be something like an on boarding process for new employees that steps through paperwork, HR, benefits, legal stuff, systems access, etc. When you are done you are looking at talking to multiple people to ensures proper bits have been covered, creating email and calendar access, granting access to specific systems, etc. The whole process might take a few days, including check points and reminders. Somebody may need to review where each person is in the process and multiple people may be on-boarded at the same time.

BPM creates a way for that to happen in a manner that can be visually reviewed, understood, enforced and then updated.

2 Likes