Architectural Design Questions: Central "microservice" to process payment, business and accounting for multiple applications on different servers

Phoenix Forum Questions / Help
, , , ,
1

Hi

I’ve been contemplating this issue for quite some time and haven’t come across with a good solution and was wondering on your viewpoints and how you’d tackle this problem in the most efficient way possible.

The Problem:

I have many different projects built with Elixir/Phoenix and some are on the way to being released. If I only had 1-2 disconnected apps this would not be a problem.

During the years I’ve added a variety of bits and pieces such as a user management. CMS, Payment processing, accounting, invoicing and a variety of tasks which I’ve had in different libraries and programming language.
IN order not to “reproduce” the same functionality on all application I even created a phoenix library which installed such as phx.gen.auth.
HOWEVER due to my fiddling with many different technolgies on frontend (switching between tailwind, bulma, pure css + others), backend processing, even elixir apps, server side, cloudwise (kubernetes)
I soon found out that managing such a library created too many dependencies and it complicated the workflow enormously. Backwards compatibiity, upgrades, bugfixes and differences in Elixir, Erlang and PHoenix AND other libraries upgrades and deprecations made it impossible to actually get anything done over the array of the majority of the projects as the dependency cycle grew. Ex: I’ve had 2 apps implode 2 times because of a Nebulex upgrade which was not immediately visible in the authentication system (seconds changed to milliseconds, users kept getting logged out in a few minutes).

A HUGE FRICTION OCCURRED AND FRUSTRATION ENSURED. I even gave up the library concept after 3 years of hard work and decided to go anew with the projects with the latest phoenix. But i’m constantly finding I need certain bits here and there and the library idea is still around because of this.

I’ve tried umbrella apps, monoliths, the library approach and can’t quite find a sweet spot. Because I feel that the productivity/efficiency/capabilities system is kind of “missing”. The ONe thing I’m mostly missing is a plugin system akin to systems such as PHP have. Or even the Tcl’s dynamic interpreter stuff which means pushing changes accross systems is almost seamless.

I was thinking that the admin interface should be built on something totally different and that it could look identically on all apps to minimize this friction in the event of using a central reusable library model. What are your thoughts on this?

Latest Ideas

I was considering building a central microservice system which would instead do all of these things to minimize friction.
The idea would be that it would communicate with an API where i’d be able to build as simple library to handle this.
THe “problem” i’m facing is for payment processing and online shop i’d have to redirect from App A → App B → App C → App B → App A

  • App A (Any App making usage of the microservice)
  • App B( MicroService)
  • which would redirect to the payment processor (App C) Which could also mean a more complex architecture because there’s also a gateway in between

This would probably confuse the end users as iframes aren’t really that good.
An option I thought about was to add a subdomain which redirects to App B (the microservice) without it being “visible” to the end user. Although the API communication would still need to occur.
This would greatly simplify everything as I’d only have to manage the API library instead of the whole cycle of dependencies and I’d keep
I’d also like for a way to interconnect with the system such as that user info may be available accross platforms.

What are your thoughts on this?

Thanks.

2

You could put an app in front of all your services. The browser would only talk to that app, and that app would call the different services.

We have a similar pattern, it is an had oc implementation of the Saga Orchestrator Pattern:

  • The frontend sends a message to start some operation. In your case that would be a payment.
  • The elixir app creates the operation in database and starts a workflow
  • The workflow talks to some service to validate the payment and get some technical data
  • The workflow then talks to some other service to create a payment URL with your provider, and sends the URL back to the frontend (in our case we send those updates with an async message queue, but you could wait in the original HTTP request to get the response directly)
  • At some point the workflow has finished its work and stops
  • The user pays on the bank payment page
  • The user is redirected to your frontend (or you receive a workflow). That is sent to your elixir app that will start the workflow with the new payload, load the operation state, and continue a few steps until it is done.

Note that at each step you must persist state if you want to resume from there in case of crash.

Also, in case of error, your workflow state must remember each step it has executed in order to be able to run compensations. For instance if the followed steps were this: create_payment -> bank_payment -> mark_order_paid -> error Then you should run compensations: cancel_order -> refund -> mark_payment_failed.