heathen

What is the best way to open external connections in a CQRS\ES application?

I’m working on a project one part on which requires information (for example, external systems resource consumption info) from one or many external sources of the same type. These external systems don’t have push or subscription options so I need to poll them periodically. The tricky part is that users must be able to add new or delete already added sources at any time as well as to change connection properties (like url, auth info, polling interval and so on). It is possible that in the future it will be possible to get information on subscription scheme.

This particular application’s part is a kind of connector\translator events from the external systems to the core domain. I’m going to use Commanded on the whole project.

I want to put this part into a Connectors subdomain and describe the Connection aggregate (names are debatable). I want to start polling right after this aggregate creation and stop on deletion.

So the question is what is the best way to start the polling processes?

In general it could be a GenServer process (as I need to keep some service information like the last success call timestamp, prevent parallel connections and so on) with poll request scheduling. I evaluate Commanded’s process manager but is it a good idea to make external calls right from it (through ports\adapters, of course, but still)? Or is it better to start an additional process via the same process manager on ConnectorAdded\ConnectorResumed events and stop it on ConnectorDeleted\ConnectorPaused?

6 comments

#event-sourcing #ddd #cqrs

1 1804 6

2019-02-27 10:58:36 UTC

Most Liked

bottlenecked

Hmm… I think that @factoryd is on to something here perhaps. Greg Young warns on a few occasions “please dont tell me that you’ve built an event sourced system”, meaning that it only makes sense applying the pattern to a small part of your system where it’s actually really needed, not least because ES comes with a ton of overhead. There are actually quite a few wartime stories about botched ES attempts, and a common theme is attempting to force ES everywhere.

The reason I’m saying this is that it looks like you should perhaps think about separating your app into 2 parts: the first should only concern itself with the polling aspect (configuration screens, external apis etc.) that will produce commands that will then be pushed on to the second part of your system which should only be concerned with accounting (your core business, which may actually benefit from ES), that will simply consume the commands from the first part, convert those into events, apply them and finally publish them for the rest of the system to act upon (you know, the canonical ES flow).

Again, that would be my advice: do not mix a ton of different concerns in an “event sourced application” because this pattern tends to multiply the effects of bad system design decisions

Post #5

bottlenecked

OK, I see. Perhaps it’s worth a shot then as a learning experience! But if this solo/personal/internal project ends up being used everywhere (as infrastructure projects are sometimes wont to be) you should make sure that you will be given the required time (by your manager for example) to revisit your designs in case they end up not fitting the problem

Post #7