Looking for some feedback on a Text-Processing feature with LiveView and OTP

Learning Questions & Discussions Resources Chat/Questions
, , ,
1

I’ve been tinkering on a LiveView project to get a feel for these OTP tools like Dynamic Supervisors, Registry, ETS, Et Al., and I’ve ran into a few spots where I could use some feedback. I have a good idea of what could or needs to happen, but I’m having trouble taking the concept into implementation with regards to some areas of OTP.

The feature I have in mind is a text input where a LiveView with a 1:1 Session Process sends TextChanged events to a variety of Text Analysis strategies for different sorts of metrics. (Sentiment, Editing, Difficulty, etc.) Some of these metrics are stateless (current word count), some of them are stateful (word counts over time).

The plan was we Dynamically Supervise an AnalyzerSubscriber that, depending on the kind of metric, collects various results from background jobs (handling AnalysisResultProduced events), puts the result in ETS under our Session ID, and publishes an AnalysisResultPrepared event so our LiveView knows to fetch from a handy read-model.

So there’s likely some sliding window type metric aggregation, maybe some rate-limiting, and different kinds of ETS tables.

You can find the WIP repo/branch here: https://github.com/zblanco/libu/tree/live_chat

A few questions:

  • I want to Dynamically Supervise these Subscriber / Collection processes under the also dynamically supervised Session Process - should I be using a spawn_link, Process.monitor or some different method altogether?
    • The subscriber supervisor and child processes should die with the session, but the session shouldn’t die with the supervisor or its children. I’m thinking monitors since a spawn link will kill the session if the subscriber supervisor dies? Are there some consideration I’m missing here?
  • I’m a little bit in the “don’t know what I don’t know” category with regards to many of these OTP tools. I can tell libraries like Flow, GenStage, and Broadway might fit in here, but I’m not quite sure how. Some of these Text Processing jobs have dependent aspects (e.g. different jobs might depend on a Tokenized Result).
    • The plan was to consume an event like %AnalysisResultProduced{analyzer: :tokenizer, result: ....} to then do some dependent work - is there a way to do this kind of dynamic/dependent processing with Flow? My spidey-sense says I might be re-inventing the wheel.

Any input would be appreciated!

2 Likes