Receiving & processing events via incoming Webhooks

I’m an Elixir developer (relatively new) from Australia and I am loving it :smile:

I’m working on a SaaS startup which is an industry-specific / niche Learning Management System (LMS). Courses will be launched and played externally (by multiple 3 party solutions) before returning to my system. However, I need to retrieve and process course progress events (course started, course module completed, course completed etc.) for all courses / students on my platform, from the 3rd party services (whilst course are in progress). If the startup is successful, this will end up being a lot of events coming in via webhooks, all the time!

I realise the default answer is probably something along the lines of this: Webhook Service @ PagerDuty

But I am wondering if we should looking at utilising Amazon SQS (our app is already hosted in AWS) with the new Broadway library? I am a little confused about which of GenStage, Flow and Broadway would best fit my use case. Appreciate any thoughts/advice on this. Thanks!

4 Likes

Where in Australia are you based? I’m up in Coffs harbour area but will be at the Melbourne meetup on the 21st if you’re down that way.

The work load on each hook sounds pretty low so personally I’d just process them in a normal phoenix request flow and plan to scale horizontally at that point. That should get you really far and is easy to scale horizontally.

1 Like

Wow…I am in Coffs Harbour too!
I thought there were only two Elixir devs in town (me and another guy I know).
Now that there is three of us, we’re going to have to start up our own regional NSW meetup group. Haha.
We should catch up one day…

Thanks for the feedback on my question.

3 Likes

Wow that is crazy.

A meetup would be great I was flying down to the Sydney elixir meetups each month just to get to hang out with other programmers for a while but we have a new baby so I wanted to cut out that as an expense for a bit. I never had luck meeting technical people at the startup grind coffs events either.

Congrats on the new baby. We have a 5 month old in my house too!
I’ll send you an email about a catch up.