How would you build a zapier clone in Elixir?

dopomecana

I recently had a interview where I had to design a zapier clone.
From the feedback I understood that my solution was not very elixirish. I think what am I missing is the big picture thinking in Elixir.
How would you guys do it? How would you structure your services?

For those who don’t know what zapier is:

Zapier is an automation platform that lets you connect different apps and software, so that when something happens in one place, a different action gets taken somewhere else. Here are some examples: You could use Zapier to get a team chat message every time someone fills out a form, an email when someone updates a shared spreadsheet, or automatically add someone to an email newsletter audience or CRM when they fill out a contact form.

Basically this may be viewed as an incoming API call that triggers one or more actions(external API calls)

Any Ideas would be much appreciated. Thank you!

4 comments

/phoenix #software-design

0 2650 4

2021-06-16 02:15:36 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

dimitarvp

That’s not very specific, can you give us a clue how did you go about it and what was not liked by the interviewers first?

Post #1

dopomecana

In my mind was something like this: Incoming API call → Processing data → Another API call from me to another service.
But I think that I made too much use of the database instead of relying on the OTP Agent or ETS.
At my current company had lawsuits in the past, not specifically my company but our clients..and we have to had anything saved in the database so I have this bad habit on relying to much on my postgres db.
Going back to the interview…The fact that I started designing with logging the event in the db and the update whenever the status of the event changed may be considered a bottleneck for a massive load? The db logging was in my on the spot plan a good way to handle recovering data in case of a system failure.
But afterwards I thought that in this case something like Kafka maybe used in order to keep the events for a short period in case we need to process past events after a network failure for example.
What do you think?

Post #2

dimitarvp

Zapier is an automation platform. I’d give you extra points for wanting to have a complete paper trail in a similar project! Not criticizing your interviewers because I don’t have all the context but with the info I have, I’d say they judged you wrongly here.

And no, don’t use too much external technology – especially one as powerful and complex to set up as Kafka – in interview assignments. Keep it simple. You did well for choosing to roll your own event queue / modification log.

Post #3

thomas.fortes

Metrics are good, but if the collection of the metrics cause excessive performance penalties it should be reworked, if everything has to hit the DB all the time, the DB will soon turn into the bottleneck of your application, there is a reason why in every other stack Redis/RabbitMQ/Kafka/Etc are used for queues, and that’s because they are optimized for it in a way that traditional DBs are not.

In the BEAM world you can get pretty far with OTP without resorting to external services, I personally would use a GenServer, and unless strictly necessary to log every single event, (think financial industries) I would aggregate and log them by batch in a reasonable interval.

Post #4

— All posts loaded —