henriquesati

henriquesati

Can I trust task and spawn to finalize instructions without being supervised?


this is my current project structure, the get request has a field that needs external validation
the idea is the endpint dispatch the validation async to a task or spawn (for what I researched, if I dont need a explicity return, spawn its more lightweith and swits me more) and then retrieve the result with a get request via another endpoint, and leave tha
the task to registry the result on the ets and the db, the possible states for the value
would be
value = processing
value = acceptd (waiting for payment)
value = denied ({error, reason}
can I trust the task to perform all its duties without being supervised?
the info registered on db will have a timestamp of 2 or 3 mins, after this everything will be dropped, so it the get request doesnt find the value on the ets for any reason (if too many request hits the post endpoint, the value may be dropped from the ets, I will yet think on the cache logic) neither on the db, it can be assumed that the response was {:error, reason}, but fot that I need to trust that everything delegated to the the asyncs will be executed
also, do I need to protect my system against ets random crashs? and if yes, besides using supervisor to restart it there is a way to retrieve the data?
fyi, the application will be running on cloud with 2cpu and 3gb ram

First Post!

dimitarvp

dimitarvp

Short answer: not really, unless you take special care, which would mean (1) Oban jobs – possibly chunked – and (2) state inside the DB with explicit information about how far you got and what’s the current status of each step.

Nope. If ETS randomly loses your data then that’s a all-hands-on-deck OTP bug. Nothing you could. Don’t even think of protecting against that.


Can’t say I completely grasped your problem but IMO you should take special care to break down these separate jobs as a start.

Most Liked

kip

kip

ex_cldr Core Team

As @dimitarvp says, if ETS itself crashes, that’s an OTP bug. However ETS tables are owned by a process and the tables disappear when the process exits (for whatever reason). So you may benefit from some additional resilience around the owner process. For that, ETS provides an heir option to :ets.new/2. If the owning process dies, then the heir process will take ownership.

The eternal library helps keep ETS table alive even when the owner process dies using the heir mechanism.

Even so, if the owner and the heir keep crashing you can still create a situation when you lose the ETS data. So if its mission critical, serialising the ETS data periodically (at some appropriate “save point”) may still make sense.

Where Next?

Popular in Questions Top

ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement