Send data to a master server or poll each child server?

I have a master server where a website is running and child ones where some calculations take place. Something is calculated on a child server and it should be displayed on the master server on the website. It occurs every about 10-30 minutes. It’s not deterministic.

My question is, what’s better: to require each child server send data to the master server or require the master server poll each child server? What are cons and procs of these approaches?
Don’t tell me “it depends on your requirements”. If you do, then tell me first what requirements would affect your decision if you were in my shoes?

Is the terminology correct? Do you have a master/child server or are
they just stand-alone servers like a web-front and a calculation server?

First question you should answer is who initiates the calculation. If it
is the master server I would have the master server start the
calculation on the child server and then wait for the response.

If the child servers are continously running and doing reports it sounds
like a publish subscribe pattern and I would have the child servers
publish to however is currently subscribing (your master server).

Generally polling is easier to setup but not as reactive and involves
lots of unnecessary polls when nothing has happened.

pushing data is more reactive and only need to communicate when
something has changed.

From a robustness point of view, polling is also easier. If the polling
server crashes and comes up it just starts polling again. If the server
being polled crashes, the polling server handles the errors and start
receiving info then the other server is up again.

For the pub/sub solution you need to make sure the subscribers get
notified when the published goes down so that they can re-subscribe.

Cheers

1 Like

I suppose the 2nd option.

The child servers.

Isn’t the same true in case of pushing?

Do you mean “push”? Which is the opposite of “poll”. That is, a child server would be an initiator and push data to a master server.

The child servers.

In this case I would have the master pull the data from the child
servers. If one of the computations server goes down you can display an error
message.

Isn’t the same true in case of pushing?
If you do simplistic pushing. I.e your child servers blindly push data
to the master and if either go down they just keep pushing. I don’t
think this make sense though. The master server must know if a
computation server has gone down so both servers must do error handling.
In the poll scenario only the one polling need to handle errors as
the other server simple responds to requests.

1 Like
  1. Then a disadvantage will be is that data on a server will a bit outdated for N minutes because a server will poll data once per N minutes. Although, it’s not too bad.

  2. On child servers data will be stored in Sqlite database. How will a master server access it? I might not want to create a web/rest interface on each child server. Then how else? Somehow via ssh?

  1. Then a disadvantage will be is that data on a server will a bit outdated for
    N minutes because a server will poll data once per N minutes. Although, it’s
    not too bad.

Yes, if that is not acceptable you need to do pub/sub

  1. On child servers data will be stored in Sqlite database. How will a master
    server access it? I might not want to create a web/rest interface on each child
    server.

I just assumed you had some sort of application server running on the
machines. If elixir you have the option of distributed erlang, which
would be the quickest to implement.

I sort of had the feeling you wanted to do heavy computation on child
nodes and that the master would distribute these jobs and collect the
results. This is quite a common setup and one where elixir and erlang do
very well.

Then how else? Somehow via ssh?

But perhaps you are after something much simpler.

If you want a hacky quick way to just get data over then perhaps you can just run the
computation on the child servers and rsync over the results into a
folder, then setup inotify on changes in your master and bob’s your
uncle :slight_smile:

Right, the calculations won’t be heavy.