any advice if this would be sane or crazy?
That depends entirely on what using it means.
Basically an agent on client computer that it is always connected to server side command and control system. Agent accepts jobs sends back telemetry, results and alerts. Obviously if not in JS the client portion needs to be ported to whatever the language the agent is written in (I think there is at a minimum an incomplete port to Python floating around).
Good point MQTT is most likely a better option
Phoenix.Prescense or the usage of anything built on top of CRDTs is useful in a system with multiple peers or siblings where there is no single source of truth. In a system where there is a single backend that is the ruler of all data, it is completely overkill and using something like indeed the mentioned message queuing protocols is a smarter idea.