zac

Fighting `Internal Server Error` on machine suspend/resume cycle

Has anyone built a Phoenix / Ash / LiveView app that supports server suspension, without getting a nasty user experience in the process?

Every time our server suspends, whatever view I’m on turns into a big white page with:

Internal Server Error

Hitting refresh brings it back to life, sometimes you have to refresh twice, but it more-or-less picks up at a reasonable spot (not necessarily exactly where you were, but close enough).

The problem, of course, is the unpleasant “Internal Server Error.” I want that to go away.

I’m not even sure it’s possible, really – in which case, we leave a machine running all the time. But hoping someone has some experience on the topic.

Here’s a look at the logs as the app goes through a suspend/resume cycle. I’m not running debug level yet, so not a lot of info here:

2025-09-23T07:39:00Z app[2860103b762e38] iad [info]07:39:00.761 [error] Postgrex.Protocol (#PID<0.4659.0>) disconnected: ** (DBConnection.ConnectionError) tcp recv (idle): closed
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]07:39:00.765 request_id=GGfKbJC6d6JrX-QAAAjR [error] ** (DBConnection.ConnectionError) tcp recv (idle): closed
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ecto_sql 3.13.2) lib/ecto/adapters/sql.ex:1098: Ecto.Adapters.SQL.raise_sql_call_error/1
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ecto_sql 3.13.2) lib/ecto/adapters/sql.ex:996: Ecto.Adapters.SQL.execute/6
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ecto 3.13.3) lib/ecto/repo/queryable.ex:241: Ecto.Repo.Queryable.execute/4
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ecto 3.13.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ecto 3.13.3) lib/ecto/repo/queryable.ex:145: Ecto.Repo.Queryable.exists?/3
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ash_sql 0.2.93) lib/aggregate_query.ex:119: anonymous fn/5 in AshSql.AggregateQuery.add_single_aggs/5
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (elixir 1.18.4) lib/enum.ex:2546: Enum."-reduce/3-lists^foldl/2-0-"/3
2025-09-23T07:39:00Z app[2860103b762e38] iad [info]    (ash_sql 0.2.93) lib/aggregate_query.ex:81: AshSql.AggregateQuery.run_aggregate_query/4
2025-09-23T07:39:01Z app[2860103b762e38] iad [info]07:39:01.609 [error] Postgrex.Protocol (#PID<0.4660.0>) disconnected: ** (DBConnection.ConnectionError) tcp recv (idle): closed
2025-09-23T07:44:54Z proxy[2860103b762e38] iad [info]App waste-walk has excess capacity, autosuspending machine 2860103b762e38. 0 out of 1 machines left running (region=iad, process group=app)
2025-09-23T07:44:56Z app[2860103b762e38] iad [info]Virtual machine has been suspended
2025-09-23T07:58:46Z proxy[2860103b762e38] iad [info]Starting machine
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.782463043 [01K5SPF8VBK4G6FEJTKMM9PZ4V:main] Running Firecracker v1.12.1
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.782590762 [01K5SPF8VBK4G6FEJTKMM9PZ4V:main] Listening on API socket ("/fc.sock").
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.782800962 [01K5SPF8VBK4G6FEJTKMM9PZ4V:fc_api] API server started.
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.784474288 [01K5SPF8VBK4G6FEJTKMM9PZ4V:fc_api] The API server received a Get request on "/".
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.784495308 [01K5SPF8VBK4G6FEJTKMM9PZ4V:fc_api] The request was executed successfully. Status code: 200 OK.
2025-09-23T07:58:46Z app[2860103b762e38] iad [info]2025-09-23T07:58:46.784820917 [01K5SPF8VBK4G6FEJTKMM9PZ4V:fc_api] The API server received a Put request on "/logger" with body "{\"log_path\":\"logs.fifo\",\"level\":\"info\"}".
2025-09-23T07:58:47Z runner[2860103b762e38] iad [info]Machine started in 343ms
2025-09-23T07:58:47Z proxy[2860103b762e38] iad [info]machine started in 352.485279ms
2025-09-23T07:58:47Z proxy[2860103b762e38] iad [info]machine became reachable in 9.492925ms
2025-09-23T07:58:47Z app[2860103b762e38] iad [info]07:58:47.099 request_id=GGfZp_tRZwZy9CcAAAFC [info] GET /sign-in

6 comments

/phoenix #liveview #flyio

0 240 6

2025-09-24 05:52:37 UTC

Most Liked

LostKobrakai

There are heartbeats on the socket. They’re needed to detect connection issues where the connection thinks it’s fine, but no data makes it through.

Post #4

rhcarvalho

I have a feeling LiveView + auto-suspending the server will bring you a lot of headaches, probably not a good fit.

One of the hardest parts of working with LiveView is managing what happens on disconnection and reconnection, how the UI is affected, ensuring a seamless user experience.

Intentionally disconnecting the server will lead to lots of such reconnection states, which either means you’re going to learn all the tricks to make it work great, or users will have a bad experience with unexpected page behavior.

The “live” in LiveView is the assumption that you want a realtime connection between server and client, such that either server and client can send events to each other anytime. If there’s no connection between them, then the server cannot initiate a reconnection on its own.

Post #5

Last Post!

rhcarvalho

Hey @zac

If at some point in time you go from N servers down to 1 (or really just any of the servers suspend), then a fraction of your connected users will be impacted.

I understand you’re only considering this idling behavior as a cost saving mechanism? If that’s the case, consider whether a VPS on Hetzner, Digital Ocean, etc would serve your needs at a lower price point. I know choosing a vendor here is not trivial, and will affect deployments, existing contracts and business relationships, …

Yeah, I would be curious to hear what their support would suggest. AFAIK they are users of Elixir and Phoenix LiveView, so there must be in-house expertise to orient you.

Post #7