Getting PubSub Error on New server, ETS

Phoenix Forum Questions / Help
,
1

We have been running our evercam server, on Ubuntu 14.04 for past few years, and Its been going good, we thought to switch between Server, So we bought a new one with Ubuntu 17.03,

Erlang= 19.3
Elixir 1.5.2

All of a sudden on new server, we have started getting such errors as

2017-11-09 06:18:15.603 [error]     (stdlib) :ets.lookup_element(EvercamMedia.PubSub.Local0,
"cameras:stevebuckscam01", 2)    (phoenix_pubsub) lib/phoenix/pubsub/local.ex:153: Phoenix.PubSub.Local.subscribers_with_fastlanes/3    (phoenix_pubsub)
lib/phoenix/pubsub/local.ex:121: Phoenix.PubSub.Local.do_broadcast/6    (phoenix_pubsub)
lib/phoenix/pubsub/local.ex:96: Phoenix.PubSub.Local.broadcast/6    (elixir) lib/enum.ex:675:
Enum."-each/2-lists^foreach/1-0-"/2    (elixir) lib/enum.ex:675: Enum.each/2    (phoenix_pubsub)
lib/phoenix/pubsub/pg2_server.ex:30: Phoenix.PubSub.PG2Server.do_broadcast/7  
(evercam_media) lib/evercam_media/snapshot/db_handler.ex:35:
EvercamMedia.Snapshot.DBHandler.handle_info/2

Everything was running quite good on old server, Can anyone point where there is an error? or what is wrong? May anything related to Ubuntu version?

2 Likes
2

By PubSub I’m guessing that is Phoenix’s PubSub in the ets lookup? One of the phoenix people maybe can see what is up? I’ve not seen this… :-/

1 Like
3

Yes. it is,

1 Like
4

What Phoenix.PubSub version?

1 Like
5

"phoenix_pubsub": {:hex, :phoenix_pubsub, "1.0.2"

It was/is working normal on old machine…

But on new machine its making problem.

1 Like
6

Ah, what version of Erlang/OTP and Elixir was the old and new machines both?

1 Like
7

Can you also include the entire stack trace? Does it happen on startup, periodically, or under certain conditions? It’s possible you are racing the PubSub server startup that would be caused by something starting in your supervision tree before the PubSub server. A fix for that would be to add your endpoint/pubsub supervisor above your DBHandler process in your supervision tree.

2 Likes
8

Both machines are on the same version.

Erlang 19.3
Elixir 1.5.2

1 Like
9

Actually, This is the full stack trace, we got in logs,

On Startup: No
Periodically Yes, and On Certain conditions yes,

For example when we try to call this end point https://github.com/evercam/evercam-server/blob/master/lib/evercam_media_web/router.ex#L69

Errors start coming, many in numbers, But not for the camera, we get, but for some other cameras.

I think, endpoint supervisor is, above of everything. You can see it here,

Out of Curiosity, this (stdlib) :ets.lookup_element(EvercamMedia.PubSub.Local0, "cameras:stevebuckscam01", 2)

0 with the Local, is that could be a problem? (Even I dont know from where the Zero is coming and why.)

1 Like
10

The 0 is the name of the shard, which is fine. The confusing thing is this function in the code is wrapped n a try/catch from the argument error that ets would potentially throw from the lookup function, but your logs contain a half stack trace here without the actual error?

2 Likes
11

So peeking at the code, your DBHandler has several try/catch/rescue blocks that call out to a Util.error_handler function which Logger.error’s the stacktrace – which is the source of the confusing stacktfrace. I suggest you start there by unwrapping these try/catches to figure out the root cause of the exceptions. My hunch is the error lies in your code, not Phoenix.PubSub but your error wrapper/logger is masking it.

5 Likes
12

Thanks a lot @chrismccord,

we solved the issue by debugging it from the start, and Also removing most of the try and catch.

Thanks for your support on this.

13

I also faced some different issues in Phoenix.PubSub and I found some coding mistake in my code after reading your post. :blush: