GenStateMachine terminating after upgrading to 1.8

Hi there, after upgrading the docker image from 1.7-alpine to 1.8-alpine I’ve started having errors like:

iex(web@127.0.0.1)1> [debug] Tzdata polling for update.
[info] tzdata release in place is from a file last modified Thu, 03 May 2018 23:55:14 GMT. Release file on server was last modified Mon, 31 Dec 2018 01:25:12 GMT.
[debug] Tzdata downloading new data from https://data.iana.org/time-zones/tzdata-latest.tar.gz
[debug] Tzdata data downloaded. Release version 2018i.
[info] [swarm on web@127.0.0.1] [tracker:cluster_wait] joining cluster..
[info] [swarm on web@127.0.0.1] [tracker:cluster_wait] no connected nodes, proceeding without sync
[info] Tzdata has updated the release from 2018e to 2018i
[debug] Tzdata deleting ETS table for version 2018e
[debug] Tzdata deleting ETS table file for version 2018e
[error] GenStateMachine #PID<0.1215.0> terminating
** (ErlangError) Erlang error: {:bad_action_from_state_function, {:reply, :undefined, {:error, :timeout}}}
    (stdlib) gen_statem.erl:1290: :gen_statem.parse_actions_reply/7
    (stdlib) gen_statem.erl:1206: :gen_statem.loop_event_actions_list/10
    (ssl) tls_connection.erl:133: :tls_connection.init/1
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
State: [data: [{'State', {:connection, {:state, {:static_env, :client, :gen_tcp, :tls_connection, :tcp, :tcp_closed, :tcp_error, 'data.iana.org', 443, #Port<0.72>, #Reference<0.3182053341.1157758977.143183>, #Reference<0.3182053341.1157758977.143191>, :ssl_session_cache, {:ssl_crl_cache, {{#Reference<0.3182053341.1157758977.143186>, #Reference<0.3182053341.1157758977.143187>}, []}}, {#Reference<0.3182053341.1157758977.143184>, #Reference<0.3182053341.1157758977.143185>}, {:extracted, [decoded: {{#Reference<0.3182053341.1157627905.158774>, 269714418870597844693661054334862075617, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'FR'}], [{:AttributeTypeAndValue, {2, 5, 4, 10}, {:utf8String, "Dhimyotis"}}], [{:AttributeTypeAndValue, {2, 5, 4, 11}, {:utf8String, "0002 48146308100036"}}], [{:AttributeTypeAndValue, {2, 5, 4, 3}, {:utf8String, "Certigna Root CA"}}]]}}, {<<48, 130, 6, 91, 48, 130, 4, 67, 160, 3, 2, 1, 2, 2, 17, 0, 202, 233, 27, 137, ...>>, {:OTPCertificate, {:OTPTBSCertificate, :v3, 269714418870597844693661054334862075617, {:SignatureAlgorithm, {1, 2, 840, 113549, 1, 1, 11}, :NULL}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'FR'}], [{:AttributeTypeAndValue, {2, 5, 4, 10}, {:utf8String, "Dhimyotis"}}], [{:AttributeTypeAndValue, {2, 5, 4, 11}, {:utf8String, "0002 48146308100036"}}], [{:AttributeTypeAndValue, {2, 5, 4, ...}, {:utf8String, "Certigna Root CA"}}]]}, {:Validity, {:utcTime, '131001083227Z'}, {:utcTime, '331001083227Z'}}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'FR'}], [{:AttributeTypeAndValue, {2, 5, 4, ...}, {:utf8String, "Dhimyotis"}}], [{:AttributeTypeAndValue, {2, 5, ...}, {:utf8String, ...}}], [{:AttributeTypeAndValue, {2, ...}, {...}}]]}, {:OTPSubjectPublicKeyInfo, {:PublicKeyAlgorithm, {1, 2, 840, 113549, 1, ...}, :NULL}, {:RSAPublicKey, 836713073434683715080053038330557555441140845079122187080482019747832207641593150016109833288762200431223102679878791986594522899889489651882341636633437574782853916034326904187478474959712522381042699395470100721347031881200586989949845784439668826588232815601158156178642546152730916173602748827875790981644984187175184706195923738843901798646613866970800923309125704473641506228404548339198290412098508502384742917771192460470228049939854733167069964729953589800737914298667237756123752867989182234026314659559411900912946549035808728487154206232807997471857227534964371055162955336721403506486109141118107040080952199061017003517091508099408860810586468494302879550780349248631669414797620546662519273428031514340086889010208881250204199996281843646093339244363983439913896870342109869793706754676582996002826630504502903238301313729083698962896277667790687193004528078647961973286273096115797636066383795591105393354501464564120728914167214628184749255286722105473095380338479546678807230887592681874562476432797999177668064771672650653864088792809033493484642415066518500636820520768636767119805538908024614791273844265816566415741080049066433044225833760261407521467204381357575949894699885834142802874280563352869508613274719, 65537}}, :asn1_NOVALUE, :asn1_NOVALUE, [{:Extension, {2, 5, 29, ...}, true, {:BasicConstraints, ...}}, {:Extension, {2, 5, ...}, true, [...]}, {:Extension, {2, ...}, false, ...}, {:Extension, {...}, ...}, {:Extension, ...}, {...}]}, {:SignatureAlgorithm, {1, 2, 840, 113549, 1, 1, 11}, :NULL}, <<148, 184, 158, 79, 240, 227, 149, 8, 34, 231, 205, 104, 65, 247, 28, ...>>}}}, decoded: {{#Reference<0.3182053341.1157627905.158774>, 106100277556486529736699587978573607008, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'CN'}], [{:AttributeTypeAndValue, {2, 5, 4, 10}, {:utf8String, "UniTrust"}}], [{:AttributeTypeAndValue, {2, 5, 4, 3}, {:utf8String, "UCA Extended Validation Root"}}]]}}, {<<48, 130, 5, 90, 48, 130, 3, 66, 160, 3, 2, 1, 2, 2, 16, 79, 210, 43, 143, ...>>, {:OTPCertificate, {:OTPTBSCertificate, :v3, 106100277556486529736699587978573607008, {:SignatureAlgorithm, {1, 2, 840, 113549, 1, 1, 11}, :NULL}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'CN'}], [{:AttributeTypeAndValue, {2, 5, 4, 10}, {:utf8String, "UniTrust"}}], [{:AttributeTypeAndValue, {2, 5, 4, ...}, {:utf8String, "UCA Extended Validation Root"}}]]}, {:Validity, {:utcTime, '150313000000Z'}, {:utcTime, '381231000000Z'}}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, ...}, 'CN'}], [{:AttributeTypeAndValue, {2, 5, ...}, {:utf8String, ...}}], [{:AttributeTypeAndValue, {2, ...}, {...}}]]}, {:OTPSubjectPublicKeyInfo, {:PublicKeyAlgorithm, {1, 2, 840, 113549, ...}, :NULL}, {:RSAPublicKey, 689603717979852636831081426524261160465705539154429034072886964586226830541206711367761253770112725458465044497361739659482435934368052465546153546798794021727234724250491430601994902725573258440903769572247434719710136763019389835974660379560646189661431052777862620577580627735499519427983731365873646101116220703443112119094192330629089538925644308340292126955780669276375525797477380635249258541277353787220035677250408659616695990328739419944306797836041926101961617154636604738870383637100044978431121318943912528365357866046388476749437076957867974020718613998841641003305315809368105957269212277478816744375233680463674812053558530520195020911324127390436183137612053158831455696931169855471108960460199610930021503675868841756093916690737925568854144895439113023618139270733541586805299082563244530623575372402602522085830629158312024768857183218701826386841425816972120556588392270337545276474712543306659887836405319075385033422367187135702233624763183702955559196400695312060899684142798408078723091214153237781607055313585627048184929762700635100359534649156945631489915475941635189136663455081054904713071171644045263631869342706234148072080520374089220391876551645591673277270360119934713372188503089344323356273673087, 65537}}, :asn1_NOVALUE, :asn1_NOVALUE, [{:Extension, {2, 5, ...}, false, <<...>>}, {:Extension, {2, ...}, true, ...}, {:Extension, {...}, ...}]}, {:SignatureAlgorithm, {1, 2, 840, 113549, 1, 1, 11}, :NULL}, <<54, 141, 151, 204, 66, 21, 100, 41, 55, 155, 38, 44, 214, 251, ...>>}}}, decoded: {{#Reference<0.3182053341.1157627905.158774>, 124779693093741543919145257850076631279, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'CN'}], [{:AttributeTypeAndValue, {2, 5, 4, 10}, {:utf8String, "UniTrust"}}], [{:AttributeTypeAndValue, {2, 5, 4, 3}, {:utf8String, "UCA Global G2 Root"}}]]}}, {<<48, 130, 5, 70, 48, 130, 3, 46, 160, 3, 2, 1, 2, 2, 16, 93, 223, 177, ...>>, {:OTPCertificate, {:OTPTBSCertificate, :v3, 124779693093741543919145257850076631279, {:SignatureAlgorithm, {1, 2, 840, 113549, 1, 1, 11}, :NULL}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, 4, 6}, 'CN'}], [{:AttributeTypeAndValue, {2, 5, 4, ...}, {:utf8String, "UniTrust"}}], [{:AttributeTypeAndValue, {2, 5, ...}, {:utf8String, ...}}]]}, {:Validity, {:utcTime, '160311000000Z'}, {:utcTime, '401231000000Z'}}, {:rdnSequence, [[{:AttributeTypeAndValue, {2, 5, ...}, 'CN'}], [{:AttributeTypeAndValue, {2, ...}, {...}}], [{:AttributeTypeAndValue, {...}, ...}]]}, {:OTPSubjectPublicKeyInfo, {:PublicKeyAlgorithm, {1, 2, 840, ...}, :NULL}, {:RSAPublicKey, 807357890495810441736093858448857437005503159464873532889665061149450049267517568326721240183937035290246259362027107260353517758367028415845334110414958031516623538551857389070581663469088666049393971117960149876075288689766187835155300506484080398181950755350958981821837155690536732783500797879256336226989139781867345426694117084418485476628396578713470902249528774086342778340937739102189625649632153199712773655761339619448871167814180865130901759199267818785696680498600165221744076301240144746595359653009786074739865727340157151621935422271237645861554 (truncated)

moving back to 1.7 fixes the problem, however I don’t have this problem locally with 1.8. Any idea? If I try to download the tzdata file with curl everything is working fine and it’s also saying that it has been actually downloaded already.

1 Like

This sounds like Workaround for Docker with Erlang/OTP 21.3

3 Likes

Thanks a lot, that’s it, I’ve tried to google it but haven’t found those links!

2 Likes

I’m glad it helped!

1 Like

Hi, I am having the same problems but this seemingly has nothing to do with Docker. This happened after upgrading from Elixir 1.7 to 1.8 a few days ago.

Apr 04 10:41:50 yorck-signage-1 vista_api[24692]: 10:41:50.240 [error] GenStateMachine #PID<0.1895.0> terminating
Apr 04 10:41:50 yorck-signage-1 vista_api[24692]: ** (exit) {:shutdown, :sender_died, :killed}
Apr 04 10:41:50 yorck-signage-1 vista_api[24692]:     (stdlib) gen_statem.erl:1158: :gen_statem.loop_event_result/9
Apr 04 10:41:50 yorck-signage-1 vista_api[24692]:     (ssl) tls_connection.erl:133: :tls_connection.init/1
Apr 04 10:41:50 yorck-signage-1 vista_api[24692]:     (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Apr 04 10:41:55 yorck-signage-1 systemd[1]: Stopped Digital Signage Displays.
Apr 04 10:42:00 yorck-signage-1 systemd[1]: Started Digital Signage Displays.

Right now, the only solution for me is going back to Elixir 1.7. The solution in Workaround for Docker with Erlang/OTP 21.3 is also just using a Docker image with Elixir 1.7 (istead of :latest).

This also has apparently nothing to so with Elixir but with Erlang. This happens occasionally when performing HTTP requests over TLS (some requests to external APIs). I am using hackney as client.

Interestingly, :httpc seems to work for me (but I cannot use it since it doesn’t play well with {} characters in the URL which is needed to talk to some Microsoft Graph API :angry:)

In what ways is it not working? According to RFC 1738 the curly brace characters should be encoded as %7B and %7D, which you can urldecode as normal. What issue were you having? Is :httpc not encoding them properly? It might not supposed to encode them in the path first and you are probably supposed to do it yourself as is common with a lot of such simple API’s, but if it is supposed to encode it for you and you find the docs about that then it is absolutely a bug and should be reported to OTP.

Either way yeah, OTP 21.3 I’ve heard is slightly bugged on TLS, you can always use another docker image for Erlang 1.8 on OTP 21.2. :slight_smile:

:httpc is working perfectly fine.

But it REQUIRES me to encode curly brackets to %7B and %7D as required in the RFC. So far so good.

But unfortunately Microsoft’s Graph API requires plain curly brackets in the query string (GET-parameter) if you want to add filters. This is against the RFC but somehow Microsoft didn’t respond to my bug report and I still need this query to work.

On the docker image. Ah, I see. It’s the OTP version, not Elixir causing problems.

Yeah that absolutely goes against the spec. I’m not sure what conforming HTTP clients would allow you to submit such URL’s? o.O

Hackney does. One might argue if “SHOULD encode” implies that one can omit it. I didn’r read the RFC tohugh.

conforming HTTP clients

“Conforming” ^.^

OK, the RFC states “these must be encoded”. That’s pretty clear then. Well well.

1 Like