Github CI down? Is it working for you?

Chat / Discussions
1

My Github CI has been down since yesterday morning. Any else experiencing this issue?

Error: 14:04:41.018 [error] Failed to connect to ‘raw.githubusercontent.com’ to download locale :“en-IE”. Reason: {:tls_alert, {:unexpected_message, ‘TLS client: In state hello_middlebox_assert at ssl_gen_statem.erl:736 generated CLIENT ALERT: Fatal - Unexpected Message\n {unexpected_msg,{internal,{encrypted_extensions,#{sni => {sni,}}}}}’}}

People on Twitter also seem to be complaining.

https://twitter.com/Peter_Logo_/status/1560302390246522880

2 Likes
2

They had a pretty big hickup and even though they claim it’s resolved, various people on HN were saying they still had problems: GitHub Status - Incident with Actions

1 Like
3

It seems to be an issue with Erlang version, I’ve upgraded to OTP 25 and error gone.

ref: github actions SSL error · Issue #184 · elixir-cldr/cldr · GitHub

4

We had to make this change to unclog the CI on Github. Hope this helps people.

--- otp: ['24.3.4']
+++ otp: ['25.0.4']

Still a very weird fix – what if you are on a very legacy setup and cannot freely update such a integral part of your stack to a next major revision? I shudder to think. :scream:

5

Currently in this boat. Can’t just update OTP immediately. It seems to work okay on my up-to-date local Arch environment, but fails in our Ubuntu 22.04 production container (which is also up to date). Also, curl and wget inside the container can fetch the file just fine.

Interested in any other workarounds folks may find.

6

Please anyone has some solution for this?

Migration to OTP 25 isn’t on the table for us due to legacy code.

This is super weird issue.

We locally build with mix release and Dockerfile (the target OS is different from dev OS) and it fails on mix assets.deploy with the dependency of dart-sass failing to download itself from GitHub.

However, running mix assets.deploy on dev OS there’s no issue with CA certs. (dev OS has same elixir/OTP setup).

7

Weird error, I’m getting exactly the same thing running the official Docker image elixir:1.13.4 only in Github CI. Not related to Dart but to downloading Tailwind.

Anyone upgraded to the 1.14rc image? Does that fix it?

EDIT: Looks like it will because it’s on OTP 25 - still rc code :grimacing:

8

There’s official docker image for elixir:1.13.4 based on OTP 25 and you might get away with that?

I found out that OTP 25 added two new methods to get OS provided CA-certs.

When I get on my laptop again I will simply disable HTTPS check in dart-sass library until it gets fixed in its castore dependency - if I understand this issue correctly…

EDIT: either upgrade to OTP 25 or wait for new release of castore if your code (and its dependencies) depend on this CA library.

1 Like
9

Oh, nice spotting. I missed it in the list of images on Docker Hub. Thanks!

10

If anyone else is running into some odd errors related to this, these are the steps I had to go through to get back up and running:

  1. Update erlang to 25.0.4
  2. Update elixir to 1.13.4-otp-25
  3. Update tailwind mix package to the latest (updates the castore dep it has, as well)
  4. Update alpine image I’m using to 3.16 (was on 3.15)

Without all of those, I was getting similar errors to this thread, or erts errors.

1 Like
11

Did anyone identify the exact root cause of this problem? (i.e. beyond upgrading to OTP25 magically fixes it)

12

There’s some discussion here as well:

1 Like
13

For at least ex_cldr the issue was resolved by disabling TLS 1.3 on OTP versions less than 25. As always a warm thank you to @voltone for the pointer in the right direction.

2 Likes
14

DartSass v.05.1 has the fix– thanks for the report and thanks @kip & @voltone for the fix :slight_smile:

2 Likes
15

Hi. I’m seeing this error,

{:tls_alert, {:unexpected_message, 'TLS client: In state hello_middlebox_assert at ssl_gen_statem.erl:736 generated CLIENT ALERT: Fatal - Unexpected Message\n {unexpected_msg,{internal,{encrypted_extensions,\#{}}}}'}}

Not sure of the source, but I believe it’s from Tesla via Hackney …

@cli Tesla.client([], {Tesla.Adapter.Hackney, [recv_timeout: 60_000, ssl_options: [verify: :verify_none]]})

Please what may we do to fix?

This started after updating to otp_win64_24.3.4.4

16

Upgrade to OTP 25 or change those settings to:

@cli Tesla.client([], {Tesla.Adapter.Hackney, [recv_timeout: 60_000, ssl_options: [versions: [:"tlsv1.2"], verify: :verify_none]]})

2 Likes
17

This works. Thank you :+1:

18

For the curious, apparently this is due to a bug on OTP-24.3.4.3

2 Likes
19

Getting this on cli too on Fedora 36:

localhost ~/IdeaProjects/sentrypeer_theme $ mix tailwind.install

22:41:58.962 [debug] Downloading tailwind from https://github.com/tailwindlabs/tailwindcss/releases/download/v3.1.8/tailwindcss-linux-x64

22:41:59.510 [notice] TLS :client: In state :hello_middlebox_assert at ssl_gen_statem.erl:736 generated CLIENT ALERT: Fatal - Unexpected Message
 - {:unexpected_msg, {:internal, {:encrypted_extensions, %{sni: {:sni, []}}}}}
** (RuntimeError) couldn't fetch https://github.com/tailwindlabs/tailwindcss/releases/download/v3.1.8/tailwindcss-linux-x64: {:error, {:failed_connect, [{:to_address, {'objects.githubusercontent.com', 443}}, {:inet, [:inet], {:tls_alert, {:unexpected_message, 'TLS client: In state hello_middlebox_assert at ssl_gen_statem.erl:736 generated CLIENT ALERT: Fatal - Unexpected Message\n {unexpected_msg,{internal,{encrypted_extensions,\#{sni => {sni,[]}}}}}'}}}]}}
    lib/tailwind.ex:328: Tailwind.fetch_body!/1
    lib/tailwind.ex:230: Tailwind.install/0
    (mix 1.13.4) lib/mix/task.ex:397: anonymous fn/3 in Mix.Task.run_task/3
    (mix 1.13.4) lib/mix/cli.ex:84: Mix.CLI.run_task/2
20

Update OTP to the latest patch and it should work again afaik.

1 Like