Potential Elixir/Erlang bug

I have a bug that started randomly failing our tests on Heroku.
In a very particular scenario once every couple hundred of runs Regex.split(~r/\s+/, "10 EUR") returns a one-element list ["10.00 EUR"] instead of two-element list ["10.00", "EUR"]. If I call it directly, it always works. However, in this very particular setup presented in this repo, it reliably fails every now and then.

I can see the behaviour both locally and on Heroku so I am starting to believe I’ve detected a bug in Elixir.

If anyone is interested, here is a repo with a minimal test setup.
Let me know if you can reproduce it with instructions from the repo and if you have any idea about how to fix it.

4 Likes

I could reproduce it using your test repo with erlang 23.3 and elixir 1.10.4-otp-23,

[..]
STR: "10.00 EUR"
LST: ["10.00", "EUR"]
VALUE: "10.00"
STR: "10.00 EUR"
LST: ["10.00 EUR"]
VALUE: ""

  1) test this test fails every couple of runs (BugTest)
     test/bug_test.exs:5
     ** (MatchError) no match of right hand side value: :error
     code: for x <- 1..1000 do
     stacktrace:
       test/bug_test.exs:7: anonymous fn/2 in BugTest."test this test fails every couple of runs"/1
       (elixir 1.10.4) lib/enum.ex:3371: Enum.reduce_range_inc/4
       test/bug_test.exs:6: (test)
.
Finished in 0.05 seconds
1 doctest, 1 test, 1 failure

I have no idea of the cause, but out of curiosity I added a

IO.puts(x) to the test, to check when the error occurred

  test "this test fails every couple of runs" do
    for x <- 1..1000 do
      IO.puts(x)
      {:ok, m} = Money.cast(%{amount: "10.00", currency: "EUR"})
    end
  end

and the error disappeared. :flushed: (are we really living in a simulation then? :smirk:)

1 Like

I couldn’t reproduce either with your repo or a simple benchee script:

Benchee.run(
  %{
    "Regex.split/2" =>
      fn ->
        [_a, _b] = Regex.split(~r/\s+/, "10.00 EUR")
      end,
    },
  time: 10,
  memory_time: 2
)

Which ran for over a million iterations

I’m on:

Erlang/OTP 24 [erts-12.0.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit]

Interactive Elixir (1.12.1) - press Ctrl+C to exit (type h() ENTER for help)
1 Like

Yes, I questioned my sanity too :smiley:

1 Like

Just tried erlang 24.0.2 and elixir 1.12.1 and the bug disappeared. I’ll try to pinpoint which upgrade fixes the issue.

1 Like

I can reproduce on some combinations of elixir/erlang…

never on otp 24
never on elixir 1.11 and above (on otp 23 or 24)

seems specific to the otp-23 builds eg elixir 1.10.4-otp-23 will fail - but 1.10.4-otp-22 will not (both against otp 23.3.4.3)

3 Likes

Perhaps can be traced by:

OTP 24

iex> :re.version
"8.44 2020-02-12"

OTP 23

iex> :re.version
"8.43 2019-02-23"

OTP 22

1> re:version().
<<"8.43 2019-02-23">>

OTP 21

1> re:version().
<<"8.42 2018-03-20">>
3 Likes