Double equals roughness precision

Me being a curious monkey with iex.

That’s pretty impressive roughness precision, 16 digits

iex(52)> 4 == 3.9999999999999998
iex(53)> 4 == 3.999999999999999
iex(1)> 4==3.999999999999999999999999999999999999999999999999 
iex(2)> 4===3.999999999999999999999999999999999999999999999999

It looks strange anyway :slight_smile:

ahah that is interesting!

My guess, we overflow the mantissa at that point and hit a random bit?

=== performs strict equality, where float will never be equal to integer.

BTW It’s fine to be a curious monkey, I am one too :slight_smile:

yeah aware of the triple strictness.

Some languages will treat 3.999 as flat out 4 and some won’t. I was looking to see how elixir would handle integer vs float rough comparison and then hmmm how precisely do you go before saying enough is enough.

curious monkey is a good monkey :smiley:


iex(1)> 3.9999999999999998

well yes same goes for the core, not just in double equals

iex(1)> 3.9999999999999998
iex(2)> 3.999999999999999

But this is where things get a bit more interesting :slight_smile:

iex(1)> 3.9999999999999998
iex(2)> 3.9999999999999997
iex(3)> 3.9999999999999996
iex(4)> 3.9999999999999995
iex(5)> 3.9999999999999994
iex(6)> 3.9999999999999993
iex(7)> 3.9999999999999992
iex(8)> 3.9999999999999991
iex(9)> 3.9999999999999990
iex(10)> 3.9999999999999999

0 - 3 => none
4 - 7 => 6
7 - 9 => round up


a =
b = Decimal.from_float(3.9999999999999998)

Decimal.cmp(a, b)

Returns :eq, e.g. it deems those to be equal.

Granted that’s not a language feature per se but it’s the officially sanctioned way of doing things in this area regardless.

These are all just consequences of the underlying floating point, which can’t store arbitrarily precise decimals. This isn’t really about ==. == definitely checks for equality if you’re comparing floats, it doesn’t round. However, typing a decimal number in your code may not result in a float that is equal to the number you typed.


it’s about ULPs! Unit in the last place - Wikipedia


that’s pretty cool, use all your life and never know it existed situation. thank you!

This is super easy to explain:

  1. Floating points are still discrete, so it cannot present all values, only certain stops. You can think of each floating point as a “range” between nearest 2 discrete values (like uncertainty in physics).
  2. 3.9999999999999998 and 4.0 have the same representation in binary64 IEEE format:
    iex(1)> <<3.9999999999999998::float-64>>
    <<64, 16, 0, 0, 0, 0, 0, 0>>
    iex(2)> <<4.0::float-64>>
    <<64, 16, 0, 0, 0, 0, 0, 0>>
  3. == is coercing comparison, in contrast to JS, it will coerce only numeric (integers and floating point) values, so in this case both values are coerced to floating point and compared, and as both have the same representation, these are the same.

that is awesome deep dive, thanks!

To provide even more insight, let’s look into what values we can encode:

# Decode 4.0 into its components according to IEEE 754
<<sign::1, exp::11, frac::52>> = <<4.0::float-64>>

# Encode next smaller possible value
<<f::float-64>> = <<sign::1, (exp - 1)::11, (frac - 1)::52>>
f #=> 3.9999999999999996

# And one more
<<f::float-64>> = <<sign::1, (exp - 1)::11, (frac - 2)::52>>
f #=> 3.9999999999999990

So this mean we can encode only 4.0, 3.9999999999999996 and 3.9999999999999990, nothing between these two (at least in 64 bits). That mean that Elixir cannot precisely encode 3.9999999999999998, as there is no bit pattern that will match that value, and it need to round it to some other value. As you can see, it will round it to the nearest representable value, which is 4.0. So that obviously mean that when you try to compare it with value that is (almost) representable, then it will tell you that these values differ.


Well, you end with Decimal.from_float(4.0) as well there, as AST do not store floats with arbitrary precision. Decimal.from_float/1 is function, not macro, so even if Elixir would provide that information in AST, it would not be available to that function at runtime. If you would use Decimal.parse("3.9999999999999998") then it would tell you that there is difference between these two.

iex> {d1, ""} = Decimal.parse("3.9999999999999998")
{#Decimal<3.9999999999999998>, ""}
iex> d2 =
iex>, d2)

so how would we represent and operate on 3.9999999999999997, is this used for quick comparison?

It would be probably encoded as 3.9999999999999996, but there is no guarantee for that AFAIK. And if you want to check equality for 2 floats, then you should use:

abs(a - b) < 1.0e-15

As IEEE binary64 is more or less precise up to 15 significant digits, everything else can (and should) be treated as noise.


Thank you, it has been pretty informative deep dive on a matter I rarely - never - touch

True, I should have used Decimal.parse here, my bad.