Comparison of Decimals not logical

Wow thanks a lot you really care!! :heart_eyes:

2 Likes

@Nobbz Interesting! This would suggest that if we reorder the Struct fields in Decimal, we could make the comparison operators work:

defmodule EpicDecimal do
   defstruct "1sign": 0, "2exp": 0, "3coef": 0
end

x = %EpicDecimal{"3coef": 0}
y = %EpicDecimal{"3coef": 3}
z = %EpicDecimal{"3coef": 3, "1sign": -1}
z < x && x < y

But this is probably a horrible idea for some other reasons ^^’. (one thing I can think of is, because the sign is separate, that -0 < 0, which should be equal)

1 Like

If you are comparing Decimals to Decimals that would really work yes, but Decimal.new(1) < 0 would still return false, since integer is always greater than a map…

2 Likes

It took me a bit by surprise that Decimal.compare/2 returns Decimal<-1>, Decimal<0>, or Decimal<1>. I guess it makes sense that you want to stay within the same domain and you view compare as sgn(a - b) (Sign function). However coming from other languages one might be primed to expect integer -1, 0, or 1.

Using Decimal.cmp/2 returning :lt, :eq, or :gt may actually be clearer.

2 Likes

O_o… that is very odd indeed. I would expect any function named compare to indeed return integers, similar to for instance the UFO-operator that Rubyists might be familiar with.

I actually think that a built-in Decimal datatype would be a great addition to both Elixir and Erlang.

1 Like

The current way that > works with the Decimal Struct is just a side effect of the implementation of Maps. Below a certain number of entries the order of Map.keys is fixed, above that it can vary since the underlying implementation switches and Map.keys no longer returns a fixed order.

FWIW, this is a place where I think Structs are a suboptimal solution. With something this small a tuple would make more sense and with some care in construction would work “better” with normal Erlang/Elixir operators.

I guess the intent is to make a data type that can be used for money.

4 Likes

Having complete comparison over all terms is actually a very nice feature that allows us to implement data structures such as ordered dictionaries very easily.

We could implement such functions if we could access map keys in guards.

It wouldn’t work because of the exponent unfortunately.

Not sure I understand. Why would you expect compare/2 to return integers but not cmp/2? The reason compare/2 returns a Decimal is mostly because the library follows the IEEE754 specification.

Can you show how it would work better with tuples? One of the reasons it’s a struct is because we can implement protocols for it so it works with JSON encoding, inspecting, converting to string and so on.

4 Likes

I’m probably speaking out of turn, I was only considering the comparability aspect.

But if it was a simple tuple of say { sign, exponent, coefficient } then the Erlang comparision operators would “work” for at least positive numbers and it should be possible to design the tuple so it works for all numbers.

I’d have to dig much deeper into Decimal to know if this tradeoff makes any sense in the larger context of all the things Decimal does. A quick look suggests my assumptions about what the library actually does may be wrong.

1 Like

I did some more looking and thinking and using Tuples for Decimal to enable “<” use would only work if you always normalize the coefficient and somehow put leading zeros in front of it.

You could add a number one order of magnitude larger than the specified precision to fake this, but that would complicate all the rest of the operators.

Another approach would be to use actual binaries for the coefficient, but that also adds a lot of
complexity.

2 Likes

Aha! I had not considered that. Thank you, I’ve been wondering about the actual reasoning behind the ‘all things are comparable’ in Erlang for a very long time. :slight_smile:

:thumbsup:

Yes, you’re right. I had not considered that (1 * 10^3) < (123 * 10^1)

If you follow the IEEE specification, then it makes sense to return Decimals there. It is just that I would expect comparing functions (like compare/2 or cmp/2) to be used in checks like check if a is larger or equal to b. compare(a, b) >= 0. Of course this no different from checking if b is strictly smaller than a, but depending on the context it is often more readable to use a x >= y over a y < x or a x <= y over a y > x.

So that would be a reason for a comparing function to return integers instead of decimals or symbols. Of course, if a library were to offer functions like lte?(a, b), that is a great way too.


Oh, and let me just say that I really like the Decimal library :smiley:

1 Like

Recently I wrote simple, experimental library which makes dealing with Decimal library a bit easier. It is only a thin wrapper so the actual calculations are made by Decimal anyway.

1 Like

FWIW, I’ve come up with a basic tuple implementation for Decimal-like library that does work with “<”. It’s about as complex as I expected although the trick is to fiddle the exponent, not the mantissa or significand.

1 Like

Can someone explain the problem Decimal is meant to solve that isn’t solved by float? Or just exactly what the use case is for Decimal?

I understand that it can do things float can not, I’m just trying to grasp the itch this library scratches. I think you could tweak it such that < would work, but the data structure I have come
up with is heavily dependent on the precision requested.

Would doing that break the entire purpose of the library?

My guess is that it’s important that the data structure represent the number w/o additional context provided by Decimal in the process table. The representation I have come up with has the precision as a deductible fact, but it’s tricky to work with items that were build with different notions of precision.

1 Like

All right. This is a very important piece of knowledge and in my opinion it should be included in the Computing Science 101.

Try the following calculation in any programming language (all non-archaic languages follow the IEEE 754 floating point standard): 2.3 - 0.3. Is the answer what you would expect?

The problem with floating-point numbers is that we are trying to save an arbitrary number with a radix point in a fixed-size binary representation. If you type the float 7.1, what is stored is not 7.1. It actually is ± 7.099999999999999644728632119949907064437866210937500000000000. If you want to see it for yourself, try: :erlang.float_to_binary(7.1, decimals: 60). These inaccuracies become even more defined when dealing with large numbers, because then the amount of possible binary numbers between two whole numbers becomes a lot smaller. There even is a point where whole numbers are rounded, because they are less significant than the rounding issue.
Compare :erlang.float_to_binary(12345678987654321.0, decimals: 1) with :erlang.float_to_binary(12345678987654321.1, decimals: 1). (Actually, the rounding error is so prevalent here, you can just check IEx default output instead)

Another example: (0..10) |> Enum.reduce(fn x, acc -> acc + 0.1 end).

When we do arithmetic operations with floats, these inaccuracies amplify. This is fine for calculations where we are working with external measurements that are imprecise by definition, or difficult mathematical equations that we can only approximate. It is however a problem when we are working with a known, exact, decimal amount, such as when we are counting something. A good example: Money.

These rounding issues have made money disappear in the past. Please use decimals when dealing with money.


If you want to read more about this, there is a great explanation in the Python documentation (while the syntax is in Python, the examples are true for all other programming languages). There also is What every Computing Scientist should now about Floating Point, which is a very juicy article with a lot of details.

6 Likes

And if you can’t, you can use integers and work in pennies/cents.

I understand the issues about Float and rounding. What I want to understand is how Decimal is used to solve those issues, since changing the precision only delays the onset of when rounding errors occurs.

What exactly about Decimal makes it appropriate for use as Money? Is is the fact that the significand is base10 rather than binary? Is it that you set the precision appropriate to the scale
of money that you are dealing with? Why do you trust Decimal, but not Float used within it’s
reasonable range?

I can make Decimal work correctly with >, but if that breaks the entire purpose of Decimal, then there is no point in submitting the patch.

Actually you should work in mills and round to the nearest penny.

1 Like

As far as I can tell, Decimal does give you fixed point numbers oposed to floating point numbers.

A fixed point number will always have the same precision, regardless how often you do something with it. Lets say we have a fixed and a float of 1. The fixed has a precission of 3 digits after the point. So you can multiply the fixed by a quadrillion and add a 0.1 and you will have a quadrillion and a tenth. If you do the same with the float of 1, you will have only a quadrillion (or even less due to how floats deal with values they can represent exactly)

Also for fixed (a + b) + c == a + (b + c) does hold, while it does not for floats.

1 Like

Decimals (most implementations including the one in the Elkixir library) are created from two Bignums. A bignum is an integer of arbitrary size; internal logic ensures that when the number becomes larger than a small fixed-size ‘fixnum’ can handle, it is stored in multiple separate memory locations. Of course, using bignums is not always straightforward. In the case of JS, for instance, one would need a special library that does it. However, Erlang/Elixir has built-in support for them; all integers you create are secretly convertes to and from bignums whenever necessary.

A Decimal is just (mantissa * (10 ^ exponent)) where mantissa and exponent are both bignums.

A side note: Using an arbitrary integer subdivision for money works fine, until you want to convert from one valuta to the next.

1 Like

I think you’re correct, rounding only occurs when Decimal is converted to another type. My mistake was thinking that it was just an extended floating point representation.

I looked at mostly the README and not the source code and got side tracked by all the references to rounding errors and floating point.

1 Like