CharlesO

CharlesO

Rounding Error with Decimal Sum

Hi.

Please I’m getting this unusual rounding error when summing numbers using the Decimal Library

My result is off by 0.01

  208,505,442,949.08 # Excel
  208,505,442,949.09 # sum function with Decimal

Any suggestions please?

Thanks.

def sum() do
    f = "C:/Downloads/MMR100.txt"
    {:ok, b} = :file.read_file(f)
    a = String.split(b,"\n")
    c = Enum.count(a)
    IO.puts c # 299,725

    _sum(a, "0.00") |> Decimal.to_string(:normal)

    # 208,505,442,949.08 - Excel
    # 208,505,442,949.09 - sum
  end

  defp _sum([], acc), do: acc
  defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
  defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))

Marked As Solved

NobbZ

NobbZ

Parsing and summing the list as float gives: 208505442949.07806, rounding this to 2 decimal places returns 208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…

PS: I’ve heard there are still people using Excel to manage their money…

PPS: my iex session:

iex(3)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum
208505442949.07806
iex(4)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum |> Float.round(2)
208505442949.08

Also Liked

NobbZ

NobbZ

LostKobrakai

LostKobrakai

This exactly is the reason, why you’d even need a library like Decimal. Because floats are not suited for calculating correctly.

NobbZ

NobbZ

No, I do get 9:

iex()> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.graphemes/1) |> Enum.map(&List.last/1) |> Enum.map(&String.to_integer/1) |> Enum.sum
1349409

Where Next?

Popular in Questions Top

greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
nobody
How to bind a phoenix app to a specific ip address? could not find anything about that, nowhere, unfortunately, but for me this is quite...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

Other popular topics Top

malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

We're in Beta

About us Mission Statement