CharlesO

CharlesO

Hi.

Please I’m getting this unusual rounding error when summing numbers using the Decimal Library

My result is off by 0.01

  208,505,442,949.08 # Excel
  208,505,442,949.09 # sum function with Decimal

Any suggestions please?

Thanks.

def sum() do
    f = "C:/Downloads/MMR100.txt"
    {:ok, b} = :file.read_file(f)
    a = String.split(b,"\n")
    c = Enum.count(a)
    IO.puts c # 299,725

    _sum(a, "0.00") |> Decimal.to_string(:normal)

    # 208,505,442,949.08 - Excel
    # 208,505,442,949.09 - sum
  end

  defp _sum([], acc), do: acc
  defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
  defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))

Showing Posts 1 to 10

LostKobrakai

LostKobrakai

Afaik excel does use floats, so maybe it’s excel, which is wrong.

CharlesO

CharlesO OP

all my values are 2-decimal places
The Validator Program (some proprietor java program) also tells me the sum is wrong:

MMR100: Total is not equal to sum of all monthly balance for scheme…

When i manually adjust the sum to that excel value and feed it to the validator .. it validates

NobbZ

NobbZ

Parsing and summing the list as float gives: 208505442949.07806, rounding this to 2 decimal places returns 208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…

PS: I’ve heard there are still people using Excel to manage their money…

PPS: my iex session:

iex(3)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum
208505442949.07806
iex(4)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum |> Float.round(2)
208505442949.08
CharlesO

CharlesO OP

All the numbers in that list are 2-dp, how can you get 208505442949.07806 ???

NobbZ

NobbZ

LostKobrakai

LostKobrakai

This exactly is the reason, why you’d even need a library like Decimal. Because floats are not suited for calculating correctly.

CharlesO

CharlesO OP

Every single finance department i have dealt with keeps parallel records in excel.

For some some reason there is an innate distrust of records from IT systems

NobbZ

NobbZ

No, I do get 9:

iex()> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.graphemes/1) |> Enum.map(&List.last/1) |> Enum.map(&String.to_integer/1) |> Enum.sum
1349409
LostKobrakai

LostKobrakai

sribe

sribe

Bah. I only looked at the text in the post. Did not notice the “this file has been truncated”…

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New
psy-q
I’m trying to set up Emacs with elixir-ls via lsp-mode and credo via Flycheck. This should mostly be preconfigured as Flycheck picks up c...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews