CharlesO
Hi.
Please I’m getting this unusual rounding error when summing numbers using the Decimal Library
My result is off by 0.01
208,505,442,949.08 # Excel
208,505,442,949.09 # sum function with Decimal
Any suggestions please?
Thanks.
def sum() do
f = "C:/Downloads/MMR100.txt"
{:ok, b} = :file.read_file(f)
a = String.split(b,"\n")
c = Enum.count(a)
IO.puts c # 299,725
_sum(a, "0.00") |> Decimal.to_string(:normal)
# 208,505,442,949.08 - Excel
# 208,505,442,949.09 - sum
end
defp _sum([], acc), do: acc
defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))
Trending in Questions
Hello!
Suppose you are building workflow (order / task / payment) processing system with the following requirements:
Each workflow con...
New
I’m aware of 2 such projects, are there others?
git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
Hi all,
I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
How do I tell Hologram not to escape HTML when substituting into templates?
{Textile.render(@product.long_description)}
escapes the HTM...
New
I’d like to create a file with 600 permissions (rw-------), but there’s a race condition if I create it and then change the permissions. ...
New
Hi,
we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New
When I use Mix.install with a git dependency, using git@github.com:... how do I change the SSH identity used?
I’ve got two Github accoun...
New
Other Trending Topics
Hey there,
I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first.
...
New
Hi everyone :waving_hand:
I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #elixirconf-us
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #security
- #elixirconf-eu
- #api
- #forms
- #performance










Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
LostKobrakai
Afaik excel does use floats, so maybe it’s excel, which is wrong.
CharlesO
all my values are 2-decimal places
The Validator Program (some proprietor java program) also tells me the sum is wrong:
When i manually adjust the sum to that excel value and feed it to the validator .. it validates
NobbZ
Parsing and summing the list as float gives:
208505442949.07806, rounding this to 2 decimal places returns208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…PS: I’ve heard there are still people using Excel to manage their money…
PPS: my iex session:
CharlesO
All the numbers in that list are 2-dp, how can you get
208505442949.07806???NobbZ
The magic of floats.
LostKobrakai
This exactly is the reason, why you’d even need a library like
Decimal. Because floats are not suited for calculating correctly.CharlesO
Every single finance department i have dealt with keeps parallel records in excel.
For some some reason there is an innate distrust of records from IT systems
NobbZ
No, I do get
9:LostKobrakai
sribe
Bah. I only looked at the text in the post. Did not notice the “this file has been truncated”…