CharlesO
Hi.
Please I’m getting this unusual rounding error when summing numbers using the Decimal Library
My result is off by 0.01
208,505,442,949.08 # Excel
208,505,442,949.09 # sum function with Decimal
Any suggestions please?
Thanks.
def sum() do
f = "C:/Downloads/MMR100.txt"
{:ok, b} = :file.read_file(f)
a = String.split(b,"\n")
c = Enum.count(a)
IO.puts c # 299,725
_sum(a, "0.00") |> Decimal.to_string(:normal)
# 208,505,442,949.08 - Excel
# 208,505,442,949.09 - sum
end
defp _sum([], acc), do: acc
defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))
Trending in Questions
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
Hello,
I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
Hi everyone,
I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding.
I sta...
New
Documentation
While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
So my question is quite simple and i have found no conclusive answer on forum, google or AI.
Should we use :erlang.float for Integer to ...
New
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
I’m new to elixir and just tried to install the elixirLS extension for VScode(ium) and it is throwing some errors that I would like help ...
New
Other Trending Topics
Edit: 2026 May 15 - This post is archived.
Mob is alive!!
Main docs: mob v0.7.11 — Documentation
A bit of explanation for the slightly c...
New
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Hi everyone!
The first release candidate for the Expert language server project is now available!
We’ve published a press release detai...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex










Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
LostKobrakai
Afaik excel does use floats, so maybe it’s excel, which is wrong.
CharlesO
all my values are 2-decimal places
The Validator Program (some proprietor java program) also tells me the sum is wrong:
When i manually adjust the sum to that excel value and feed it to the validator .. it validates
NobbZ
Parsing and summing the list as float gives:
208505442949.07806, rounding this to 2 decimal places returns208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…PS: I’ve heard there are still people using Excel to manage their money…
PPS: my iex session:
CharlesO
All the numbers in that list are 2-dp, how can you get
208505442949.07806???NobbZ
The magic of floats.
LostKobrakai
This exactly is the reason, why you’d even need a library like
Decimal. Because floats are not suited for calculating correctly.CharlesO
Every single finance department i have dealt with keeps parallel records in excel.
For some some reason there is an innate distrust of records from IT systems
NobbZ
No, I do get
9:LostKobrakai
sribe
Bah. I only looked at the text in the post. Did not notice the “this file has been truncated”…