CharlesO
Hi.
Please I’m getting this unusual rounding error when summing numbers using the Decimal Library
My result is off by 0.01
208,505,442,949.08 # Excel
208,505,442,949.09 # sum function with Decimal
Any suggestions please?
Thanks.
def sum() do
f = "C:/Downloads/MMR100.txt"
{:ok, b} = :file.read_file(f)
a = String.split(b,"\n")
c = Enum.count(a)
IO.puts c # 299,725
_sum(a, "0.00") |> Decimal.to_string(:normal)
# 208,505,442,949.08 - Excel
# 208,505,442,949.09 - sum
end
defp _sum([], acc), do: acc
defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))
Trending in Questions
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
Documentation
While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
Hello,
I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind.
However, when I launch mix phx.server, I get an error...
New
Hi everyone,
I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding.
I sta...
New
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
I’m working on a small exercise involving update_in/3, and I came up with this solution:
data = %{
name: "Periodic Table",
category:...
New
I’ve got trouble wrapping my head around the order in which functions are called in this snippet (from Phoenix’s authentication):
toke...
New
Other Trending Topics
Edit: 2026 May 15 - This post is archived.
Mob is alive!!
Main docs: mob v0.7.11 — Documentation
A bit of explanation for the slightly c...
New
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Hobbes is a low-level distributed database for the Elixir programming language.
Hobbes provides a simple, safe, and scalable storage lay...
New
Hi everyone!
The first release candidate for the Expert language server project is now available!
We’ve published a press release detai...
New
A little off-topic, but I feel like people here have a good head on their shoulders.
I used to be quite good at making software. Was luc...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex










Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
LostKobrakai
Afaik excel does use floats, so maybe it’s excel, which is wrong.
CharlesO
all my values are 2-decimal places
The Validator Program (some proprietor java program) also tells me the sum is wrong:
When i manually adjust the sum to that excel value and feed it to the validator .. it validates
NobbZ
Parsing and summing the list as float gives:
208505442949.07806, rounding this to 2 decimal places returns208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…PS: I’ve heard there are still people using Excel to manage their money…
PPS: my iex session:
CharlesO
All the numbers in that list are 2-dp, how can you get
208505442949.07806???NobbZ
The magic of floats.
LostKobrakai
This exactly is the reason, why you’d even need a library like
Decimal. Because floats are not suited for calculating correctly.CharlesO
Every single finance department i have dealt with keeps parallel records in excel.
For some some reason there is an innate distrust of records from IT systems
NobbZ
No, I do get
9:LostKobrakai
sribe
Bah. I only looked at the text in the post. Did not notice the “this file has been truncated”…