sax

sax

Exceed - streaming XLSX generation

Hello! This is a little late in terms of years since shipping, but I just realized that I’d never shared exceed, which is a library for creating Excel spreadsheets from streams. Maybe it will be helpful for others.

When generating smallish spreadsheets (100s of thousands of rows), it performs about 30-40% as quickly as elixlsx. When you get to millions of rows on cheap/small infrastructure, it catches up. One of the best features is that, with chunked output from a Phoenix controller, file download can start almost immediately without having to wait for the entire file to be completely generated.

Recently I’ve spent time improving performance, and benchmarking it against elixlsx, which is a fantastic library. We wrote ours after working on a project that created multi-GB spreadsheets containing tens of millions of rows, where we didn’t want to pay the cost of building the entire spreadsheet in memory.

https://github.com/synchronal/exceed

Most Liked

sax

sax

Regarding performance, I think that it could be a lot faster if we replaced the XML generation with simpler string interpolation, like elixlsx does, rather than the current way which uses a lot of nested iodata to be passed to the XmlStream library. I would be happy to be proven wrong (please do and file PRs!), but I think I’m at the limit of what can be eked out without rewriting the XML generation and/or zipping.

I think an extremely fast (binary vs iodata) stream-based XML generator would be a nice thing to have, but don’t really have the time to do it right now without some sort of sponsorship.

Where Next?

Popular in Announcing Top

danschultzer
In short Plug n’ play OAuth 2.0 provider library. Just set up a resource owner schema with Ecto (your user schema), install the dependen...
New
mischov
import Meeseeks.CSS html = HTTPoison.get!("https://news.ycombinator.com/").body for story <- Meeseeks.all(html, css("tr.athing")) do...
New
brainlid
LangChain is short for Language Chain. An LLM, or Large Language Model, is the “Language” part. This library makes it easier for Elixir a...
New
MRdotB
I needed to reuse React components from my Chrome extension in my Phoenix/LiveView backend. I noticed that for Svelte/Vue, there are live...
New
wojtekmach
Hey everyone! Req is an HTTP client for Elixir that I’ve been working on for quite some time. There is already a lot of HTTP clients out...
New
aditya7iyengar
Rummage.Ecto and Rummage.Phoenix provide ways to perform Searching, Sorting and Pagination over Ecto queries and Phoenix collections. Fo...
New
Flo0807
Hello everyone! I am excited to share our heart project Backpex with you. After building several Phoenix applications, we realized that...
New
anshuman23
Hello all, I have been working on my proposed project called Tensorflex as part of Google Summer of Code 2018.. Tensorflex can be used f...
New
zoltanszogyenyi
Hey everyone :waving_hand: Excited to join this forum - I am one of the founders and current project maintainers of a popular and open-s...
New
wfgilman
I’ve cleaned up and open sourced three financial libraries I was using for my company. They are bindings for the APIs of these three comp...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30877 112
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

We're in Beta

About us Mission Statement