Thank you, José. Enabling :lazy cut the time in half. Your suggestion also made me read the docs with more attention and I found that I could set the floats to f32 instead of using f64, which had been automatically inferred.
This made the computation light enough to fit in memory and go even faster, regardless of lazy mode.
Results:
Reading and aggregating 1 Billion Lines with Explorer
- Eager f64: 675483.00ms
- Lazy (f64): 389491.00ms
- Lazy (f32): 53575.23ms
- Eager f32: 55091.87ms
