(Just kidding you’re more than welcome, dear data scientist)
I’m trying to find an angle to a presentation I’ll be giving at FOSDEM in a few weeks. Reading up on the origin of Livebook, I see a lot of the initial momentum is coming from this paper that explores the current state of computational notebooks and what’s missing.
I think it’s fascinating and brings a lot of value. But… I’m not a data scientist. Still, I do find a lot of value in Livebook, and I use it as often as I can. My angle is more from a software-building perspective. The value for me comes more from getting feedback as fast as possible, because a notebook allows you to start from the middle, where most risk usually lives. That’s the side I like to hint at when talking about Livebook, and trying to show people why they might also be interested.
I’m not sure what I want to discuss here. Maybe I’m just trying to find some encouragement about the fact that Livebook is not only about data science (wrangling data, doing cool ML stuff, making fancy scientific graphs, etc.), but is also a great tool in our toolbox to create actual working software. Sharing code, documenting it, and exploring alternatives while we’re working in non-sciency domains. Some testimonies about people using Livebook in a similar software engineery way could spark inspiration, and validate my ideas.
Maybe my main question is whether these ideas I’m expressing here still fit in how Livebook is conceived? At the same time, I experience what I experience: Livebook helps me create better software, and tackle problems at the right place, doing high-risk stuff first.
I’d be perfectly ok if this stays a monologue, and tumbleweed silently rolls by. But I’d be just as appreciative for anyone willing to share how they use Livebook in a similar fashion.
I am also giving a talk at FOSDEM. And I’m prototyping it in Livebook Markdown cells interspersed with code cells so I can validate the code works. I was toying with the idea of using Livebook for the presentation as well, but I will probably use some other tool for that.
In my day job I use Livebook a lot for prototyping and one-off tasks. Fire up Livebook on an embedded device, install the dependencies I think I need, open a connection to some other system I am talking to, and begin sending/receiving bytes. It is a good opportunity for early feedback, when I don’t trust the spec (or my understanding of the spec). This is especially useful when I can plot the data I receive in real-time with VegaLite.
In terms of one-off tasks, I have used it to pull data from a database, filter/map/reduce it and throw it into a VegaLite plot to look for patterns. I also use it at times to parse/filter/map/reduce data in CSV files and plot it, or just poke at the data programatically. I use it to compensate my missing Excel skills Very useful tool. Better than raw IEx, because 1) it’s easy to back and forth and iterate on the data transformations and 2) I can plot the data.
I am not giving a talk at FOSDEM, however I have visited it a couple of times in the past. I am a software developer by trade for many years already.
In short, I use Livebook mostly to do the quick prototyping, gathering and checking integrations of libraries and own idea’s ( the middle code as you put it)
I love that many libraries are starting to convert their documentation into Livebooks. ( so I don’t have to anymore )
Those are nice usages! I forgot the nerves community also gets great value from Livebook.
I’ve seen José and Kip give great presentations with only Livebook as a visual aid. The only thing that makes it a bit more difficult to follow is the lack of clear borders between chunks (slides in a slideshow) of ideas.
The exploring of data in CSV files is also very recognizable. I’m currently trying to figure out how i can do a data migration for a rewrite we’re planning, in Livebook (either in pure postgress, or maybe use ETS, or an other in-memory database).
Goodluck with your preparations! Let’s keep in touch and see each other there. Thanks for sharing.
This validates how I use Livebook too. “Quick prototyping” is a good summary.
I didn’t use too many pre-built notebooks yet (except for the learning resources bundled with Livebook, while I was getting to know Livebook). https://notes.club/ is a nice initiative to make livebooks discoverable.
I never tried, however I think livebook should be very powerful when it comes to teaching either new concepts or the language, you can structure the tutorial with code examples, where the user can actually try out the code and fiddle with it. This is the missing feature in a book, where you get snippets of code, however all the setup and copy-paste of the code should be done by you.
Another use-case that I almost got to use livebook for is to write high-level documentation of a feature. While pure documentation is sufficient for most cases, writing documentation containing an actual running example with notes is very powerful in my opinion.
Hi, I’ve also used Livebook for prototyping/discovery. One of the most recent use cases that comes to mind was when we integrated App Store/Play Store subscriptions into our mobile app.
Just for the Apple app store alone this meant making calls to the Apple API to verify data we got from the client and verifying JWTs we received with API callbacks from Apple’s servers, for example. I started implementing at a low-level in Livebook (e.g. making API calls and inspecting the responses) and whenever I felt confident that my assumptions are correct, I started extracting/moving the code into a module in the project.
That’s a nice story too. And it reminds me that there is something to say about how you prototype stuff, and then move it into your project code. I have the same dynamic going on. Do you bother to add your mix project as a dependency, and then reference the project dependencies, instead of the modules in your Livebook? Btw, since v0.8.0 the doctests of modules run automatically, which is also a nice improvement, because now you can get to see the results of thoses tests too in Livebook.
you’re more than welcome, dear data scientist)
Markdown cells interspersed with code cells so I can validate the code works. I was toying with the idea of using Livebook for the presentation as well, but I will probably use some other tool for that.
Very useful tool. Better than raw IEx, because 1) it’s easy to back and forth and iterate on the data transformations and 2) I can plot the data.
