Offline-enabled apps with Phoenix LiveView and LiveSvelte

tonydang · February 16, 2024, 11:15pm

Hi everyone! I’m new to the forums and new to Elixir in general. To learn Elixir, I recently just completed my first app. It’s a todo app built with Phoenix LiveView, but with a twist, it can also work offline

By using the LiveSvelte package, I was able to use LiveView to render Svelte components which allows for offline-support. I know an offline app was not what LiveView was intended for, but I thought it was neat that it was possible to still use LiveView in such an app.

I haven’t gotten the chance to dive deeper into Phoenix Channels yet so I’m not sure if the app would perform better just using that instead. However, so far the app has been working well for me and the DX of using LiveView has been great. Anyhow, I thought I would share in case anyone else is interested in building offline-enabled apps with LiveView

Here’s a video I made walking through how I made the app:

Source code:

Live Demo:

Any questions and feedback welcomed!

arcanemachine · February 17, 2024, 12:29am

Very cool. I tried it out and it works as promised.

Would you mind putting a license on the repo so that I know what I’m allowed to do with the code?

tonydang · February 17, 2024, 12:42am

Thanks for checking out the app!

And yes, I just added an MIT license. Please feel free to use the code as you please

hellosmithy · February 21, 2024, 3:12am

Very cool to see a CRDT example with Elixir. This was on my bucket list of things to try out. Thanks for sharing.

phi · February 21, 2024, 3:46pm

Awesome work. Would you mind adding a .env.example for us newbies?

tonydang · February 21, 2024, 10:55pm

Absolutely, I just added a .env.example to the repo:

github.com

tonydangblog/liveview-svelte-pwa/blob/main/.env.example

# This file is a template for the .env file that should be created in the root of the project.
# In dev, change the file name to `.env` and run `source .env` to load the environment variables 
# before running `phx.server`.

# The `JS_BACKEND_URL` is the URL of the JavaScript backend server. A JS server is used for two 
# purposes:
#
# 1. To run Yjs server-side when merging client and server states. In the live demo of this project,
#    I'm using a Cloudflare Worker Function. The code for that can be found here: 
#    https://github.com/tonydangblog/blog/blob/main/apps/backend/src/routes/microservices/yjs/merge/index.ts
#    
#    If you don't want to use a completely separate server for this, you can run Yjs on the same 
#    server as Phoenix with nodejs or one of the available ports: 
#    https://github.com/yjs/yjs?tab=readme-ov-file#Ports 
#
# 2. I'm also using the JS backend to send my emails via Cloudflare. There are built-in ways to send 
#    emails in Phoenix, but I wanted to use Cloudflare's serverless functions to send emails. The 
#    code for that can be found here:
#    https://github.com/tonydangblog/blog/blob/main/apps/backend/src/routes/microservices/mailer/send/index.ts
#

This file has been truncated. show original

I included some comments in the file for more explanation, but in short, it holds the two variables:

JS_BACKEND_URL - The server I’m using to 1. run Yjs server-side, and 2. send emails via Cloudflare Workers.
JWT_PRIVATE_KEY - Private key used to sign JWTs needed when sending requests to the JS backend server.

Please see the comments in the file for more info and feel free to let me know if you have any questions!

phi · February 21, 2024, 11:27pm

Cheers! Will try to get the Yjs backend setup myself, hopefully no issues.

nico_amsterdam · April 6, 2024, 8:22am

Very good explanation of the offline enabled app and it’s concepts.

tonydang · April 15, 2024, 5:59pm

Hi all! I wanted to give an update on this project-

In my video, I discussed using Yjs on the server-side in order to merge the states of the various clients. This worked, but was not ideal since it required running JS from Phoenix or from a separate JS backend server (which is what I did for my project via a Cloudflare Worker).

Since then, I have found out that it is not necessary at all to run Yjs on the server in order to sync the clients. I have updated my repo to no longer need a separate JS backend. I also made a video explaining how it is possible to do so. You can check it out here if interested:

mindreframer · April 15, 2024, 7:53pm

Thanks! That’s quite interesting, but it is clearly problematic for any real-life cases with non-trivial amount of data. I think our best bet as community would be to create a Rustler wrapper package around yrs - Rust to be able to reconcile state diffs (patches). That way we could use it for realistic data amounts and keep the chattiness of the network traffic at bay.

But yeah, someone with Rust skills would have to create the wrapper first

Maybe we’ll have a production quality CRDT bindings in near future.
Cheers!

tonydang · April 15, 2024, 8:16pm

I definitely agree that sending the entire app state every time is not scalable and a Rustler wrapper would be awesome!

One other idea I had that may also work is: Instead of sending the entire app state every time, we could just send Yjs updates for each change. And, on the server, instead of keeping a single app state, we can keep a table of all updates sent. Then, when a client reconnects, it can request all the updates it has missed. I haven’t tried this out, but I think that it should work in theory.

Definitely still not as good as just having a Yjs port, but just thinking out loud

mindreframer · April 15, 2024, 8:36pm

Indeed, using the diff on the client to send just the changed data might work. The only thing I dont like, it the in-ability on the server to process the data in any meaningful way. For your Elixir app this data is just a binary blob without much meaning. It really depends on the use case, for some apps this might be OK. My use case requires some insight into the data on the server, so it wont work out here. BUT… Still great, in case the server should stay dump and clients just exchange data.

Lots of good ideas floating around!

tonydang · April 15, 2024, 8:54pm

Yup, it definitely depends on the use case. For the apps I had in mind, I was actually considering encrypting the user data so that it would only be available to the end user on the client-side. In those cases, not being able to process data server-side actually works out.

Thanks for all the feedback! I’ll definitely have to ponder this one a little more.

tonydang · May 2, 2024, 11:04pm

This came across my radar today - https://extism.org/

It allows you to compile JS to WASM which can then be ran in Elixir! I haven’t played with it at all, but perhaps this can be another way to run Yjs on a Phoenix server.

tonydang · May 3, 2024, 7:17pm

I spoke about the app a little more on the Elixir Mentor Podcast. Here’s a link to the recording if anyone is interested!

RicoTrevisan · May 5, 2024, 7:55pm

FYI: ywasm is a WASM port of Yjs.

How will you run Yjs in your Elixir server? Just start it up when the server starts? Or create a GenServer that calls on it when there’s a connection request?

tonydang · May 6, 2024, 3:27pm

Hi Rico, thanks for sharing the link!

Since I haven’t tried it out yet, I can’t give a firm answer. My initial thought is to first keep things simple- I’d would just make the call to run Yjs via WASM on each request, without a GenServer, to see if that works. Then, I’d try to improve from there once I have a better idea of how things work.

coen.bakker · May 6, 2024, 4:34pm

Just a general comment.

Automerge has worked well for me in an Phoenix LiveView web app.

In case someone is interested in an alternative to yjs.

tonydang · May 6, 2024, 6:43pm

Automerge is a really cool CRDT solution as well! I haven’t tried in much besides setting up a hello world project with it once.

I’m curious, how did you handle syncing? Were you using Automerge’s public sync server or did you build your own Phoenix solution?

coen.bakker · May 6, 2024, 6:55pm

I did the latter.

I used a Rust NIF to be able to create and merge crdt docs on the Elixir server. In combination with creating, merging and editing on the client side with their js library.