What Elixir related stuff are you doing?

Chat / Discussions
807

Launched my side-project: Newsmaker AI

Please check it out. Don’t break it, LOL.

If you want to keep up with news on social media like Twitter, but don’t want to constantly check for new posts - Newsmaker AI will do it for you. It will also summarize the content like a blog post and send it to you by email at the frequency you specify.

The stack is: Phoenix, LiveView, SQLite Postgres, Raspberry Pi, Kamal, CapRover.

That was a fun process.

Some things that I remember

  • I wanted to try out self-hosting and Kamal for deployments. Ordered a Raspberry Pi with a NVMe drive. Set it up, hooked it up to my router and started playing with deployments. Raspberry Pi is also a very useful thing even without the hosting part. I installed AdGuardHome on it and now it blocks ads for all devices in the network.

This is where the site is hosted:

IMG_0412 2
IMG_0412 21200×980 33.7 KB

  • Turned out - Kamal didn’t really work for me, because of how my Pi is exposed to the internet. I tunnel traffic with Couldflare straight to the Pi and the container, so that I don’t need to worry about dynamic ips and also Cloudflare has some nice security features. Ended up with Caprover, it works quite nicely and also has a 1-click install thingy for Cloudflare. Really happy with this setup. Only downside is - when I deploy, the container is changed and it takes some seconds for Cloudflare to pick up the change. Users see a 500 in the meantime. Not great, but not terrible either. I moved all my projects to CapRover.

  • I initially started with SQLite, because I wanted to try it. Turns out - SQLite is a really great DB. I ended up contributing to ecto_sqlite and exqlite to make it work even nicer with Phoenix. [1] [2] Also learned that Oban works very well, but Oban Web doesn’t work with SQLite (also I can’t afford it for a side-project). Created Obanalyze to have a dashboard.

  • While SQLite is nice - I did run into some Database busy errors recently. With SQLite you can have 1 writer at a time. busy_timeout helps, but I figured that it will only get worse with more users, so I jumped ship a couple of days ago and moved to Postgres. It was interesting to migrate the data, because I already had some users at this time, and I didn’t want to email them and ask to register again, because I moved to new DB :smile: There are some tools to move data from SQLite to Postgres, like pgloader, but it didn’t work out of the box for me, so I wrote my own Elixir script that loaded data with Exqlite, converted it a bit and wrote to Postgres. Found out that when inserting data with my own IDs (I wanted to have IDs 1:1 like in SQLite), I needed to reset sequences in every table, because it doesn’t happen automatically (IMO, would be nice).

  • Building the UI was also a nice challenge. I think I tried all available component libraries for LiveView and ended up buying Fluxon UI. It is really polished and I really liked that the components have the same look and feel. @andrielfn is also very supportive and fixes any issues very fast.

What’s next?

  • Going to add Bluesky support next and some other sources. Maybe Reddit, HN and something else.
  • Dedicate more time to experimenting with different prompts. It kinda works right now, but maybe could be improved.
  • Think of a better algorithm to pick content before feeding it to AI. For example: if I follow the #MyElixirStatus hashtag and pick weekly news - the amount of tweets in a week can be quite large (200-300). That’s probably already a big amount of data and if done at scale - my OpenAI bill will be significant. But how to reduce the dataset without losing the quality… :thinking: Things like this.
10 Likes