Elixir Blog Posts

New post describing how to implement a scrolling data table without running out of memory in Phoenix LiveView

Is there not a limit to how many items the browser is happy to render? If you throw 1,000,000 items into your page, what happens?

There is probably a practical limit to how many items you can shove into the DOM of a page, but the focus here was to minimize the per-connection server memory utilization.

In situations where millions of items should be rendered, it might be worth using a custom hook or some other client-side code to virtualize or GC the list.

I actually think fly.io uses phx-update="append" in their container logs page, which is much higher volume than anything I’ve ever shipped. As far as I can tell, they don’t do anything to clean up the entries. Worst-case, the user can always refresh the page or a filtering UI can be added.

Don’t mind me, I have a personal vendetta against “infinite scroll”. I have a somewhat lacklustre implementation of truly infinite scrolling so I always read posts about it with interest. Sadly, it always turns out to be finite scroll. It should be called “load a bit more until the browser collapses scroll” or “generally sufficient amount of items scroll”.

AFAIK if you want actual infinite scroll, you need virtual scrolling.

we knew that browsers start struggling with more than 20k log lines

2 Likes

How do you efficiently search through millions of names in Postgres and Ecto? This blog post dives into Postgres’ ILIKE and SIMILARITY operators and their important caveats.

Also included: The most in-depth explanation of how Postgres calculates the SIMILARITY score ever!

1 Like

Enjoyed reading it!

Regarding:

SET pg_trgm.similarity_threshold = 0.3;

Sadly, at the time of writing, we were not able to find a solution that allows us to configure this threshold for an Ecto.Repo (tips are more than welcome).

You could do:

query_args = ["SET pg_trgm.similarity_threshold = 0.3", []]

config :my_app, MyApp.Repo,
  username: "postgres",
  password: "postgres",
  database: "demo_dev",
  hostname: "localhost",
  pool_size: 10,
  after_connect: {Postgrex, :query!, query_args}

The above is an adjusted excerpt from the ebook “The Little Ecto Cookbook” by Dashbit, where they showed how to set a particular schema for a repo.

2 Likes

Thank you! Oh that’s so cool! I’ll update the blog post tomorrow! thanks so much! :slight_smile:

We’ve published a new article, “Oban Starts Where Tasks End,” to answer the question, “why do we need background jobs when we have tasks?”

6 Likes

This article will focus on two intertwined languages: Elixir and Erlang.

1 Like

Cross Site Scripting (XSS) Patterns in Phoenix

If you’ve ever been involved with a pentest on an Elixir application, you know someone calling “raw/1” directly on user input is rare, along with bypassing the view/template/render pattern to render HTML. The author of Sobelow, Griffin Byatt, mentions this in his ElixirConf talk, ElixirConf 2017 - Plugging the Security Holes in Your Phoenix Application It’s a fantastic talk, he gives the example of file upload leading to XSS. I close out this blog post with that example, and hope you find it useful in ensuring the security of your own apps.

1 Like

Thanks for the helpful information. I will be putting this to use.

1 Like

Nice find and while you are at it there is also @moduletag which allows you to do the same but at the module level. I use this tagging to segment my unit tests from their integration and system tests siblings.

1 Like

Nice series. Is this something you could also leverage for Facebook Workplace?

This epic article of 4500 words covers everything there is to know about efficient full-text search with Postgres and Ecto. It explains how Postgres’ full-text search works and how you can use indexes to speed up your search. It also covers search modifiers, query testing, extracting headlines, and so much more!

9 Likes

I fell off the writing wagon a couple years ago, after my first child was born, but I’m trying to hop back on! Here’s a rundown of a method I’ve been using to handle combinatorial explosions of query parameters in my Phoenix applications.

In this blog post, I share how to create a reproducible development environment for Elixir with Nix. Let me know what you think :slight_smile:

1 Like

Today’s article explains how to simulate a bad internet connection when developing locally by adding latency, jitter, and package loss to your connection. It’s crucial to test your app under these circumstances before deploying it. Otherwise, especially users on a mobile connection will have a bad UX when using your site.

4 Likes

Securing Elixir/Phoenix Applications: 5 Tips to Get Started

Hey everyone, this article consists of five recommendations:

  1. Read the EEF’s Secure Coding and Deployment Hardening Guidelines
  2. Use Sobelow for static analysis
  3. Check for vulnerable dependencies with MixAudit
  4. Harden your application against bot attacks (I don’t recommend reCaptcha)
  5. Use the SafeURL library to stop SSRF attacks

Hope it’s useful in your work!

2 Likes

How to serve the webfinger protocol using phoenix:

5 Likes

Learn how to build a Roles and Permissions (RAP) system for Phoenix! Not all your users should have access to all your data. A RAP helps you to set up user groups and control which data they can access through permissions.

3 Likes