Zsolt

Zsolt

Using FLAME to run code in ephemeral machines

I’m trying to get some code to run in Flyio Flame machines.

I have a phoenix app (although, I think the question applies more generally to Elixir and Flame).

There is a live file upload module that takes a few files, and using an image library resizes, converts and saves them to disk. The path to the images is saved in a Postgres db. Very simplistic, this is a learning tool, not a production app.

Resizing and saving the files is not CPU intensive, but it can use up a lot of memory, so it would help to do each operation on a new machine. (I’m aware other kinds optimizations are possible, but the purpose of this is to try Flame).

This blog post (Rethinking Serverless with FLAME · The Fly Blog) seems to state that it’s enough to wrap a chunk of code in a FLAME.call with a FLAME.Pool and those chunks should run on separate machines.

That’s it! FLAME.call accepts the name of a runner pool, and a function. It then finds or boots a new copy of our entire application and runs the function there.

I have this implementation and I’m not seeing and Flame machines start up. Am I doing something wrong? Did I misunderstand the post, and do I in fact need the entire GenServer and FLAME.place_child functionality?

The FLAME.Pool in /lib/phoenix_albums/application.ex

  @impl true
  def start(_type, _args) do
    flame_parent = FLAME.Parent.get()

    children =
      [
        PhoenixAlbumsWeb.Telemetry,
        PhoenixAlbums.Repo,
        {DNSCluster, query: Application.get_env(:phoenix_albums, :dns_cluster_query) || :ignore},
        {Phoenix.PubSub, name: PhoenixAlbums.PubSub},
        # Start the Finch HTTP client for sending emails
        {Finch, name: PhoenixAlbums.Finch},
        # Start a worker by calling: PhoenixAlbums.Worker.start_link(arg)
        # {PhoenixAlbums.Worker, arg},
        # Start to serve requests, typically the last entry
        {FLAME.Pool,
         name: PhoenixAlbums.ImageProcessor, min: 0, max: 10, max_concurrency: 1, log: :debug},
        !flame_parent && PhoenixAlbumsWeb.Endpoint
      ]
      |> Enum.filter(& &1)

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: PhoenixAlbums.Supervisor]
    Supervisor.start_link(children, opts)
  end

My upload handler in the live fodler:

 @impl Phoenix.LiveView
  def handle_event("save", params, socket) do
    unless File.exists?(@folder) do
      File.mkdir!(@folder)
    end

    uploaded_files =
      consume_uploaded_entries(socket, :avatar, fn %{path: path}, entry ->
   
        uuid = UUID.generate()
        image_folder = "#{@folder}/#{uuid}"

        unless File.exists?(image_folder) do
          File.mkdir!(image_folder)
        end

        FLAME.call(
          PhoenixAlbums.ImageProcessor,
          fn -> resize_image(:thumbnail, path, 150, image_folder) end
        )

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          resize_image(:medium, path, 800, image_folder)
        end)

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          save_original(path, image_folder)
        end)

        image = Map.merge(params, %{"user_id" => socket.assigns.user_id, "url" => image_folder})

        case Album.create_image(image) do
          {:ok, image} ->
            socket
            |> put_flash(:info, "Image created successfully.")

            {:ok, ~p"/images/#{image}"}
        end
      end)

    {:noreply, update(socket, :uploaded_files, &(&1 ++ uploaded_files))}
  end

Here I just wrapped resize_image and save_original functions, with the expectation, that these would be run on separate machines. The code runs on fly just the same as without the wrappers, and I’m not sure why.

Marked As Solved

chrismccord

chrismccord

Creator of Phoenix

Have you configured the FlyBackend? You can either pass the backend to the pool options, or per the FlyBackend docs:

The only required configuration is telling FLAME to use the
FLAME.FlyBackend by default and the :token which is your Fly.io API
token. These can be set via application configuration in your config/runtime.exs
withing a :prod block:

  if config_env() == :prod do
    config :flame, :backend, FLAME.FlyBackend
    config :flame, FLAME.FlyBackend, token: System.fetch_env!("FLY_API_TOKEN")
    ...
  end

Last Post!

willfore

willfore

You ever figure this out? I seem to be having the same issue right now. Only I’m not using single_use: true

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New

We're in Beta

About us Mission Statement