Zsolt

Zsolt

Using FLAME to run code in ephemeral machines

I’m trying to get some code to run in Flyio Flame machines.

I have a phoenix app (although, I think the question applies more generally to Elixir and Flame).

There is a live file upload module that takes a few files, and using an image library resizes, converts and saves them to disk. The path to the images is saved in a Postgres db. Very simplistic, this is a learning tool, not a production app.

Resizing and saving the files is not CPU intensive, but it can use up a lot of memory, so it would help to do each operation on a new machine. (I’m aware other kinds optimizations are possible, but the purpose of this is to try Flame).

This blog post (Rethinking Serverless with FLAME · The Fly Blog) seems to state that it’s enough to wrap a chunk of code in a FLAME.call with a FLAME.Pool and those chunks should run on separate machines.

That’s it! FLAME.call accepts the name of a runner pool, and a function. It then finds or boots a new copy of our entire application and runs the function there.

I have this implementation and I’m not seeing and Flame machines start up. Am I doing something wrong? Did I misunderstand the post, and do I in fact need the entire GenServer and FLAME.place_child functionality?

The FLAME.Pool in /lib/phoenix_albums/application.ex

  @impl true
  def start(_type, _args) do
    flame_parent = FLAME.Parent.get()

    children =
      [
        PhoenixAlbumsWeb.Telemetry,
        PhoenixAlbums.Repo,
        {DNSCluster, query: Application.get_env(:phoenix_albums, :dns_cluster_query) || :ignore},
        {Phoenix.PubSub, name: PhoenixAlbums.PubSub},
        # Start the Finch HTTP client for sending emails
        {Finch, name: PhoenixAlbums.Finch},
        # Start a worker by calling: PhoenixAlbums.Worker.start_link(arg)
        # {PhoenixAlbums.Worker, arg},
        # Start to serve requests, typically the last entry
        {FLAME.Pool,
         name: PhoenixAlbums.ImageProcessor, min: 0, max: 10, max_concurrency: 1, log: :debug},
        !flame_parent && PhoenixAlbumsWeb.Endpoint
      ]
      |> Enum.filter(& &1)

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: PhoenixAlbums.Supervisor]
    Supervisor.start_link(children, opts)
  end

My upload handler in the live fodler:

 @impl Phoenix.LiveView
  def handle_event("save", params, socket) do
    unless File.exists?(@folder) do
      File.mkdir!(@folder)
    end

    uploaded_files =
      consume_uploaded_entries(socket, :avatar, fn %{path: path}, entry ->
   
        uuid = UUID.generate()
        image_folder = "#{@folder}/#{uuid}"

        unless File.exists?(image_folder) do
          File.mkdir!(image_folder)
        end

        FLAME.call(
          PhoenixAlbums.ImageProcessor,
          fn -> resize_image(:thumbnail, path, 150, image_folder) end
        )

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          resize_image(:medium, path, 800, image_folder)
        end)

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          save_original(path, image_folder)
        end)

        image = Map.merge(params, %{"user_id" => socket.assigns.user_id, "url" => image_folder})

        case Album.create_image(image) do
          {:ok, image} ->
            socket
            |> put_flash(:info, "Image created successfully.")

            {:ok, ~p"/images/#{image}"}
        end
      end)

    {:noreply, update(socket, :uploaded_files, &(&1 ++ uploaded_files))}
  end

Here I just wrapped resize_image and save_original functions, with the expectation, that these would be run on separate machines. The code runs on fly just the same as without the wrappers, and I’m not sure why.

Marked As Solved

chrismccord

chrismccord

Creator of Phoenix

Have you configured the FlyBackend? You can either pass the backend to the pool options, or per the FlyBackend docs:

The only required configuration is telling FLAME to use the
FLAME.FlyBackend by default and the :token which is your Fly.io API
token. These can be set via application configuration in your config/runtime.exs
withing a :prod block:

  if config_env() == :prod do
    config :flame, :backend, FLAME.FlyBackend
    config :flame, FLAME.FlyBackend, token: System.fetch_env!("FLY_API_TOKEN")
    ...
  end

Last Post!

willfore

willfore

You ever figure this out? I seem to be having the same issue right now. Only I’m not using single_use: true

Where Next?

Popular in Questions Top

joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New

Other popular topics Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49084 226
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement