<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="310648" data-post-id="310648">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="niccolox" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/niccolox/120/2601_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  niccolox
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I’d be interested to see a Membrane video + AI + Fly + FLAME demo app</p>
<p>if I am reading this correctly, also interested in how existing monoliths could FLAME out expensive services like video or AI</p>
<p>some folks with bandwidth and commercial skin in the game would benefit the Elixir Phoenix ecosystem if they could do a side-by-side $ comparison of FLAME fly vs Lambda AWS etc</p>
<p>with tech landscape facing budget constraints right now, focus on ROI with FLAME</p>
<p>I really like the just-in-time or on demand or drop in Lambda, the ability to make your monolith fragment into ephemeral microservices</p>
<p>certainly another genuine game changer from <a class="mention" href="/u/chrismccord" rel="nofollow">@chrismccord</a></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310648" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-310648" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310648"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="310660" data-post-id="310660">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="adw632" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  adw632
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="niccolox" data-post="13" data-topic="60209">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/niccolox/48/2601_2.png" class="avatar"> niccolox:</div>
<blockquote>
<p>if I am reading this correctly, also interested in how existing monoliths could FLAME out expensive services like video or AI</p>
</blockquote>
</aside>
<p>Cost benefits will come back to the flag fall metering model for on demand workloads vs a commitment tier.</p>
<p>At the very low end it may be beneficial, but at some point you will be better off with a commitment tier as that always provides the largest discount because cloud service providers are guaranteed ROI when you commit. They may be some benefit to using demand based between increments in number of nodes on a  commitment tier and to handle spikes that exceed any reserve capacity in your commitments.</p>
<p>Where FLAME is different is that you can delegate <strong>specialised</strong> work to a pool that scales to zero without a lot of complexity so your primary nodes continue serving the non-specialised work.</p>
<p>This also afords targeting work to run on specialised VMs and <strong>very little change to the code to achieve a very different architecture.</strong></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310660" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-310660" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310660"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="310671" data-post-id="310671">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lud" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/120/14382_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lud
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<pre data-code-wrap="elixir"><code class="lang-elixir">  @doc false
  def __open__(%File.Stream{path: path, node: node}, modes) when node == node() do
    :file.open(path, modes)
  end

  @doc false
  def __open__(%File.Stream{path: path, node: node}, modes) do
    :erpc.call(node, :file_io_server, :start, [self(), path, List.delete(modes, :raw)])
  end
</code></pre>
<p>Alright, the node is stored in the stream struct and the file stream implementation will just ask the right node to do the work.</p>
<p>Indeed this is so simple and Just Works™ <img src="https://forum.elixirforum.com/images/emoji/apple/smiley.png?v=15" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310671" data-batch-url="/posts/batch_likers">
                        6
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-310671" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310671"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="310694" data-post-id="310694">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="tiagodavi" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/tiagodavi/120/32293_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  tiagodavi
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>This is great. this reminds me this decorator python function that runs functions on different infrastructures with GPU attached created by Fal.ai</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://fal.ai/docs/documentation">
  <header class="source">
      <img src="https://fal.ai/docs/_mintlify/favicons/fal-d8505a2e/cUEBujT4zfUs0Rxq/_generated/favicon/android-chrome-192x192.png" class="site-icon" alt="" width="192" height="192">

      <a href="https://fal.ai/docs/documentation" target="_blank" rel="noopener nofollow ugc">fal</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/361;"><img src="https://fal-d8505a2e.mintlify.app//logo/light.svg" class="thumbnail" alt="" width="120" height="48"></div>

<h3><a href="https://fal.ai/docs/documentation" target="_blank" rel="noopener nofollow ugc">fal Docs</a></h3>

  <p>Explore 1,000+ AI models for image, video, and audio generation. API references, SDKs, and guides for model APIs, serverless deployment, and GPU compute.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310694" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-310694" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310694"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="310705" data-post-id="310705">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="chrismccord" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/chrismccord/120/24233_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  chrismccord
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Phoenix</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>You can also do that with Fly GPU machines, where your app is running in a regular machine, and FLAME in GPU. For example:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">{FLAME.Pool,
 name: BBRunner,
 backend:
   {FLAME.FlyBackend,
   gpu_kind: "a100-pcie-40gb", cpu_kind: "performance", cpus: 8, memory_mb: 20480}},
</code></pre>
<p>You’ll need to consider model load time on cold start, which could be 10-20s to get things loaded into memory for sizable models. Dynamic provisioning any kind of ML setup pays this price, but something to keep in mind. You’ll also want to bake the model/xla/bublebee artificats into your build step for as long as is feasible, so all the cached artifacts are in the docker container without needing to pull them from elsewhere.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310705" data-batch-url="/posts/batch_likers">
                        4
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-310705" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310705"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #16"></div>
  </section>
</div>
    <div class="postbit" id="310706" data-post-id="310706">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="chrismccord" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/chrismccord/120/24233_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  chrismccord
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Phoenix</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>According to <a class="mention" href="/u/seanmor5" rel="nofollow">@seanmor5</a> JAX has a way now to cache compilations, so we may be able to optimize the cold load time as well <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p><aside class="onebox githubgist" data-onebox-src="https://gist.github.com/shawwn/16d89ea5121d10214459238225453b13#">
  <header class="source">

      <a href="https://gist.github.com/shawwn/16d89ea5121d10214459238225453b13#" target="_blank" rel="noopener nofollow">gist.github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://gist.github.com/shawwn/16d89ea5121d10214459238225453b13#" target="_blank" rel="noopener nofollow">https://gist.github.com/shawwn/16d89ea5121d10214459238225453b13#</a></h4>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310706" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/18">Post #17</a>
	                </div>
	            </div>
              <div id="likers-container-310706" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310706"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #17"></div>
  </section>
</div>
    <div class="postbit" id="310713" data-post-id="310713">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Hisako1337" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Hisako1337/120/1907_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Hisako1337
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>just out of curiosity: did you manage to run a 7B model (like mistral) via bumblebee right within a phx application and chat with it? I mean, “just like” a dependency? I am still a bit unsure if this stuff will “just work” when deploying to fly as usual</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310713" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/19">Post #18</a>
	                </div>
	            </div>
              <div id="likers-container-310713" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310713"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #18"></div>
  </section>
</div>
    <div class="postbit" id="310719" data-post-id="310719">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="chrismccord" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/chrismccord/120/24233_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  chrismccord
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Phoenix</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Yes. This is not using FLAME, but here’s llama2-13b running Elixir/bumblebee on Fly GPU <a href="https://gpubee.fly.dev/" rel="nofollow">https://gpubee.fly.dev/</a></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310719" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/20">Post #19</a>
	                </div>
	            </div>
              <div id="likers-container-310719" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310719"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #19"></div>
  </section>
</div>
    <div class="postbit" id="310764" data-post-id="310764">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="niccolox" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/niccolox/120/2601_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  niccolox
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="adw632" data-post="14" data-topic="60209">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/a/9d8465/48.png" class="avatar"> adw632:</div>
<blockquote>
<p>Where FLAME is different is that you can delegate <strong>specialised</strong> work to a pool that scales to zero without a lot of complexity so your primary nodes continue serving the non-specialised work.</p>
</blockquote>
</aside>
<p>yep, and its actually a big deal all by itself</p>
<p>thanks for the notes</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310764" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/21">Post #20</a>
	                </div>
	            </div>
              <div id="likers-container-310764" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310764"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #20"></div>
  </section>
</div>
    <div class="postbit" id="310825" data-post-id="310825">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="marciol" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/marciol/120/4223_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  marciol
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Some details about implementation which may not be shared in the official blog post</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://threadreaderapp.com/thread/1732783111333625905.html">
  <header class="source">
      <img src="https://threadreaderapp.com/favicon-32x32.png" class="site-icon" alt="" width="32" height="32">

      <a href="https://threadreaderapp.com/thread/1732783111333625905.html" target="_blank" rel="noopener nofollow ugc">threadreaderapp.com</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/361;"><img src="https://threadreaderapp.com/images/screenshots/thread/1732783111333625905.jpg" class="thumbnail" alt="" width="690" height="361"></div>

<h3><a href="https://threadreaderapp.com/thread/1732783111333625905.html" target="_blank" rel="noopener nofollow ugc">Thread by @chris_mccord on Thread Reader App</a></h3>

  <p>@chris_mccord: I should have mentioned it in the post, but shout out to @peregrine for coming up with the FLAME concept together. Also @josevalim helping work through the impl details, as always ❤️ 🧵 1/ Sending...…</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="310825" data-batch-url="/posts/batch_likers">
                        4
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/flame-rethinking-serverless/60209/22">Post #21</a>
	                </div>
	            </div>
              <div id="likers-container-310825" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="310825"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #21"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <a class="load-more-button" data-turbo-stream="true" href="/topics/60209/load_more?page=3">Load more posts (6 remaining)</a>
</div></template></turbo-stream>