<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="304345" data-post-id="304345">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jonatanklosko" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/jonatanklosko/120/37940_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jonatanklosko
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Livebook</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<blockquote>
<p>with defn_options … why is one (ie. “lazy transfers”) in the keyword list as where “preallocate_params: true” is another parameter all together?</p>
</blockquote>
<p>All the top-level options (including <code>:preallocate_params</code> are handled by Bumblebee). The keyword list given as <code>:defn_options</code> is what we pass to <code>Nx.Defn.jit/compile</code>, where <code>compiler: EXLA</code> specifies the compiler and the remaining options are passed to the compiler.</p>
<p>So in other words <code>:preallocate_params</code> is for Bumblebee, <code>:lazy_transfers</code> is for the EXLA compiler : )</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="304345" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/12">Post #11</a>
	                </div>
	            </div>
              <div id="likers-container-304345" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="304345"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #11"></div>
  </section>
</div>
    <div class="postbit" id="304354" data-post-id="304354">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="nutheory" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/nutheory/120/22542_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  nutheory
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Thanks for catching that preallocate issue… and all your help.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="304354" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-304354" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="304354"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="305426" data-post-id="305426">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="vgrechin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/vgrechin/120/26214_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  vgrechin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hi <a class="mention" href="/u/jonatanklosko" rel="nofollow">@jonatanklosko</a>, I’m trying stable diffusion from livebook,<br>
but also encounter memory limit during smart cell execution.<br>
How to increase memory limit in livebook to run the model?<br>
</p><div class="lightbox-wrapper"><a class="lightbox" href="https://forum.elixirforum.com/uploads/default/original/3X/d/a/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d.jpeg" data-download-href="https://forum.elixirforum.com/uploads/default/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d" title="StableDiffusionMemoryLimit" rel="nofollow"><img src="https://forum.elixirforum.com/uploads/default/optimized/3X/d/a/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d_2_533x600.jpeg" alt="StableDiffusionMemoryLimit" data-base62-sha1="vdCwHheq7hqrdEFLTQsp5EcQ661" width="533" height="600" srcset="https://forum.elixirforum.com/uploads/default/optimized/3X/d/a/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d_2_533x600.jpeg, https://forum.elixirforum.com/uploads/default/optimized/3X/d/a/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d_2_799x900.jpeg 1.5x, https://forum.elixirforum.com/uploads/default/original/3X/d/a/dacd92c5ccbb5ce1f279ac71a5e58db1fdfe8e6d.jpeg 2x" data-dominant-color="F8F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">StableDiffusionMemoryLimit</span><span class="informations">953×1071 86.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><p></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305426" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-305426" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305426"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="305451" data-post-id="305451">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jonatanklosko" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/jonatanklosko/120/37940_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jonatanklosko
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Livebook</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p><a class="mention" href="/u/vgrechin" rel="nofollow">@vgrechin</a> to reduce memory usage you can set the number of images to 1. Another thing you can do is converting smart cell to code, and trying the changes in <a href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/2" rel="nofollow">this comment</a>.</p>
<p>How much RAM does your GPU have? If it’s 4GB then IIRC that’s not enough to run the Stable Diffusion.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305451" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-305451" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305451"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="305457" data-post-id="305457">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="vgrechin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/vgrechin/120/26214_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  vgrechin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I use Orin Nano 8Gb so wonder why 4Gb is limit. I’m able to run Stable Diffusion in a docker from <a href="https://github.com/dusty-nv/jetson-containers" class="inline-onebox" rel="noopener nofollow ugc">GitHub - dusty-nv/jetson-containers: Machine Learning Containers for NVIDIA Jetson and JetPack-L4T · GitHub</a>. That’s why I’m trying here in Elixir.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305457" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-305457" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305457"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="305459" data-post-id="305459">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jonatanklosko" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/jonatanklosko/120/37940_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jonatanklosko
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Livebook</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Where is this log coming from? Do you log it yourself, or is it just the default XLA behaviour on that device?</p>
<p>cc <a class="mention" href="/u/seanmor5" rel="nofollow">@seanmor5</a></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305459" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-305459" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305459"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #16"></div>
  </section>
</div>
    <div class="postbit" id="305464" data-post-id="305464">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="vgrechin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/vgrechin/120/26214_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  vgrechin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I run exla and livebook itself on the device, but access from a browser running on another machine. I’ve got access to both these boxes, so no problem to collect any logs. Not sure if it is the default XLA behaviour because I still use XLA here for nothing else but livebook</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305464" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/18">Post #17</a>
	                </div>
	            </div>
              <div id="likers-container-305464" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305464"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #17"></div>
  </section>
</div>
    <div class="postbit" id="305466" data-post-id="305466">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jonatanklosko" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/jonatanklosko/120/37940_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jonatanklosko
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Livebook</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Oh, I meant, where is the existing “Limit” log coming from : )</p>
<p>Also, is there a different way you can see GPU usage, and is actually 50% of the GPU free?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305466" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/19">Post #18</a>
	                </div>
	            </div>
              <div id="likers-container-305466" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305466"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #18"></div>
  </section>
</div>
    <div class="postbit" id="305468" data-post-id="305468">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="vgrechin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/vgrechin/120/26214_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  vgrechin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I usually track GPU loading in ‘jtop’ for other models, but it case of Stable Diffusion it doesn’t involve much GPU, just consumes memory, reducing its free space until the error.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305468" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/20">Post #19</a>
	                </div>
	            </div>
              <div id="likers-container-305468" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305468"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #19"></div>
  </section>
</div>
    <div class="postbit" id="305495" data-post-id="305495">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jonatanklosko" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/jonatanklosko/120/37940_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jonatanklosko
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Creator of Livebook</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Sorry, I meant GPU memory usage. I’m just trying to verify that when the “Limit” above is reached, there’s still GPU memory available. XLA actually preallocates memory, so you can just do <code>Nx.tensor([1])</code> and then see if 50% of the memory is still free.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="305495" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/livebook-cuda-12-2-xla-out-of-memory-at-11006mib/58054/21">Post #20</a>
	                </div>
	            </div>
              <div id="likers-container-305495" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="305495"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>