<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="198533" data-post-id="198533">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p><em>Please pardon the slow reply, I’ve been on vacation for the past few weeks</em></p>
<p>This is entirely doable with a single worker through the use of <code>:snooze</code>. Here’s a pseudo example:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">defmodule MyApp.SequentialWorker do
  use Oban.Worker, queue: :some_queue

  import Ecto.Query, only: [where: 3]

  @impl Worker
  def perform(%Job{args: args}) do
    if executable_for_user?(args) do
      # do normal stuff
    else
      # snooze for 60 seconds, make it as short as necessary
      {:snooze, 60}
    end
  end

  defp executable_for_user?(%{"user_id" =&gt; user_id}) do
    Job
    |&gt; where(j, j.worker == Worker.to_string(__MODULE__))
    |&gt; where(j, j.queue == "some_queue")
    |&gt; where(j, j.state == "executing")
    |&gt; where(j, fragment("? @&gt; ?", j.args, ^%{user_id: user_id})
    |&gt; MyApp.Repo.exists?()
  end
end
</code></pre>
<p>By checking whether there is another job executing you can force global concurrency. In this case the worker checks for other <code>executing</code> workers with the same <code>user_id</code>, though that is arbitrary and you could scope to anything you like.</p>
<p>It isn’t as optimized as running multiple queues with global concurrency, but it will work right now and is accurate.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="198533" data-batch-url="/posts/batch_likers">
                        5
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/9">Post #8</a>
	                </div>
	            </div>
              <div id="likers-container-198533" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="198533"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #8"></div>
  </section>
</div>
    <div class="postbit" id="198162" data-post-id="198162">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="chasers" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/chasers/120/14188_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  chasers
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="carterbryden" data-post="7" data-topic="36312">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/carterbryden/48/36733_2.png" class="avatar"> carterbryden:</div>
<blockquote>
<p>Tagging it that way might work, although it doesn’t seem like it would necessarily guarantee that the jobs would always be done in order or limited to 1 concurrency, just that they mostly would, which might be an issue in my use case</p>
</blockquote>
</aside>
<p>You could limit concurrency by having the worker check your server genserver if a job was currently running. If a job was running, then fail the next oban job and update the <code>scheduled_in</code> or <code>scheduled_at</code>.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="198162" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/8">Post #7</a>
	                </div>
	            </div>
              <div id="likers-container-198162" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="198162"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #7"></div>
  </section>
</div>
    <div class="postbit" id="198151" data-post-id="198151">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="carterbryden" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/carterbryden/120/36733_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  carterbryden
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Thanks for the ideas. Tagging it that way might work, although it doesn’t seem like it would necessarily guarantee that the jobs would always be done in order or limited to 1 concurrency, just that they mostly would, which might be an issue in my use case.</p>
<p>The separate table of jobs sounds like it would work, I was just hoping I could throw it all at oban and let it sort everything out. Sounds like this use case is a bit out of scope for Oban as is, so a hybrid approach like that might be in order. Thanks!</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="198151" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/7">Post #6</a>
	                </div>
	            </div>
              <div id="likers-container-198151" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="198151"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #6"></div>
  </section>
</div>
    <div class="postbit" id="198066" data-post-id="198066">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="chasers" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/chasers/120/14188_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  chasers
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Tag the job with a server ID, when the server is done with current job, get the next job by server id tag and schedule it to run now?</p>
<p>Or just keep a separate table of jobs to schedule by server id, and when they’re done with their current job pull from that and put into Oban?</p>
<p>You can still have a process per server managing that connection / listening for when commands are done etc.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="198066" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/6">Post #5</a>
	                </div>
	            </div>
              <div id="likers-container-198066" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="198066"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #5"></div>
  </section>
</div>
    <div class="postbit" id="197961" data-post-id="197961">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="carterbryden" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/carterbryden/120/36733_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  carterbryden
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Thanks for replying! That makes sense, so then having N queues means that it’ll need to query N times per period (second?) To check if there are jobs for that queue?</p>
<p>Are there any other ways you’d recommend solving this particular problem?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="197961" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/5">Post #4</a>
	                </div>
	            </div>
              <div id="likers-container-197961" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="197961"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #4"></div>
  </section>
</div>
    <div class="postbit" id="197960" data-post-id="197960">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Oban, as it works currently, uses per-queue polling for scheduled jobs. That is the biggest issue with running a lot of queues, where “a lot” depends on your hardware but is about 50+ queues.</p>
<p>If you don’t need scheduled jobs, or you don’t need per-second resolution, then you can start queues with a higher <code>poll_interval</code> and eliminate a lot of db load.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="197960" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/4">Post #3</a>
	                </div>
	            </div>
              <div id="likers-container-197960" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="197960"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #3"></div>
  </section>
</div>
    <div class="postbit" id="197951" data-post-id="197951">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="carterbryden" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/carterbryden/120/36733_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  carterbryden
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Sorry, I’m not sure I understand how uniqueness would help in this case. I thought that unique jobs prevented a duplicate of the same job from being created. Do they let me limit concurrency for a set of jobs - in this instance, all jobs connecting to the same server?</p>
<p>I suppose I could have a queue for ALL server connection tasks, and limit it to 1 concurrency, but I only need to limit concurrency for tasks connecting to the same server. Jobs connecting to server A shouldn’t have to wait for jobs connecting to server B to finish (or vice versa).</p>
<p>Thanks for the reply!</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="197951" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/3">Post #2</a>
	                </div>
	            </div>
              <div id="likers-container-197951" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="197951"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #2"></div>
  </section>
</div>
    <div class="postbit" id="197947" data-post-id="197947">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="fuelen" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/fuelen/120/9870_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  fuelen
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>AFAIK from discussion in Slack it wasn’t designed to have a lot of queues.<br>
Oban has an ability to set uniqueness based on arguments of the worker. I didn’t use this feature, but it should help for your use case.<br>
When I needed similar functionality, Oban didn’t have uniqueness by argument and I solved it by using combination of static queues with 1 worker and <code>:erlang.phash/2</code>, so <code>server1</code> always bind to <code>queue3</code>, <code>server2</code> to <code>queue5</code> and so on.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="197947" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/oban-having-many-dynamic-queues-okay-or-a-bad-idea/36312/2">Post #1</a>
	                </div>
	            </div>
              <div id="likers-container-197947" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="197947"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-first-post cat-first-post" title="First post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>