<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="355716" data-post-id="355716">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="begedin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/120/25357_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  begedin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Yeah, I get that.</p>
<p>What I’m saying here is, because multiple processes try to insert the job at about the same time (single digit ms appart) apparently, uniqueness isn’t guaranteed at insert time either?</p>
<p>I ran an experiment of spawning 1000 parallel tasks, each inserting the same job with the same arguments, and while not 1000 get inserted, it’s definitely in the range of several tens of them.</p>
<p>I guess i understand why that’s the case, but I’m wondering if I’m missing something.</p>
<p>The chain option might at least allow me they don’t execute at the same time, even if they are duplicate inserts, right?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="355716" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/12">Post #11</a>
	                </div>
	            </div>
              <div id="likers-container-355716" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="355716"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #11"></div>
  </section>
</div>
    <div class="postbit" id="355899" data-post-id="355899">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="begedin" data-post="12" data-topic="63700">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/48/25357_2.png" class="avatar"> begedin:</div>
<blockquote>
<p>What I’m saying here is, because multiple processes try to insert the job at about the same time (single digit ms appart) apparently, uniqueness isn’t guaranteed at insert time either?</p>
</blockquote>
</aside>
<p>Uniqueness in OSS, and Pro prior to v1.5, is best effort. It uses advisory locks and queries, but competing transactions can allow duplicates in from race conditions. That’s changed in Pro v1.5, which uses a unique index for enforcement, so it isn’t subject to race conditions.</p>
<aside class="quote no-group" data-username="begedin" data-post="12" data-topic="63700">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/48/25357_2.png" class="avatar"> begedin:</div>
<blockquote>
<p>The chain option might at least allow me they don’t execute at the same time, even if they are duplicate inserts, right?</p>
</blockquote>
</aside>
<p>Right; it would.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="355899" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-355899" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="355899"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="368729" data-post-id="368729">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="begedin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/120/25357_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  begedin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Sorry to sort of resurrect this. We managed to get past this by redoing our jobs to be idempotet even on duplication.</p>
<p>We’ve upgraded to 1.5 and made use of new uniqueness options, but after upgrading to annother project to 1.6, suddenly hit an insert bottleneck. We’d be able to get around it if we could make the on_conflict handling for jobs in the Smart engine optional, as in if we had an option to set it to :nothing here:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir"># Redacted
</code></pre>
<p>Any plans to do something like that?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="368729" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-368729" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="368729"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="368783" data-post-id="368783">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="begedin" data-post="14" data-topic="63700">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/48/25357_2.png" class="avatar"> begedin:</div>
<blockquote>
<p>We’d be able to get around it if we could make the on_conflict handling for jobs in the Smart engine optional, as in if we had an option to set it to :nothing here:</p>
</blockquote>
</aside>
<p>If the jobs aren’t marked <code>unique</code> then they won’t use <code>on_conflict</code>. That’s essentially setting it to <code>:nothing</code> <img src="https://forum.elixirforum.com/images/emoji/apple/slightly_smiling_face.png?v=15" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="368783" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-368783" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="368783"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="368806" data-post-id="368806">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="begedin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/120/25357_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  begedin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Wouldn’t <code>:nothing</code> just discard any subsequent inserts, if the job matching the uniqueness already exists?<br>
Meaning, we retain the first insert only, discarding all future ones.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="368806" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-368806" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="368806"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="368848" data-post-id="368848">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Sorry, perhaps I misunderstood. Retaining the original job is what it’s doing already, but while also marking the original job as having a conflict. That could be disabled as an optimization, conceivably, but it’s a little surprising to hear there are so many unique conflicts that it’s a performance issue.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="368848" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-368848" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="368848"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #16"></div>
  </section>
</div>
    <div class="postbit" id="368861" data-post-id="368861">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="begedin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/begedin/120/25357_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  begedin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Yeah, based on our numbers, it definitely looks like that would help. We might not have the most suitable use case for oban here, but it’s where we’re at <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="368861" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/18">Post #17</a>
	                </div>
	            </div>
              <div id="likers-container-368861" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="368861"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #17"></div>
  </section>
</div>
    <div class="postbit" id="390462" data-post-id="390462">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="apoorv-2204" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/apoorv-2204/120/25991_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  apoorv-2204
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I want to allow enqueuing of duplicate arg base jobs. But execution should happen once at at a time for given arg key.<br>
How to achieve that?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="390462" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/19">Post #18</a>
	                </div>
	            </div>
              <div id="likers-container-390462" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="390462"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #18"></div>
  </section>
</div>
    <div class="postbit" id="390512" data-post-id="390512">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sorentwo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sorentwo/120/37360_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sorentwo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Oban Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="apoorv-2204" data-post="19" data-topic="63700">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/apoorv-2204/48/25991_2.png" class="avatar"> apoorv-2204:</div>
<blockquote>
<p>But execution should happen once at at a time for given arg key.<br>
How to achieve that?</p>
</blockquote>
</aside>
<p>That’s a <a href="https://oban.hexdocs.pm/unique_jobs.html#uniqueness-vs-concurrency" rel="noopener nofollow ugc">concurrency issue, not uniqueness</a>.</p>
<p>Limiting concurrency by args is exactly what <a href="https://oban.pro/docs/pro/Oban.Pro.Engines.Smart.html#module-queue-partitioning" rel="noopener nofollow ugc">queue partitioning</a> in Pro accomplishes.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="390512" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/setting-up-proper-uniqueness-options-for-queue-in-oban/63700/20">Post #19</a>
	                </div>
	            </div>
              <div id="likers-container-390512" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="390512"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>