<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="211887" data-post-id="211887">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="derek-zhou" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/120/19943_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  derek-zhou
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="zhangzhen" data-post="22" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/48/7537_2.png" class="avatar"> zhangzhen:</div>
<blockquote>
<p>Cowboy creates a process for each request. Why were also these handler processes brought down?</p>
</blockquote>
</aside>
<p>It is most likely several requests are handled at the same time and all got stuck at the hanged GenServer call. 5 seconds timeout happened and they all error out.</p>
<aside class="quote no-group" data-username="zhangzhen" data-post="22" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/48/7537_2.png" class="avatar"> zhangzhen:</div>
<blockquote>
<p>The GenServer process handles messages in its mailbox one by one, realizing the synchronization. Why were the new state not kept in the GenServer and also in the ETS table.</p>
</blockquote>
</aside>
<p>I find it very peculiar in the way you use the GenServer and ETS. you seems to be dynamically creating GenServer based on an external (webhook) originated batch_id? This is very vulnerable for malicious traffic.<br>
And all the GenServer store some data in a shared ETS table using the batch_id as the key? (is is a set, bag or something else?) Why do you need the ETS, you can just keep the batch in the GenServer,</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="211887" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cowboy-handlers-proably-exit-not-being-logged-or-reported-anywhere/39071/23">Post #22</a>
	                </div>
	            </div>
              <div id="likers-container-211887" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="211887"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #22"></div>
  </section>
</div>
    <div class="postbit" id="211963" data-post-id="211963">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="zhangzhen" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/120/7537_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  zhangzhen
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="derek-zhou" data-post="23" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/48/19943_2.png" class="avatar"> derek-zhou:</div>
<blockquote>
<p>It is most likely several requests are handled at the same time and all got stuck at the hanged GenServer call. 5 seconds timeout happened and they all error out.</p>
</blockquote>
</aside>
<p>The explanation sounds reasonable and I personally agree with it, but no debug info supports it.</p>
<aside class="quote no-group" data-username="derek-zhou" data-post="23" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/48/19943_2.png" class="avatar"> derek-zhou:</div>
<blockquote>
<p>you seems to be dynamically creating GenServer based on an external (webhook) originated batch_id?</p>
</blockquote>
</aside>
<p>yes. My implementation is based upon the talk Lance Halvorsen <a class="mention" href="/u/lance" rel="nofollow">@lance</a> gave at ElixirConf 2018: <a href="https://youtu.be/1IOobarmwQg" rel="noopener nofollow ugc">ElixirConf 2018 - Elixir at a Walking Pace - Lance Halvorsen</a>.</p>
<aside class="quote no-group" data-username="derek-zhou" data-post="23" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/48/19943_2.png" class="avatar"> derek-zhou:</div>
<blockquote>
<p>And all the GenServer store some data in a shared ETS table using the batch_id as the key? (is is a set, bag or something else?)</p>
</blockquote>
</aside>
<p>It is a set.</p>
<aside class="quote no-group" data-username="derek-zhou" data-post="23" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/48/19943_2.png" class="avatar"> derek-zhou:</div>
<blockquote>
<p>Why do you need the ETS, you can just keep the batch in the GenServer,</p>
</blockquote>
</aside>
<p>The reason is performance and scalability issues the GenServer approach might cause, which is described in Section 10.3  of the book Sasa Juric <a class="mention" href="/u/sasajuric" rel="nofollow">@sasajuric</a> wrote: <a href="https://www.manning.com/books/elixir-in-action-second-edition" rel="noopener nofollow ugc">Elixir in Action, Second Edition</a>.</p>
<p>I’m planning to use RabbitMQ to solve this issue. Once a Webhook event is received at Endpoint, it is put into a queue and 201 response is sent back to Nextflow runner immediately. The BatchServer takes the events from the queue and can process them at its own pace.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="211963" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cowboy-handlers-proably-exit-not-being-logged-or-reported-anywhere/39071/24">Post #23</a>
	                </div>
	            </div>
              <div id="likers-container-211963" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="211963"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #23"></div>
  </section>
</div>
    <div class="postbit" id="211966" data-post-id="211966">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="derek-zhou" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/120/19943_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  derek-zhou
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="zhangzhen" data-post="24" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/48/7537_2.png" class="avatar"> zhangzhen:</div>
<blockquote>
<p>yes. My implementation is based upon the talk Lance Halvorsen <a class="mention" href="/u/lance" rel="nofollow">@lance</a> gave at ElixirConf 2018: <a href="https://youtu.be/1IOobarmwQg" rel="noopener nofollow ugc">ElixirConf 2018 - Elixir at a Walking Pace - Lance Halvorsen</a>.</p>
</blockquote>
</aside>
<p>If I were you, I will pre-create all my GenServers with legal batch_id. Then I will just reject illegal batch_id from the web hook. This is both simpler and safer.</p>
<aside class="quote no-group" data-username="zhangzhen" data-post="24" data-topic="39071">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/48/7537_2.png" class="avatar"> zhangzhen:</div>
<blockquote>
<p>The reason is performance and scalability issues the GenServer approach might cause, which is described in Section 10.3 of the book Sasa Juric <a class="mention" href="/u/sasajuric" rel="nofollow">@sasajuric</a> wrote: <a href="https://www.manning.com/books/elixir-in-action-second-edition" rel="noopener nofollow ugc">Elixir in Action, Second Edition</a>.</p>
</blockquote>
</aside>
<p>Since you are accessing the data through a GenServer anyway, it is much faster to skip ETS altogether. I will not consider heavy weight tools until I am 100% sure my need cannot be served anyway else. it is the KISS doctrine.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="211966" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cowboy-handlers-proably-exit-not-being-logged-or-reported-anywhere/39071/25">Post #24</a>
	                </div>
	            </div>
              <div id="likers-container-211966" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="211966"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #24"></div>
  </section>
</div>
    <div class="postbit" id="213197" data-post-id="213197">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="zhangzhen" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/zhangzhen/120/7537_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  zhangzhen
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p><a class="mention" href="/u/hubertlepicki" rel="nofollow">@hubertlepicki</a><br>
The GET /api/batches/CT20_DS_downsample_1000X/items/RD2103883FFP request was sent to my web endpoint. The endpoint did nothing and no log information was printed out (the log level is set to :debug). I have no idea of what I do to address this problem.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="213197" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cowboy-handlers-proably-exit-not-being-logged-or-reported-anywhere/39071/26">Post #25</a>
	                </div>
	            </div>
              <div id="likers-container-213197" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="213197"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #25"></div>
  </section>
</div>
    <div class="postbit" id="223753" data-post-id="223753">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="prodis" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/prodis/120/12531_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  prodis
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I had a similar problem with long requests (more than 60 seconds to respond) in a Phoenix API. Basically the process was gone and no more information or logs about it.</p>
<p>At the end I figured out that the reason was Cowboy exiting from the current process and closing the HTTP connection. From Cowboy 2.0, there is an idle timeout configuration option, that defaults to 60.000 milliseconds: <a href="https://ninenines.eu/docs/en/cowboy/2.9/manual/cowboy_http/" class="inline-onebox" rel="noopener nofollow ugc">Nine Nines: cowboy_http(3)</a></p>
<p>You can setup this timeout via Plug.Cowboy <a href="https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#module-options" rel="noopener nofollow ugc"><code>protocol_options</code></a> if you are using Phoenix.</p>
<p>For example:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">config :api, MyAPIWeb.Endpoint,
  http: [
    port: 4000,
    protocol_options: [idle_timeout: :infinity]
  ],
  ...
</code></pre>
<p>For a more detailed explanation and examples, <a href="https://www.poeticoding.com/dealing-with-long-running-http-requests-and-timeouts-in-phoenix/" rel="noopener nofollow ugc">this blog post</a> helped me.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="223753" data-batch-url="/posts/batch_likers">
                        4
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cowboy-handlers-proably-exit-not-being-logged-or-reported-anywhere/39071/27">Post #26</a>
	                </div>
	            </div>
              <div id="likers-container-223753" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="223753"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>