<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="123360" data-post-id="123360">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="OvermindDL1" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/OvermindDL1/120/2677_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  OvermindDL1
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="Jaypee" data-post="42" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>The fact that I posted my request on Elixirforum and nowhere else is a bit related to the fact that my benchmarks focus on Elixir <strong>only</strong> as it is what I have to evaluate: So Rust, Go, OCaml or others (surely) fine and powerfull languages are, for the moment at least, out of my scope,</p>
</blockquote>
</aside>
<p>Don’t forget that Elixir is a fantastic ‘glue’ language between languages as well, it’s Ports and Nif’s are not to be discounted.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="Jaypee" data-post="42" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>As I mentionned in the original post, my tinies “files exchange” servers will be located on several computers/VM. This is indeed one of the main reasons that leaded me to test the Elixir/Erlang/OTP system as its distributed aspect seems to be praised everywhere,</p>
<p>I’m only querying practical and clear advices to lead an “Elixir n00b” like me on the jungle of Elixir concepts (GenServers, GenStages, Supervisors, Tasks, Agents and the like) in order to achieve my goals while trying as much as possible to avoid the waste of time generated by exploring some unefficient dead ends (NB: this experimentation is done by me in addition to my regular job, partly outside of my working hours). Just like did Joe Armstrong or Scott Thompson in this thread, for instance…<img src="https://forum.elixirforum.com/images/emoji/apple/+1.png?v=15" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"></p>
</blockquote>
</aside>
<p>Well the first decision is what protocol to use, like do you want to use a standard protocol like HTTPS or SFTP, or do you want some custom protocol inside an SSL connection?  If you just want something secure and fast then the beam has a built-in SFTP module, the server documentation of which is at:  <a href="http://erlang.org/doc/man/ssh_sftpd.html" class="inline-onebox" rel="nofollow">ssh_sftpd — OTP 29.0.2 (ssh 6.0.1)</a>  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>If using https then cowboy/plug/phoenix (depending on what extra functionality and features you want, and what kind of clients will be pulling it) will work well.</p>
<p>Though more information and details can be given if this can be filled out.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<ol>
<li>What client(s) do you want pulling from it (something prebuilt and if so what, or will you make it yourself)?</li>
<li>What protocol(s) do you want to use (something pre-defined, or making something custom)?</li>
<li>What do you want the BEAM server to do that something like an SFTP server itself cannot do?</li>
<li>Etc… anything else that might be relevant.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></li>
</ol> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123360" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/45">Post #44</a>
	                </div>
	            </div>
              <div id="likers-container-123360" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123360"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #44"></div>
  </section>
</div>
    <div class="postbit" id="123467" data-post-id="123467">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Jaypee" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Jaypee
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Fisrt, thanks for your help.</p>
<aside class="quote no-group" data-username="OvermindDL1" data-post="45" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/overminddl1/48/2677_2.png" class="avatar"> OvermindDL1:</div>
<blockquote>
<p>What client(s) do you want pulling from it (something prebuilt and if so what, or will you make it yourself)?</p>
</blockquote>
</aside>
<p>Will make it myself as the client will be (only) the server: BTW each server should be able to send and receive files and these files should only be transfered from a server (node ?) to another server (no others clients).</p>
<aside class="quote no-group" data-username="OvermindDL1" data-post="45" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/overminddl1/48/2677_2.png" class="avatar"> OvermindDL1:</div>
<blockquote>
<p>What protocol(s) do you want to use (something pre-defined, or making something custom)?</p>
</blockquote>
</aside>
<p>I’m not really firmly decided on that for the moment. My specs are that the files should be crypted during the transfer (so sftp would fit) but also that integrity should be verified at the end of each transfer (I don’t think that sftp achieves that) by some checksum or others means .<br>
A plus could be that theses servers should also resume partials transfers but this is optionnal…</p>
<aside class="quote no-group" data-username="OvermindDL1" data-post="45" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/overminddl1/48/2677_2.png" class="avatar"> OvermindDL1:</div>
<blockquote>
<p>What do you want the BEAM server to do that something like an SFTP server itself cannot do?</p>
</blockquote>
</aside>
<p>The integrity thing mentionned above ?</p>
<aside class="quote no-group" data-username="OvermindDL1" data-post="45" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/overminddl1/48/2677_2.png" class="avatar"> OvermindDL1:</div>
<blockquote>
<p>Etc… anything else that might be relevant. <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
</blockquote>
</aside>
<p>I’d like this server to be the most efficient possible (big files transfers and simultaneous transfers of numerous small files)  as we will do a benchmark and compare the Elixir one to some others same servers made in C++ by another team.<br>
(I actually try to promote Elixir inside my company ! <img src="https://forum.elixirforum.com/images/emoji/apple/wink.png?v=15" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> )</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123467" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/46">Post #45</a>
	                </div>
	            </div>
              <div id="likers-container-123467" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123467"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #45"></div>
  </section>
</div>
    <div class="postbit" id="123486" data-post-id="123486">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="blatyo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/blatyo/120/4554_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  blatyo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Conduit Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Surprised no one’s mentioned BitTorrent yet. By definition, a way to efficiently distribute files across peers. Found this slide deck about the details: <a href="http://www.erlang-factory.com/static/upload/media/1474809327462227martingausbyimplementingbittorrentinelixireuc2016.pdf" rel="noopener nofollow ugc">http://www.erlang-factory.com/static/upload/media/1474809327462227martingausbyimplementingbittorrentinelixireuc2016.pdf</a></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123486" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/47">Post #46</a>
	                </div>
	            </div>
              <div id="likers-container-123486" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123486"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #46"></div>
  </section>
</div>
    <div class="postbit" id="123490" data-post-id="123490">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Jaypee" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Jaypee
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Perhaps because implementing a whole P2P protocol such as BitTorrent seems to be, as stated in the slides, a bit of, an “hard challenge” (at least for me) to achieve the goals that, at first glance, didn’t seem so complicated to me (but I’m certainly wrong indeed): securely transfering  files between (only) two servers/nodes with integrity checks ?<br>
Anyway, thanks for your advice.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123490" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/48">Post #47</a>
	                </div>
	            </div>
              <div id="likers-container-123490" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123490"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #47"></div>
  </section>
</div>
    <div class="postbit" id="123498" data-post-id="123498">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="blatyo" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/blatyo/120/4554_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  blatyo
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>Conduit Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>If it’s just two servers, yea, BitTorrent is overkill.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123498" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/49">Post #48</a>
	                </div>
	            </div>
              <div id="likers-container-123498" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123498"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #48"></div>
  </section>
</div>
    <div class="postbit" id="123528" data-post-id="123528">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="OvermindDL1" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/OvermindDL1/120/2677_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  OvermindDL1
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="Jaypee" data-post="46" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>Will make it myself as the client will be (only) the server: BTW each server should be able to send and receive files and these files should only be transfered from a server (node ?) to another server (no others clients).</p>
</blockquote>
</aside>
<p>If it’s purely a BEAM mesh network then you can transfer a file as easily as sending a message, you can even access a file on a remote node from another node directly.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="Jaypee" data-post="46" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>but also that integrity should be verified at the end of each transfer (I don’t think that sftp achieves that)</p>
</blockquote>
</aside>
<p>Well there is a <code>check-file</code> extension in the sftp protocol for that but it’s an optional extension so I don’t know if the system supports it.  Might be good to emulate it by just having each end expose a md5/sha1/whatever prefixed file of the same name that when accessed will hash the file and return the hash as that would be pretty simple to set up.</p>
<aside class="quote no-group" data-username="Jaypee" data-post="46" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>The integrity thing mentionned above ?</p>
</blockquote>
</aside>
<p>You can pack a CRC/hash along with it, just send something like a tuple or a map with fields for the hash, filename, and content.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="Jaypee" data-post="46" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>I’d like this server to be the most efficient possible (big files transfers and simultaneous transfers of numerous small files) as we will do a benchmark and compare the Elixir one to some others same servers made in C++ by another team.<br>
(I actually try to promote Elixir inside my company ! <img src="https://forum.elixirforum.com/images/emoji/apple/wink.png?v=15" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> )</p>
</blockquote>
</aside>
<p>I/O is what the beam does very well, <em>especially</em> with <code>sendfile</code> calls too if you don’t mind setting up transfer-specific sockets between the servers, but even the distribution network is fine for that until it gets too loaded down and messages get delayed.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="123528" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/50">Post #49</a>
	                </div>
	            </div>
              <div id="likers-container-123528" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="123528"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #49"></div>
  </section>
</div>
    <div class="postbit" id="131903" data-post-id="131903">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Jaypee" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Jaypee
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Now I experiment two differents ways:</p>
<ul>
<li>
<p>The first one by using the <a href="https://github.com/mikejdorm/sftp_ex" rel="noopener nofollow ugc">Michael Dorman’s Sftp_ex library</a> to implement some kind of simplified SFTP client/server which seems to be ok for the moment,</p>
</li>
<li>
<p>The second one simply using the node mechanism</p>
</li>
</ul>
<p>For the “node” experiment, everything is ok when I use read and save file from one node to a remote one but I’m stuck when I try to stream a file from a node to the remote one (in case of huge files, streaming seems to be a better idea than read/save) as the struct of stream does not allow that.<br>
If someone has some smart idea to achieve this… <img src="https://forum.elixirforum.com/images/emoji/apple/roll_eyes.png?v=15" title=":roll_eyes:" class="emoji" alt=":roll_eyes:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="131903" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/51">Post #50</a>
	                </div>
	            </div>
              <div id="likers-container-131903" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="131903"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #50"></div>
  </section>
</div>
    <div class="postbit" id="132833" data-post-id="132833">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="OvermindDL1" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/OvermindDL1/120/2677_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  OvermindDL1
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="Jaypee" data-post="51" data-topic="21119">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/f6c823/48.png" class="avatar"> Jaypee:</div>
<blockquote>
<p>For the “node” experiment, everything is ok when I use read and save file from one node to a remote one but I’m stuck when I try to stream a file from a node to the remote one (in case of huge files, streaming seems to be a better idea than read/save) as the struct of stream does not allow that.<br>
If someone has some smart idea to achieve this… <img src="https://forum.elixirforum.com/images/emoji/apple/roll_eyes.png?v=15" title=":roll_eyes:" class="emoji" alt=":roll_eyes:" loading="lazy" width="20" height="20"></p>
</blockquote>
</aside>
<p>For the stream you’ll need to send each result from the stream via a message to a receiver to stuff back into a receiving stream.  A simple hoist.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="132833" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/52">Post #51</a>
	                </div>
	            </div>
              <div id="likers-container-132833" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="132833"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #51"></div>
  </section>
</div>
    <div class="postbit" id="132974" data-post-id="132974">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Jaypee" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Jaypee
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Yes, I found that as, for instance, using the <a href="https://hexdocs.pm/elixir/Stream.html#resource/3" rel="noopener nofollow ugc">Stream.resource/3 function</a> (that’s what Michael Dorman does in his Sftp_ex library for streams BTW) but as I’m lazy as a Stream, I wondered if someone didn’t have already done that then cut and paste and so on… <img src="https://forum.elixirforum.com/images/emoji/apple/wink.png?v=15" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="https://forum.elixirforum.com/images/emoji/apple/innocent.png?v=15" title=":innocent:" class="emoji" alt=":innocent:" loading="lazy" width="20" height="20"><br>
But now as I give up the “node solution” (it seems that <a href="https://forum.elixirforum.com/t/is-this-a-correct-way-to-avoid-wasting-or-reaching-the-atom-limit/" rel="nofollow">the nodes and Erlang clusters are not to be used for my distributed files servers problem</a>) and I fall back on my SFTP solution, using… Sftp_ex !</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="132974" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/53">Post #52</a>
	                </div>
	            </div>
              <div id="likers-container-132974" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="132974"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #52"></div>
  </section>
</div>
    <div class="postbit" id="141166" data-post-id="141166">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Jaypee" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Jaypee
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Some news:<br>
Now my “SFTP Server/Client” in Elixir (nearly) works (except <a href="https://forum.elixirforum.com/t/resume-file-download-upload/22704" rel="nofollow">the “resume / reget” stuff</a> <img src="https://forum.elixirforum.com/images/emoji/apple/rage.png?v=15" title=":rage:" class="emoji" alt=":rage:" loading="lazy" width="20" height="20">,. The checksum is ok, with severam encoding choices, thanks to streaming, is ok):<br>
The server part is inspired (but not using as a dependency) by <a href="https://github.com/Codenaut/exsftpd" rel="noopener nofollow ugc"><strong>Exsftpd</strong> from Codenaut</a> and the Client part by <a href="https://github.com/mikejdorm/sftp_ex" rel="noopener nofollow ugc"><strong>Sftp_Ex</strong> from Mike Dorman</a> or <a href="https://github.com/i22-digitalagentur/sftp_client" rel="noopener nofollow ugc"><strong>SFTP Client</strong> from Tobias Casper </a> (I have two versions)<br>
Great Thanks to these peoples and to those who put me on the Elixir/Erlang tracks here and there ! <img src="https://forum.elixirforum.com/images/emoji/apple/+1.png?v=15" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"><img src="https://forum.elixirforum.com/images/emoji/apple/+1.png?v=15" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"><img src="https://forum.elixirforum.com/images/emoji/apple/+1.png?v=15" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"></p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="141166" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/secure-files-transfers-servers-in-elixir/21119/54">Post #53</a>
	                </div>
	            </div>
              <div id="likers-container-141166" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="141166"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #53"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <a class="load-more-button" data-turbo-stream="true" href="/topics/21119/load_more?page=4">Load more posts (8 remaining)</a>
</div></template></turbo-stream>