<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="190567" data-post-id="190567">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lucaong" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/120/21794_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lucaong
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="lud" data-post="80" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/48/14382_2.png" class="avatar"> lud:</div>
<blockquote>
<p>Like “I want all keys that are a 3-tuple where the first element is <code>__MODULE__</code> , the second element is <code>:some_key</code> , and with any 3rd element”</p>
</blockquote>
</aside>
<p>Yes, one can use <code>min_key: {__MODULE__, :some_key}, max_key: {__MODULE__, :some_key, 0, 0}</code>, using exclusive ranges. Note that this is not perfect, as it would for example include a tuple like <code>{__MODULE__, "xxx"}</code> or <code>{__MODULE__, :some_key, 0, -1}</code>.</p>
<p>That said, for practical cases it might be specific enough, when you know there won’t be any 2- or 4-tuple key of that kind. You can also use a filter function on top of min/max key to filter out anything that is not a 3-tuple.</p>
<p>If you have more limits on the third element, one can do better than that, leveraging on the <a href="https://hexdocs.pm/elixir/operators.html#term-ordering" rel="noopener nofollow ugc">term ordering</a>.</p>
<p>What would be really useful would be to have a “minus infinite”. The nice thing about it, is that it would be smaller than any term. That way, one could change the above to  <code>min_key: {__MODULE__, :some_key, minus_infinite}, max_key: {__MODULE__, :some_key, 0, minus_infinite}</code>, and match only the tuples you describe. As far as I know though, there is nothing like that in Erlang/Elixir.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="190567" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/83">Post #82</a>
	                </div>
	            </div>
              <div id="likers-container-190567" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="190567"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #82"></div>
  </section>
</div>
    <div class="postbit" id="190569" data-post-id="190569">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lud" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/120/14382_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lud
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Well the problem is that it would also return <code>{__MODULE__, :other_scope, 123}</code> since tuples are compared by size first.</p>
<p>So, yes, I can then use a function filter, but what do you think about implementing a <code>:key_filter</code> in the select options ? That would work just like min/max, without loading unnecessary data from the disk.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="190569" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/84">Post #83</a>
	                </div>
	            </div>
              <div id="likers-container-190569" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="190569"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #83"></div>
  </section>
</div>
    <div class="postbit" id="190608" data-post-id="190608">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lucaong" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/120/21794_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lucaong
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hi <a class="mention" href="/u/lud" rel="nofollow">@lud</a></p>
<aside class="quote no-group" data-username="lud" data-post="84" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/48/14382_2.png" class="avatar"> lud:</div>
<blockquote>
<p>what do you think about implementing a <code>:key_filter</code> in the select options ? That would work just like min/max, without loading unnecessary data from the disk.</p>
</blockquote>
</aside>
<p>Such a <code>key_filter</code> would have to load the keys, but it could skip loading the value associated to them, so it might be a good idea indeed. Note though, that <code>min_key</code>/<code>max_key</code> are more performant not so much because they avoid loading the data, but primarily because they avoid enumerating all entries, and make efficient use of the BTree instead. Therefore, a <code>key_filter</code> would be more or less as performant as a normal filter, unless one is working with large data.</p>
<p>In summary, and back to the question of selecting tuple keys, the best way is:</p>
<ul>
<li>Specify <code>min_key</code>/<code>max_key</code>, making use of the term order, to filter the range of keys as accurately as possible. This is the most performant way, and often enough.</li>
<li>If that leaves some unwanted keys in the range, use <code>filter</code>/<code>take_while</code>/<code>drop_while</code> functions to filter them out. Using <em>only</em> the pipe functions would enumerate all keys, which is not performant, but using them on top of <code>min_key</code>/<code>max_key</code> is a good approach.</li>
</ul>
<p>I might actually add a <code>key_filter</code> function, let me evaluate the trade off between a more complex API and the performance benefit.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="190608" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/85">Post #84</a>
	                </div>
	            </div>
              <div id="likers-container-190608" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="190608"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #84"></div>
  </section>
</div>
    <div class="postbit" id="190610" data-post-id="190610">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lud" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/120/14382_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lud
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<blockquote>
<p>I might actually add a <code>key_filter</code> function, let me evaluate the trade off between a more complex API and the performance benefit.</p>
</blockquote>
<p>Nice, thank you.</p>
<p>Looking at the state of the process I thought that all keys were kept in memory. But I had only a tiny number of keys.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="190610" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/86">Post #85</a>
	                </div>
	            </div>
              <div id="likers-container-190610" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="190610"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #85"></div>
  </section>
</div>
    <div class="postbit" id="190612" data-post-id="190612">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lucaong" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/120/21794_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lucaong
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="lud" data-post="86" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lud/48/14382_2.png" class="avatar"> lud:</div>
<blockquote>
<p>Looking at the state of the process I thought that all keys were kept in memory. But I had only a tiny number of keys.</p>
</blockquote>
</aside>
<p>As you discovered, keys are not kept in memory, apart from the root of the BTree <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> Your observation also gives me the chance to briefly explain why that’s the case.</p>
<p>The design goal of <code>CubDB</code> is to be a good embedded database that can run everywhere where Elixir/Erlang can run. This includes, for example, embedded devices (which is the primary field where I personally use <code>CubDB</code>). Therefore, <code>CubDB</code> in its first production release attempts to be as efficient as possible in the use of resources without any form of caching, leaving the trade-off between memory usage and speed to the user. On many use-cases, such as data logging and data stores on tiny embedded linux devices, this is an advantage.</p>
<p>Future releases might introduce optional caching, but it will always be an optional layer. Same goes with compression: <code>CubDB</code> initially implemented compression, but that was removed when it turned out that it’s a trade-off that is better left to the user.</p>
<p>In sum, <code>CubDB</code> strives to implement a small, very reliable, usable, and performant core in pure Elixir with minimal overhead. As a future step, optimizations that introduce trade-offs might be added as options on top.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="190612" data-batch-url="/posts/batch_likers">
                        9
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/87">Post #86</a>
	                </div>
	            </div>
              <div id="likers-container-190612" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="190612"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #86"></div>
  </section>
</div>
    <div class="postbit" id="219929" data-post-id="219929">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Sebb" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Sebb/120/21818_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Sebb
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I’m looking for a place to store device-config in an embedded device (nerves).<br>
Main concern is data-corruption in case of a power loss (which will happen very often).<br>
CubDB seems to be a good fit. (before I was thinking SQLite but it would be overkill).<br>
Did you ever stress-test CubDB with frequent power cuts while writing?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="219929" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/88">Post #87</a>
	                </div>
	            </div>
              <div id="likers-container-219929" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="219929"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #87"></div>
  </section>
</div>
    <div class="postbit" id="219964" data-post-id="219964">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lucaong" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/120/21794_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lucaong
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hi <a class="mention" href="/u/sebb" rel="nofollow">@Sebb</a> ,<br>
that’s exactly the primary use case that I designed CubDB around. Power failures won’t corrupt data or break atomicity of transactions. If you set <code>auto_file_sync</code> to <code>false</code>, you might loose some recently written transaction (but still not break atomicity of the ones that are committed). If you set it to <code>true</code> (the default in <code>1.0.0</code>), successful writes are always persistent.</p>
<p>I use CubDB in production for persisting configuration and data logging on embedded devices since years now, and never experienced a data corruption issue caused by a power failure. I do perform stress tests when releasing notable changes, but I am also looking at options for a more systematic approach (such as a raspberry pi with a system to randomly cut power and a test script reporting inconsistencies).</p>
<p>One warning from experience: supplying too low voltage to a RPi <em>can</em> produce data corruption on the file system, of the kind that neither CubDB nor SQLite can prevent, like random bit flipping. Very long USB cables (over 1.5m) with poor quality power supply used to power a RPi 4 are sometimes enough to cause such a voltage drop.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="219964" data-batch-url="/posts/batch_likers">
                        6
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/89">Post #88</a>
	                </div>
	            </div>
              <div id="likers-container-219964" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="219964"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #88"></div>
  </section>
</div>
    <div class="postbit" id="220005" data-post-id="220005">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Exadra37" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Exadra37/120/9183_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Exadra37
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="lucaong" data-post="89" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/48/21794_2.png" class="avatar"> lucaong:</div>
<blockquote>
<p>If you set it to <code>true</code> (the default in <code>1.0.0</code>), successful writes are always persistent.</p>
</blockquote>
</aside>
<p>I don’t know your implementation, but the BEAM as a delay to write to the disk of around 2 seconds, even when it returns that its persisted, its not indeed persisted:</p>
<aside class="quote quote-modified" data-post="7" data-topic="19319">
  <div class="title">
    <div class="quote-controls"></div>
    <img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/exadra37/48/9183_2.png" class="avatar">
    <div class="quote-title__text-content">
      <a href="https://forum.elixirforum.com/t/unexpected-behaviour-with-mnesia-not-actually-durable/19319/7" rel="nofollow">Unexpected behaviour with mnesia - not actually durable?</a> <a class="badge-category__wrapper " href="/c/questions-help/questions/53" rel="nofollow"><span data-category-id="53" style="--category-badge-color: #C14BFB; --category-badge-text-color: #000000; --parent-category-badge-color: #C14BFB;" data-parent-category-id="171" data-drop-close="true" class="badge-category --style-square --has-parent" title="Elixir Questions / Help"><span class="badge-category__name">Questions</span></span></a>
    </div>
  </div>
  <blockquote>
    The issue It’s not related with any blocking in Erlang per se… As I mentioned in the post I <a href="https://forum.elixirforum.com/t/mnesia-with-memento-lost-all-records-for-all-tables/36221/38" rel="nofollow">linked</a> previously,  it’s because the BEAM has a default of 2 seconds to write data to the disk when the file is open with the defaults for delayed_write, as per <a href="https://erlang.org/doc/man/file.html#open-2" rel="nofollow">docs </a>: 

delayed_write 
The same as {delayed_write, Size, Delay} with reasonable default values for Size and Delay (roughly some 64 KB, 2 seconds). 

You can confirm this with a script that writes directly to the file system: 
defmodule FileIO do…
  </blockquote>
</aside>

<p>You can replicate it with the script I provide in the above post that uses directly <code>File</code> to create a file descriptor and then writes to the disk with <code>IO.binwrite</code> and you will confirm that persistence is only guaranteed after 2 seconds. This can be configured to be zero seconds, but then will affect performance,</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="220005" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/90">Post #89</a>
	                </div>
	            </div>
              <div id="likers-container-220005" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="220005"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #89"></div>
  </section>
</div>
    <div class="postbit" id="220039" data-post-id="220039">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Sebb" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Sebb/120/21818_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Sebb
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="lucaong" data-post="89" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/48/21794_2.png" class="avatar"> lucaong:</div>
<blockquote>
<p>I use CubDB in production for persisting configuration and data logging on embedded devices since years now, and never experienced a data corruption issue caused by a power failure.</p>
</blockquote>
</aside>
<p>It is very unlikely to encounter a data loss even with a naive implementation, because you have to get the power-fail while you are writing. And normally write times are short and power failures are rare.</p>
<aside class="quote no-group" data-username="lucaong" data-post="89" data-topic="23397">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/48/21794_2.png" class="avatar"> lucaong:</div>
<blockquote>
<p>I am also looking at options for a more systematic approach</p>
</blockquote>
</aside>
<p>I propose the following:</p>
<p>Device booting into an Application running CubDB writing lots of data to the disk. Before the writing starts, the App also sends a command to an Relais to cut the power. The Problem is, that it takes some time from sending the command to the actual power fail, so we need to find a way to be sure we are writing while power-loss.<br>
The Relais will automatically enable power after some time, so the process starts again.</p>
<p>I have all the hardware at hand to do that, and I’m willing to do it.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="220039" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/91">Post #90</a>
	                </div>
	            </div>
              <div id="likers-container-220039" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="220039"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #90"></div>
  </section>
</div>
    <div class="postbit" id="220043" data-post-id="220043">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="lucaong" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/lucaong/120/21794_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  lucaong
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hi <a class="mention" href="/u/sebb" rel="nofollow">@Sebb</a> , what you describe looks exactly like what I wanted to implement for automated stress testing.</p>
<p>Let me also explain how atomicity and corruption protection in <code>CubDB</code> works internally. <code>CubDB</code> data structure is an append-only B+tree saved in a single file. Every time an entry is written, the B+tree is <strong>not</strong> modified in place. Instead, exactly like immutable data structures, the new leaf and the path to the new B+tree root is appended to the data file, referencing the rest of the tree that did not change. The last thing that is appended to the file is a transaction header, containing a pointer to the new B+tree root and some meta data such as the size. The header is always written at an offset multiple of the page size.</p>
<p>Upon start, <code>CubDB</code> searches the data file backwards for the most recent “sane” header. If a header is corrupted, incomplete, or completely missing, <code>CubDB</code> simply skips backwards looking for a previous one.</p>
<p>Compaction operations take care of cleaning up entries that are not reachable anymore, similarly to garbage collection in memory. Compaction runs concurrently, creating a new compacted copy of the data file, and cleaning up the old one only when done, and can be stopped at any time without corruption.</p>
<p>The “immutable” and append-only nature of the data structure used by <code>CubDB</code> also makes it possible achieve isolation and to read concurrently to writes: readers simply traverse the most recent B+tree at the time the read started, as writers cannot mutate it in place.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="220043" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/cubdb-a-pure-elixir-embedded-key-value-database/23397/92">Post #91</a>
	                </div>
	            </div>
              <div id="likers-container-220043" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="220043"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #91"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <a class="load-more-button" data-turbo-stream="true" href="/topics/23397/load_more?page=10">Load more posts (34 remaining)</a>
</div></template></turbo-stream>