<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="12192" data-post-id="12192">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Qqwy" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Qqwy/120/1349_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Qqwy
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>TypeCheck Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I think you might want to look at the <a href="https://github.com/Qqwy/tensor" rel="noopener nofollow ugc">Tensor</a> library that I made a while back, which allows for vectors, matrices and any higher-order tensors as well. It works with sparse maps, making it more efficient when it is only partially filled. It probably provides the functionality you want (a multitude of different ways of filtering, adding/removing rows or colums, flipping or rotating, fast access to a single element, row or column, easy ways to sort rows or columns, etc).</p>
<p>It is of course also available on <a href="https://hex.pm/packages/tensor" rel="nofollow">Hex.pm</a>.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="12192" data-batch-url="/posts/batch_likers">
                        6
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/12">Post #11</a>
	                </div>
	            </div>
              <div id="likers-container-12192" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="12192"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #11"></div>
  </section>
</div>
    <div class="postbit" id="12227" data-post-id="12227">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="dimitarvp" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/dimitarvp/120/38664_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  dimitarvp
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Your library looks pretty nice. I’d put some benchmark comparisons versus the vanilla (or Erlang) data structures for some bragging rights. <img src="https://forum.elixirforum.com/images/emoji/apple/wink.png?v=15" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> <code>benchwarmer</code> should do just fine.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="12227" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-12227" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="12227"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="12292" data-post-id="12292">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="JordiPolo" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  JordiPolo
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I’ve been experimenting with using map with a two dimensional key(tuple) for a map as originally suggested.</p>
<p>But it seems that operations will be much slower. I’ve just tried to reimplement a couple of them and bear in mind that my code sucks, but sucks as much doing lists then doing maps I guess and still lists seems to be faster than maps.<br>
This is the code, map implementation:<br>
<a href="https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/data_frame_map.ex" class="onebox" target="_blank" rel="noopener nofollow ugc">https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/data_frame_map.ex</a><br>
list implementation(has many more functions, do not care about most):<br>
<a href="https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/data_frame_map.ex" class="onebox" target="_blank" rel="noopener nofollow ugc">https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/data_frame_map.ex</a></p>
<p>The benchmark:<br>
<a href="https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/dataframe/benchmark.ex" class="onebox" target="_blank" rel="noopener nofollow ugc">https://github.com/JordiPolo/dataframe/blob/map_experiment/lib/dataframe/benchmark.ex</a></p>
<p>Results:<br>
transpose(list of lists)        314.23<br>
transpose(maps)                 123.85 - 2.54x slower<br>
tail(maps)                3303.23<br>
tail(list of lists)       1539.64 - 2.15x slower<br>
rows(list of lists)         85.83<br>
rows(maps)                   3.86 - 22.21x slower<br>
columns(list of lists)        198.51<br>
columns(maps)                 129.22 - 1.54x slower</p>
<p>I’m specially worried by the <code>columns</code> function as I know the list version was done with no love for performance whatsoever, still seems to outperform a map with tuples as keys.</p>
<p>I’ll try Tensor next, although I want to keep experimenting with map a little bit more.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="12292" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-12292" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="12292"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="12299" data-post-id="12299">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="bbense" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/bbense/120/1074_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  bbense
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>It’s very hard to make any reasonable comment without seeing the actual functions that do the transformations.</p>
<p>If performance is really the critical issue, then this is the kind of thing you write a NIF for. Elixir is simply not well suited to doing standard linear algebra type calculations as fast as the hardware can go. Even a naive implementation in C is going to beat the pants off of any Elixir implementation.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="12299" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-12299" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="12299"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="12301" data-post-id="12301">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="OvermindDL1" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/OvermindDL1/120/2677_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  OvermindDL1
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Did you use a tuple for the key in the map?  If so then I’d recommend trying to invert it and test again, I.E. put the, say, x coordinate as the map key then the value is another map of y values.  I’m curious to see how/if that would help.  <img src="https://forum.elixirforum.com/images/emoji/apple/slight_smile.png?v=15" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>(or y in the outer and x in the inner depending on access patterns)</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="12301" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-12301" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="12301"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="13542" data-post-id="13542">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Qqwy" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Qqwy/120/1349_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Qqwy
                  </h3>
		          </div>
						
			          <div class="user-title">
									<span>TypeCheck Core Team</span>
			          </div>
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>What is actually funny is that most matrix math libraries that work with ‘native’ code is much faster than C code, as this is a piece of software written in Fortran that has had a history of twenty years of efficiency improvements. And the Fortran compiler can also do more rigorous rewriting than the C compiler can.</p>
<p><a class="mention" href="/u/dimitarvp" rel="nofollow">@dimitarvp</a>: This is a great idea. Hopefully I can find some time in the near future to create a benchmark.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="13542" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-13542" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="13542"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #16"></div>
  </section>
</div>
    <div class="postbit" id="13620" data-post-id="13620">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="OvermindDL1" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/OvermindDL1/120/2677_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  OvermindDL1
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="Qqwy" data-post="17" data-topic="1928">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/qqwy/48/1349_2.png" class="avatar"> Qqwy:</div>
<blockquote>
<p>What is actually funny is that most matrix math libraries that work with ‘native’ code is much faster than C code, as this is a piece of software written in Fortran that has had a history of twenty years of efficiency improvements. And the Fortran compiler can also do more rigorous rewriting than the C compiler can.</p>
</blockquote>
</aside>
<p>That is because Fortran has no aliasing constructs, which allow the compiler to optimize better.  In C++ this can be forced, but generally only through template work.  Rust on the other hand has similar no aliasing constructs as Fortran, hence why it can beat C++ in some benchmarks too.  Though for Matrix work some C++ libraries like Eigen, which use heavy template magic, end up as some of the fastest executable code regardless, and are wonderfully easy to use, but hard to write, thankfully it is already written for you.  ^.^</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="13620" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/performant-data-structure-for-a-table-think-2d-array-or-list-of-lists/1928/18">Post #17</a>
	                </div>
	            </div>
              <div id="likers-container-13620" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="13620"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>