<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="360050" data-post-id="360050">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="tmbb" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  tmbb
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Since the Explorer team doesn’t seem to be interested in adding random variable sampling or advanced scientific functions, I have started my own project inspired by <a href="https://scipy.org/" rel="noopener nofollow ugc">SciPy</a> with the goal of adding efficient computation over multidimensional arrays in Elixir. The code is based on bindings to the Rust library <a href="https://docs.rs/ndarray/latest/ndarray/" rel="noopener nofollow ugc">ndarray</a> and supporting packages. I’m linking it here in case anyone finds it interesting.</p>
<p>Forum topic here: <a href="https://forum.elixirforum.com/t/sciex-scientific-programming-for-elixir-based-on-bindings-to-rusts-ndarray/70058" class="inline-onebox" rel="nofollow">SciEx - scientific programming for Elixir (based on bindings to rust's ndarray)</a></p>
<p>The following snippet shows an example of what’s possible (keep in mind this in the very early stages):</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex(4)&gt; a = SciEx.Random.draw_from_normal(0.0, 1.0, 10)
#SciEx.F64.Array1&lt;[0.980546308591242, -0.4124865578379766, -0.2678229458353892, 0.13895119159920113, -0.05895938590440134, 0.5463383504051798, -0.47139019152800626, 1.1432875805108502, -1.053279376911445, -0.4890897953025381], shape=[10], strides=[1], layout=CFcf (0xf), const ndim=1&gt;
iex(5)&gt; b = SciEx.Random.draw_from_normal(0.0, 1.0, 10)
#SciEx.F64.Array1&lt;[-0.6112979257481231, 0.04187616378275234, 0.024658043411414837, -0.7423086166282216, 2.5287898587121993, -1.5968438074887625, 0.3028680311098233, 1.1657682958696634, -1.3245816353940532, 1.1693983808728805], shape=[10], strides=[1], layout=CFcf (0xf), const ndim=1&gt;
iex(6)&gt; use SciEx.Operators
SciEx
iex(7)&gt; 0.5 * a + 0.7 * b  
#SciEx.F64.Array1&lt;[0.06236460627193485, -0.17692996427106167, -0.1166508425297042, -0.45014043584015456, 1.7406732081463385, -0.8446214900395438, -0.02368747398712684, 1.3876815973641894, -1.4538468332315597, 0.5740339689597472], shape=[10], strides=[1], layout=CFcf (0xf), const ndim=1&gt;
</code></pre> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="360050" data-batch-url="/posts/batch_likers">
                        1
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/12">Post #11</a>
	                </div>
	            </div>
              <div id="likers-container-360050" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="360050"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #11"></div>
  </section>
</div>
    <div class="postbit" id="369249" data-post-id="369249">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="billylanchantin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/120/28428_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  billylanchantin
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<h1><a name="p-369249-release-explorer-v0110-1" class="anchor" href="#p-369249-release-explorer-v0110-1" aria-label="Heading link" rel="nofollow"></a>[Release] Explorer - v0.11.0</h1>
<p><a href="https://github.com/elixir-explorer/explorer/releases/tag/v0.11.0" rel="noopener nofollow ugc">Version <code>0.11.0</code> has been released</a>!</p>
<p>This one is mostly minor improvements and bugfixes. One notable change is that the DataFrame print format was altered to save vertical space and to hint when rows were hidden.</p>
<p>Before:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex&gt; Explorer.Datasets.iris() |&gt; Explorer.DataFrame.print()
+-----------------------------------------------------------------------+
|              Explorer DataFrame: [rows: 150, columns: 5]              |
+--------------+-------------+--------------+-------------+-------------+
| sepal_length | sepal_width | petal_length | petal_width |   species   |
|    &lt;f64&gt;     |    &lt;f64&gt;    |    &lt;f64&gt;     |    &lt;f64&gt;    |  &lt;string&gt;   |
+==============+=============+==============+=============+=============+
| 5.1          | 3.5         | 1.4          | 0.2         | Iris-setosa |
+--------------+-------------+--------------+-------------+-------------+
| 4.9          | 3.0         | 1.4          | 0.2         | Iris-setosa |
+--------------+-------------+--------------+-------------+-------------+
| 4.7          | 3.2         | 1.3          | 0.2         | Iris-setosa |
+--------------+-------------+--------------+-------------+-------------+
| 4.6          | 3.1         | 1.5          | 0.2         | Iris-setosa |
+--------------+-------------+--------------+-------------+-------------+
| 5.0          | 3.6         | 1.4          | 0.2         | Iris-setosa |
+--------------+-------------+--------------+-------------+-------------+
</code></pre>
<p>After:</p>
<pre data-code-wrap="elixir"><code class="lang-elixir">iex&gt; Explorer.Datasets.iris() |&gt; Explorer.DataFrame.print()
+--------------------------------------------------------------------------+
|               Explorer DataFrame: [rows: 150, columns: 5]                |
+--------------+-------------+--------------+-------------+----------------+
| sepal_length | sepal_width | petal_length | petal_width |    species     |
|    &lt;f64&gt;     |    &lt;f64&gt;    |    &lt;f64&gt;     |    &lt;f64&gt;    |    &lt;string&gt;    |
+==============+=============+==============+=============+================+
| 5.1          | 3.5         | 1.4          | 0.2         | Iris-setosa    |
| 4.9          | 3.0         | 1.4          | 0.2         | Iris-setosa    |
| 4.7          | 3.2         | 1.3          | 0.2         | Iris-setosa    |
| …            | …           | …            | …           | …              |
| 6.2          | 3.4         | 5.4          | 2.3         | Iris-virginica |
| 5.9          | 3.0         | 5.1          | 1.8         | Iris-virginica |
+--------------+-------------+--------------+-------------+----------------+
</code></pre>
<p>See here for details on the new format:</p>
<ul>
<li><a href="https://hexdocs.pm/explorer/Explorer.DataFrame.html#to_table_string/2" rel="noopener nofollow ugc"> <code>Explorer.DataFrame.to_table_string/2</code></a></li>
</ul>
<h2><a name="p-369249-added-2" class="anchor" href="#p-369249-added-2" aria-label="Heading link" rel="nofollow"></a>Added</h2>
<ul>
<li><code>Explorer.DataFrame.estimated_size/1</code> - Estimates memory size of a DataFrame</li>
<li><code>Explorer.DataFrame.to_table_string/2</code> - Represents a DataFrame as a string<br>
for printing</li>
<li><code>Explorer.Series.degrees/1</code> - Converts radians to degrees</li>
<li><code>Explorer.Series.radians/1</code> - Converts degrees to radians</li>
<li><code>:quote_style</code> option to CSV functions</li>
</ul>
<h2><a name="p-369249-fixed-3" class="anchor" href="#p-369249-fixed-3" aria-label="Heading link" rel="nofollow"></a>Fixed</h2>
<ul>
<li>Fix bug where <code>:region</code> was incorrectly required in <code>%FSS.S3.Entry{}</code></li>
<li>Fix trigonometric functions to not raise on f32</li>
<li>Fix warning from <code>:table_rex</code> dependency when printing</li>
<li>Fix formatting of <code>Explorer.DataFrame.mutate_with/2</code> options</li>
<li><code>Explorer.Series.fill_missing/2</code> now works for all integer and float dtypes</li>
<li><code>Explorer.Series.frequencies/1</code> now works for <code>{:list, _}</code> dtype</li>
<li>Fix inefficiency with categorization</li>
<li>Fix typespecs
<ul>
<li><code>Explorer.DataFrame.select/2</code></li>
<li><code>Explorer.DataFrame.ungroup/1</code></li>
<li><code>Explorer.Series</code> functions that may return lazy series</li>
</ul>
</li>
</ul>
<h2><a name="p-369249-changed-4" class="anchor" href="#p-369249-changed-4" aria-label="Heading link" rel="nofollow"></a>Changed</h2>
<ul>
<li>Printing a DataFrame looks different
<ul>
<li>Adds a row of <code>…</code> to indicate there are hidden rows. Includes a new option<br>
<code>limit_dots: :bottom | :split</code> to specify how to do this.</li>
<li>Drops the row separators except when composite dtypes are present.</li>
<li>Allows you to pass through valid options to <code>TableRex.render!/2</code>. This<br>
gives you a little more flexibility in case you don’t like the defaults.</li>
</ul>
</li>
<li><code>Explorer.DataFrame.print/1</code> now documents its default <code>:limit</code> of 5 rows</li>
<li><code>Explorer.DataFrame.concat_rows/1</code> has improved error messages</li>
<li>Accessing a DataFrame with a range now raises if the range is out of bounds</li>
</ul>
<h2><a name="p-369249-new-contributors-5" class="anchor" href="#p-369249-new-contributors-5" aria-label="Heading link" rel="nofollow"></a>New Contributors</h2>
<ul>
<li><a class="mention" href="/u/szajbus" rel="nofollow">@szajbus</a> made their first contribution in<br>
<a href="https://github.com/elixir-explorer/explorer/pull/1030" class="inline-onebox" rel="noopener nofollow ugc">Make region truly optional by szajbus · Pull Request #1030 · elixir-explorer/explorer · GitHub</a></li>
<li><a class="mention" href="/u/viniciussbs" rel="nofollow">@viniciussbs</a> made their first contribution in<br>
<a href="https://github.com/elixir-explorer/explorer/pull/1037" class="inline-onebox" rel="noopener nofollow ugc">Add Regex as a valid column type in typespec by viniciussbs · Pull Request #1037 · elixir-explorer/explorer · GitHub</a></li>
<li><a class="mention" href="/u/pejrich" rel="nofollow">@pejrich</a> made their first contribution in<br>
<a href="https://github.com/elixir-explorer/explorer/pull/1040" class="inline-onebox" rel="noopener nofollow ugc">added error message text to exploring_explorer.livemd by pejrich · Pull Request #1040 · elixir-explorer/explorer · GitHub</a></li>
<li><span class="mention">@jdbarillas</span> made their first contribution in<br>
<a href="https://github.com/elixir-explorer/explorer/pull/1049" class="inline-onebox" rel="noopener nofollow ugc">Add quote_style option to csv IO functions by jdbarillas · Pull Request #1049 · elixir-explorer/explorer · GitHub</a></li>
<li><a class="mention" href="/u/petrkozorezov" rel="nofollow">@petrkozorezov</a> made their first contribution in<br>
<a href="https://github.com/elixir-explorer/explorer/pull/1083" class="inline-onebox" rel="noopener nofollow ugc">Enable `Series.frequencies` over `{:list, _}` type by petrkozorezov · Pull Request #1083 · elixir-explorer/explorer · GitHub</a></li>
</ul>
<h2><a name="p-369249-full-changelog-6" class="anchor" href="#p-369249-full-changelog-6" aria-label="Heading link" rel="nofollow"></a>Full Changelog</h2>
<ul>
<li><a href="https://github.com/elixir-explorer/explorer/compare/v0.10.1...v0.11.0" class="inline-onebox" rel="noopener nofollow ugc">Comparing v0.10.1...v0.11.0 · elixir-explorer/explorer · GitHub</a></li>
</ul>
<p>The full changelog includes all contributions with individual attributions. Special shoutout to <a class="mention" href="/u/mhanberg" rel="nofollow">@mhanberg</a> for helping out with some gnarly version wrangling!</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="369249" data-batch-url="/posts/batch_likers">
                        4
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/13">Post #12</a>
	                </div>
	            </div>
              <div id="likers-container-369249" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="369249"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #12"></div>
  </section>
</div>
    <div class="postbit" id="371692" data-post-id="371692">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="billylanchantin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/120/28428_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  billylanchantin
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<h1><a name="p-371692-release-explorer-v0111-1" class="anchor" href="#p-371692-release-explorer-v0111-1" aria-label="Heading link" rel="nofollow"></a>[Release] Explorer - v0.11.1</h1>
<p><a href="https://github.com/elixir-explorer/explorer/releases/tag/v0.11.1" rel="noopener nofollow ugc">Version 0.11.1 released!</a></p>
<p>This is a small one, mostly because we broke printing for lazy dataframes <img src="https://forum.elixirforum.com/images/emoji/apple/grimacing.png?v=15" title=":grimacing:" class="emoji" alt=":grimacing:" loading="lazy" width="20" height="20"> (thanks for the patch, <a class="mention" href="/u/mhanberg" rel="nofollow">@mhanberg</a>!). But it comes with a few improvements too.</p>
<p>Of particular note is that <a href="https://hexdocs.pm/explorer/Explorer.DataFrame.html#group_by/3-options" rel="noopener nofollow ugc"><code>Explorer.DataFrame.group_by</code> is no longer stable by default</a>. Before, groups would always be returned in their original order which is nice but has a non-trivial performance penalty. Now groups are returned in a random order for a performance boost (thanks, <a class="mention" href="/u/petrkozorezov" rel="nofollow">@petrkozorezov</a>!). But you can always set <code>group_by(..., stable: true)</code> if stability is needed.</p>
<h3><a name="p-371692-added-2" class="anchor" href="#p-371692-added-2" aria-label="Heading link" rel="nofollow"></a>Added</h3>
<ul>
<li><code>Explorer.DataFrame.dump_ipc_schema</code></li>
<li><code>Explorer.DataFrame.dump_ipc_record_batch</code></li>
<li><code>Explorer.Series.cumulative_count</code></li>
<li><code>:stable</code> option for <code>Explorer.DataFrame.group_by</code></li>
</ul>
<h3><a name="p-371692-fixed-3" class="anchor" href="#p-371692-fixed-3" aria-label="Heading link" rel="nofollow"></a>Fixed</h3>
<ul>
<li>Fix printing lazy data frame with new default print options (as of v0.11.0)</li>
<li>Fix mutate docs formatting</li>
</ul>
<h3><a name="p-371692-new-contributors-4" class="anchor" href="#p-371692-new-contributors-4" aria-label="Heading link" rel="nofollow"></a>New Contributors</h3>
<ul>
<li><a class="mention" href="/u/wolfdan" rel="nofollow">@WolfDan</a> made their first contribution in <a href="https://github.com/elixir-explorer/explorer/pull/1103" class="inline-onebox" rel="noopener nofollow ugc">Add dump arrow schema and batchrecord by WolfDan · Pull Request #1103 · elixir-explorer/explorer · GitHub</a></li>
</ul>
<h3><a name="p-371692-full-changelog-5" class="anchor" href="#p-371692-full-changelog-5" aria-label="Heading link" rel="nofollow"></a>Full Changelog</h3>
<ul>
<li><a href="https://github.com/elixir-explorer/explorer/compare/v0.11.0...v0.11.1" class="inline-onebox" rel="noopener nofollow ugc">Comparing v0.11.0...v0.11.1 · elixir-explorer/explorer · GitHub</a></li>
</ul>
<p>The full changelog includes all contributions with individual attributions.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="371692" data-batch-url="/posts/batch_likers">
                        4
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/14">Post #13</a>
	                </div>
	            </div>
              <div id="likers-container-371692" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="371692"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #13"></div>
  </section>
</div>
    <div class="postbit" id="371786" data-post-id="371786">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="tmbb" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  tmbb
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>For my SciEx package I’d like to have “cheap” conversions between series of floats and integers (and complex numbers if Explorer supports such thinks, which I think it doesn’t) into 1D ndarrays. I don’t expect Explorer to deal with ndarrays internally - that’s completely outside the scope for Explorer but I wonder if there could be a “portable” way of converting a series into a rust <code>Vec&lt;f64&gt;</code> or <code>Vec&lt;f32&gt;</code> which could be returned into Elixir as a reference and passed into a rust function that would accept a <code>Vec&lt;_&gt;</code>. Making a 1D ndarray from a <code>Vec&lt;_&gt;</code> is effectively free. Currently, the only way of getting a rust <code>Vec&lt;_&gt;</code> for use with SciEx is to convert the Series into an elixir list of floats and then use Rustler to convert the list of floats into a <code>Vec&lt;_&gt;</code>.</p>
<p>I think one could use the C ABI for that, but I’m not totally sure (?). I’m not sure whether converting an Arrow array into a <code>Vec&lt;_&gt;</code> if cheap or not, but I believe it must be reasonably cheap, or at least cheaper than converting into an Elixir list and from that into a <code>Vec&lt;_&gt;</code>…</p>
<p>Is this something the Explorer team would be interested in?</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="371786" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/15">Post #14</a>
	                </div>
	            </div>
              <div id="likers-container-371786" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="371786"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #14"></div>
  </section>
</div>
    <div class="postbit" id="371824" data-post-id="371824">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="billylanchantin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/120/28428_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  billylanchantin
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p><a class="mention" href="/u/tmbb" rel="nofollow">@tmbb</a></p>
<blockquote>
<p>[…] complex numbers if Explorer supports such thinks, which I think it doesn’t</p>
</blockquote>
<p>No, neither Explorer nor Polars support them. You could “make your own” with the struct dtype, but you’d need to implement all the relevant functions yourself.</p>
<blockquote>
<p>Is [exposing the underlying <code>Vec&lt;_&gt;</code>s] something the Explorer team would be interested in?</p>
</blockquote>
<p>I would say no at this time. The problem is that this functionality would be highly specific to the Polars backend.</p>
<p>I wonder though, since SciEx and the PolarsBackend are both written Rustler, could SciEx add a nif that takes a DataFrame as an argument? Then it would be trivial to convert one into an ndarray.</p>
<p>Another thought: using an external file format rather than routing through Elixir. If SciEx worked with parquet or ipc, then it might be faster to write-to-then-read-from a file.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="371824" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/16">Post #15</a>
	                </div>
	            </div>
              <div id="likers-container-371824" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="371824"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #15"></div>
  </section>
</div>
    <div class="postbit" id="372536" data-post-id="372536">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="tmbb" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  tmbb
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="billylanchantin" data-post="16" data-topic="61150">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/48/28428_2.png" class="avatar"> billylanchantin:</div>
<blockquote>
<p>No, neither Explorer nor Polars support them. You could “make your own” with the struct dtype, but you’d need to implement all the relevant functions yourself.</p>
</blockquote>
</aside>
<p>Ok, fair enough, no complex numbers going back and forth. Which makes sense, given how rarely complex numbers come up in real datasets of the kind Explorer is used to.</p>
<aside class="quote no-group" data-username="billylanchantin" data-post="16" data-topic="61150">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/48/28428_2.png" class="avatar"> billylanchantin:</div>
<blockquote>
<p>I would say no at this time. The problem is that this functionality would be highly specific to the Polars backend.</p>
</blockquote>
</aside>
<p>I understand, but do you really plan o having different backends for Explorer? I don’t think that’s a likelye possibility and I was assuming you’d want to keep using Polars, despite ir being decoupled in practice.</p>
<aside class="quote no-group" data-username="billylanchantin" data-post="16" data-topic="61150">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/48/28428_2.png" class="avatar"> billylanchantin:</div>
<blockquote>
<p>I wonder though, since SciEx and the PolarsBackend are both written Rustler, could SciEx add a nif that takes a DataFrame as an argument? Then it would be trivial to convert one into an ndarray.</p>
</blockquote>
</aside>
<p>It would only be “trivial” if I can rely on the fact that a dataframe consists of a number of <code>ChunkedArray</code>s, which again is specific to the Polars backend. The only way to convert data form Explorer to SciEx “cheaply” is by assuming something about the layout of the data on the rust side.</p>
<p>And in any case, one of the problems is that becuase Polars and SciEx are compiled independently, they may actually use different rust compilers or simply different memory representations. Any kind of data conversion would have to serialize the data to a file (as you suggest) or use a fixed representation which could be read from the other side.</p>
<p>I guess I like the serialization idea and I’ll pursue it.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="372536" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/explorer-series-1d-and-dataframes-2d-for-fast-and-elegant-data-exploration-in-elixir/61150/17">Post #16</a>
	                </div>
	            </div>
              <div id="likers-container-372536" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="372536"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>