<turbo-stream action="append" target="posts_list"><template>    <div class="postbit" id="346610" data-post-id="346610">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Sorc96" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Sorc96/120/28361_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Sorc96
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Thank you for adding another well thought-out contribution to this topic, I appreciate it!</p>
<p>I see you’re approaching this from the custom mapping side, and of course, that makes most of the other problems disappear. I haven’t really seen an Elixir project that does this, though, so there does not seem to be any guidance on how to do it well.</p>
<aside class="quote no-group">
<blockquote>
<p>Also obligatory shout-out to the Ecto developers for doing wonderful work with only a fraction of the resources that entities like Microsoft have.</p>
</blockquote>
</aside>
<p>Absolutely. Don’t get me wrong, the database part of Ecto is really amazing and probably better than what I have seen anywhere else. The validation part of Ecto is pretty good as well. I’m mostly talking about the mapping capabilities that don’t sem to be “good enough”, but of course, that’s not a problem if you use custom repositories to do the mapping.</p>
<aside class="quote no-group">
<blockquote>
<p>I like your rephrasing a lot, I will steal it if you don’t mind</p>
</blockquote>
</aside>
<p>I’d be happy about that!</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="346610" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/32">Post #31</a>
	                </div>
	            </div>
              <div id="likers-container-346610" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="346610"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #31"></div>
  </section>
</div>
    <div class="postbit" id="346663" data-post-id="346663">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="sodapopcan" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sodapopcan/120/34668_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  sodapopcan
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Whoa ok, wrong time to take a couple of days off, lol.</p>
<aside class="quote no-group quote-modified" data-username="tfwright" data-post="15" data-topic="67389">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/tfwright/48/20299_2.png" class="avatar"> tfwright:</div>
<blockquote>
<aside class="quote no-group" data-username="sodapopcan" data-post="14" data-topic="67389">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sodapopcan/48/34668_2.png" class="avatar"> sodapopcan:</div>
<blockquote>
<p>I don’t know of a web framework that doesn’t push you to model data based on its storage</p>
</blockquote>
</aside>
<p>I wonder if this is really true though?</p>
</blockquote>
</aside>
<p>To address this, you’re right and I’m only speaking in my narrow experience.  The active record pattern (the general one, not Rails’ specifically) pushes this by its very nature.  Ecto, of course, doesn’t do that.  I haven’t brought it up because a lot of the conversation is geared toward many tables to one schema, but of course another thing I don’t see enough of (though certainly many people do it) is having schemas only be small slices of tables.  One thing that happens a lot when doing table-first design is you end up with all these join tables, always to <code>users</code> but also to other entities, that are actually first-class concepts but that isn’t always apparent you think of it as a join table.  Often <code>Accounts.User</code> is referenced everywhere when often it would be better suited to have its own context-specific representation.  If it’s 1-1 to <code>User</code> then the users table can still be used for storage, but that means you can ignore the account-specific columns.</p>
<p>Anyway, there is an overwhelming amount of stuff here and on the forum in general so I’m going to leave it at that for now.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="346663" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/33">Post #32</a>
	                </div>
	            </div>
              <div id="likers-container-346663" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="346663"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #32"></div>
  </section>
</div>
    <div class="postbit" id="346763" data-post-id="346763">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="jkwchui" src="/assets/icons/user-9f439610.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  jkwchui
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Since OP started by describing this as a graph-relational impedance mismatch: <a href="https://forum.elixirforum.com/t/attempt-at-adapting-kuzudbs-rust-crate-into-elixir-via-nif-any-tips-thoughts/66100" class="inline-onebox" rel="nofollow">Attempt at adapting KuzuDB's Rust crate into Elixir via NIF, any tips/thoughts?</a></p>
<p>I find graph databases (neo4J, dgraph etc) painful to setup/deploy.  KuzuDB is a new embedded graph database (think what sqlite is to postgresQL), now with a fledgling Elixir binding.  It does ask for a strongly typed schema, but maybe there is some new approach to modeling applications that somewhat breaks free from “ecto is your application”.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="346763" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/34">Post #33</a>
	                </div>
	            </div>
              <div id="likers-container-346763" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="346763"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #33"></div>
  </section>
</div>
    <div class="postbit" id="346766" data-post-id="346766">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="dimitarvp" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/dimitarvp/120/38664_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  dimitarvp
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="jkwchui" data-post="34" data-topic="67389">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/j/ea666f/48.png" class="avatar"> jkwchui:</div>
<blockquote>
<p>It does ask for a strongly typed schema</p>
</blockquote>
</aside>
<p>This pulls it ahead from others IMO.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="346766" data-batch-url="/posts/batch_likers">
                        0
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/35">Post #34</a>
	                </div>
	            </div>
              <div id="likers-container-346766" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="346766"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #34"></div>
  </section>
</div>
    <div class="postbit" id="346767" data-post-id="346767">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="bgoosman" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/bgoosman/120/36055_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  bgoosman
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>Hi, author of kuzu_nif here. KuzuDB is really exciting for graph. It even has an in browser version (kuzu-wasm)</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="346767" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/36">Post #35</a>
	                </div>
	            </div>
              <div id="likers-container-346767" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="346767"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #35"></div>
  </section>
</div>
    <div class="postbit" id="347163" data-post-id="347163">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="Sorc96" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/Sorc96/120/28361_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  Sorc96
                    <span class="op-star" title="Thread Starter">
                      <img alt="OP" class="op-star-icon" src="/assets/thread-icons/thread-icon-thread-starter-df91e872.png" />
                    </span>
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I have been thinking about this for the past few days. I don’t have real world experience with graph databases, so take this with a grain of salt.</p>
<p>As long as we directly use entities from the database access tool in our domain, they will dictate the design of the domain. I’m afraid this can only truly be avoided by doing our own mapping to domain entities.</p>
<p>Then the question is how constrained are we in designing the domain due to the limitations of the data access library/framework? I’m assuming that a graph database could store the data in a representation that is closer to what we would like in our code, therefore requiring simpler mapping features from the tool that we use to access that data.</p>
<p>However, mapping to a relational database is still clearly possible, it may just need more ergonomic ways to use some of the advanced features of Ecto. In the end, I think this is about convenience. Writing custom mappers with arbitrary design is very different from having a set way of defining mapping using existing tooling.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="347163" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/37">Post #36</a>
	                </div>
	            </div>
              <div id="likers-container-347163" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="347163"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #36"></div>
  </section>
</div>
    <div class="postbit" id="347207" data-post-id="347207">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="billylanchantin" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/120/28428_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  billylanchantin
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<p>I also feel the pain of the “impedance mismatch” between core app logic and the persistence layer. I find this is particularly true with DB-specific concepts like transactions. The semantics of interacting with the DB forces you to consider things which are difficult to express natively in the app. That in turn increases the cognitive load of determining what effects local code changes have on global properties like overall throughput.</p>
<p>I don’t have much to add to this discussion in terms of what to do about it in Elixir. I agree with a lot of what’s been said, and I’ve more or less resigned myself to the idea that this is a fundamental problem in data-intensive applications. Ergonomics can always be improved, and <code>Ecto</code> does a good job IMO, but the friction will never entirely disappear.</p>
<p>I will add however that this pain caused me to pay attention to the Rama project over at Red Planet Labs:</p>
<ul>
<li><a href="https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/" rel="noopener nofollow ugc">https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/</a></li>
</ul>
<p>Their pitch is to overcome this problem by writing applications in a system where there’s less distinction between app and persistence. There is, frankly, a <em>ton</em> of info to digest over there. The best tl;dr I can provide is that their platform is a combo of event-sourced data storage and materialized views on the data, but where the views are defined/consumed by the app itself using native data-structures.</p>
<p>Unfortunately I don’t think I’ll ever use Rama itself. It’s proprietary and also requires the app be written in Java or Clojure. But I find the ideas appealing and I’m amenable to (my understanding of) the underlying sentiment: a more integrated approach – one where the app “speaks” persistence natively – may be necessary to overcome the mismatch.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="347207" data-batch-url="/posts/batch_likers">
                        3
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/38">Post #37</a>
	                </div>
	            </div>
              <div id="likers-container-347207" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="347207"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-standard-post cat-standard-post" title="Post #37"></div>
  </section>
</div>
    <div class="postbit" id="347211" data-post-id="347211">
  <section>
    <div class="post-wrap">


					<div class="post-header">
		        <div class="user-avatar">
		          <img alt="D4no0" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/D4no0/120/33624_2.png" width="120" height="120" />
		        </div>
					
						<div class="user-details">
		          <div class="user-name">
		            <h3>
                  D4no0
                  </h3>
		          </div>
						
						</div>
					
					</div>

	        <div class="thread-main">
	            <div class="post-body" data-turbo="false">
								<aside class="quote no-group" data-username="billylanchantin" data-post="38" data-topic="67389">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/48/28428_2.png" class="avatar"> billylanchantin:</div>
<blockquote>
<p>I also feel the pain of the “impedance mismatch” between core app logic and the persistence layer. I find this is particularly true with DB-specific concepts like transactions. The semantics of interacting with the DB forces you to consider things which are difficult to express natively in the app. That in turn increases the cognitive load of determining what effects local code changes have on global properties like overall throughput.</p>
</blockquote>
</aside>
<p>Honestly transactions feel like a lot like pipelines that are built with <code>with</code>, in terms of their properties to short-circuit and rollback.</p>
<p>My opinion on this is that this friction will never truly disappear, you can look at higher-level ORMs from languages like c#/java where they tried to abstract the data storage as much as they could and it’s clear that their approach is much less flexible than ecto, not to mention that it’s more complex overall than having the low-level options, like queries.</p>
<p>I think that ecto key quality is extensibility. If you are not a stranger to how metaprogramming works, you can literally tailor the ecto to your project needs, saw this being applied successfully in a lot of good projects. The same goes for coupling, if you don’t want to couple too much to ecto, you can literally do schemaless queries or use ecto for validations only at the edge of the system.</p>
<p>I also think that adding more features to ecto is a mistake, as a lot of issues encountered with ecto are from project to project, hence an abstraction for that specific project is much easier and manageable to do.</p> 
	            </div>

	            <div class="base-line">
	                <div class="thread-counters">
	                    <span class="thread-count count-likes js-likers-trigger" title="Likes" data-post-id="347211" data-batch-url="/posts/batch_likers">
                        2
                      </span>
                      <!-- <span class="thread-count js-solved-indicator" title="Marked as solution"></span> -->
	                </div>
	                <div class="go-to-post">
	                  <a title="Go to post" alt="Go to post" href="https://forum.elixirforum.com/t/design-discussion-ecto-is-your-application/67389/39">Post #38</a>
	                </div>
	            </div>
              <div id="likers-container-347211" 
                   class="likers-container"
                   data-first-post="false"
                   data-batch-url="/posts/batch_likers">
                   <div class="likers-placeholder" 
                     data-likers-post-id="347211"
                     data-batch-url="/posts/batch_likers">
                  <div class="post-likers"></div>
                </div>
              </div>
	        </div>
			

    </div>

    <div class="triangle-top-right type-last-post cat-last-post" title="Last post!"></div>
  </section>
</div>
</template></turbo-stream><turbo-stream action="replace" target="load-more-container"><template><div id="load-more-container" class="load-more-container">
    <span class="all-loaded">— All posts loaded —</span>
</div></template></turbo-stream>