Chutzpah is fine, but what I care about is what is good for the Elixir ecosystem.
By that thinking rustler should be called rust, bandit should be http, and ecto should be sql, and so on. Even libraries that provide access to a specific API tend not to call themselves that.
Of course, there are libraries that do this (sort, openstreetmap, ..). It ages poorly for the decent libraries, and is simply unhelpful for libraries that don’t end up working out. I don’t know where the sql library will end up, but calling it sql is poor form imho.
As an extensive user of Elixir libraries and someone who has introduced Elixir into new places more than once, this sort of thing is not just aggravating, it tarnishes how Elixir looks during technical due-diligence.
I actually offered some feedback above, and most of the questions I asked are still there.
I’ll hold off on going deeper into things like D/X for custom types until I understand those points a bit better. Cheers.
You cannot accuse someone of squatting on a name when they are actively using that name to work on and ship a package. If a good name is actually being squatted on by a dead package then you can request a takeover of that name.
A global namespace is probably a bad idea in general and technologies like ATProto show a better path, but I digress.
You may be surprised to learn just how deep this problem actually is. ORMs are able to get away with this by choosing to implement only a subset of SQL. This project will either have to do the same or the parser will have to grow to the 50-100kloc range. SQL is a demonic language.
I think the subset path has merit and I like the idea of an ORM-style tool where the syntax is extremely close to actual SQL.
I mean, that sounds like exactly the design philosophy of ecto, but with a priority on composition. Genuine question, without composition, what’s the advantage of a extremely sql like syntax thats not sql?
I think you have it the wrong way around. ORMs, and I’m including Ecto here, mainly exist to make up for two big shortcomings of SQL: that it does not compose well, and that it does not perform structural transformations well. I’ll focus on the first one.
SQL does not compose well because it’s a bad language. Other languages, like QUEL that came before it, were much more elegant. SQL was designed carelessly but was backed by IBM so it won by default. Similar to JS, though I would say SQL is worse.
This is why ORMs exist: to fix a problem (composition) which should have been handled by SQL itself. Unfortunately most ORMs also contribute some nasty warts of their own, specifically the disaster that is “object-oriented programming” and the addition of globally mutable “row” objects that can be modified as side-effects and then inadvertently committed. Ecto is amazing because it’s an “ORM” that does not have this problem. Because Elixir does not have this problem.
This project, the SQL library, does have composition. That’s why I personally love the name: I think it’s hilariously ironic because if you add composition to SQL it is clearly no longer SQL. You have invented something else that looks like SQL but isn’t because it actually composes properly (unlike SQL).
The fact that it sounds exactly like the design philosophy of Ecto is the whole point. I like Ecto, so I like this!
I’m sure @Schultzer could give you a list of his perceived advantages, but personally I don’t really care. I don’t think projects need “advantages” to justify their existence. It’s okay to just make something because you feel like it. In fact, if you look closely, you will find that this is where most good things come from.
I think I just misread your point about it being a “deep problem” as an argument against supporting the level of composition that @aseigo demonstrated in his example. Certainly I concur with their feedback that as a prospective user that I would expect a library that claims to handle composition in any way to cover such basic cases and hopefully set some clear expectations about what should be composable and what is not and why, before I would consider investing much time in it (or pay much attention to benchmarks).
As a side note, I think it is precisely because Ecto does not claim to be an ORM at all that it is able to “guarantee” the level of composition it does. So, although clearly it functions as one, the API sets clear a clear boundary for this guarantee. By contrast, ActiveRecord did not, which led to some embarrassing deficiencies where only slightly unusual queries would return incorrect results depending on the order of operations. I am somewhat curious how this library can/will handle that.
Yeah, I figured there might be some misunderstanding so I tried to be very clear. I think I see the confusion now.
The example that @aseigo gave did exhibit composition. But he fed the library a nonsense input and got nonsense back out (the statements got weirdly reordered). It’s a bug.
The purpose of my reply was to point out that fixing bugs like this will be difficult because parsing SQL is extremely hard (because it is a very bad language). The article I linked gives a really clear example of this. But it can still be done, obviously.
I have never written Ruby, but now I’m curious. Could you spare any examples of this?
Unfortunately no, it’s been too long and the source is lost to time, but IIRC it has to do with referring to aliases of joins in where clauses. Since it lacked explicit bindings and tried to resolve everything automagically, it would get various edge cases wrong. Not sure if/how it was solved, but my memory was that the best it would be able to do would be to raise an error because there was no way to infer what the intended alias actually was with the existing API. Maybe they ended up changing the API, I stopped using AR shortly thereafter.
I find composition to be the least important problem to solve, especially in an enterprise envoiroment. And SQL will at somepoint reach Ecto level, maybe a slight better but no guarantee.
What is far more important is being able to sleep at night knowing that you won’t ever have to deal or think about pool_size, queue times and hard to debug db_connection errors. This is solved with a breakthrough in database integration design, leveraged by the BEAM and Elixir which makes this possible and unheard of in any language or SQL client ever built.
AFAIK, this is also the first time SQL has become a first-class citizen in a language, giving us compile time errors and bare metal performance with unheard of concurrency, simply by standing on the shoulders of giants.
For me personally, this is a passion project, to showcase the next generation what is possible and how to write Elixir with bare metal performance, and make it crystal clear what diagonal scalling is and how to create libraries which ultilizie the BEAM’s most advanded feature and to be honest the one we always undersell for reason I can’t fathrom as we can do things no other platform will ever be able to do in any immediate future, and properly always be behind due to Erlang’s 50 years head start and being the language that defined and continue to showcase what a destributed language and compute is.
I already sleep fairly well without worrying about them. They are flagged as non-critical and monitored according to frequency rather than occurrence.
I agree with @garrison that has a huge value in an of itself, but I do think you will get a lot of critical feedback about the composition issue, so maybe I’d suggest not mentioning composition in the library “highlights” until it’s a feature you think is important.
I do think maybe including those errors you mentioned, making the case for why people should care, and how exactly the SQL library avoids them would be a valuable addition to the README intro.
I’ll welcome any contribution to improve composition, The once that has been highlighted are straightforward to solve, to be more clear, my intention is not to never solve it, but as I see it right now there is way bigger problems that I’m focusing on. Ecto makes a lot of tradeoffs when it comes to this, which SQL ideally would not have to make, but currently I’m the only one making major contribution and my time is limited, but I’m here to guide anyone willing to contribute and try to solve a problem that has already been solved in the respected database engines, althought with our little twist. There is a bunch of literature out there, and SQLlite even got piping working but decided not to move forward as they keep strict compliance to the spec, which we kinda do not or more specificaly we help the user to compile proper SQL so they don’t have to think about starting with select or from. The problem stated in this thread is all about merging clauses and predicates, fairly simple when you have an AST to work with.
This thread itself shows that naming an Elixir package the same as an other language is a very bad idea. When does someone talk about the package when they write ”SQL”? When about the language?
I must admit I haven’t been able to follow the thread just because of that.
The one and only reason I would abstain from a generic name like sql would be that I don’t trust my planning skills enough to know whether a project’s scope might not change in the future, making the name an inconvenience or misleading.
@Schultzer I’ll have my SQLite thing in the open very soon (a few weeks more; and technically it’s open and working even now but it was a quiet launch and there are still 1-2 features I want to put in before I make an announcement). What would you need from my raw library (no Ecto integration yet) so yours can work with it? We should likely take this to DMs but just giving you a heads up.
