Any how I’ve yet to turn my attention to elixir with it mostly because I want to see how far I could go doing something I’ve no experience doing in rust first.
All I know is this is changing everything.
Also I wonder if GPT favors strong typed langs given the level of inference needed otherwise.
I think that the content economy is going to plummet. And that’s probably not a bad thing. That model (the current one on the Internet) seems to be:
Some org or person writes an article attempting to share some information.
Hacked and re-hacked search engines attempt to match user queries to the most appropriate pre-written article.
And supposedly, you get your needs met.
But the AI-based paradigm is already much better, 50% of the time for me using Bing Search (which simultaneously sends your query to GPT-4):
Search “engine” pre-reads all the extant content.
In response to a user query, it synthesizes an actual answer from multiple sources.
I love it as a user.
On the other hand, I make my living on the old content economy, so I’m pivoting. Right now, I’m finishing the Beta on what will probably be my last “content-economy” app/startup.
I’m creating 336 Programming Cookbooks: one for each of 21 programming languages (like Elixir), and each of those in 16 world languages, like Hindi. 21 * 16 = 336.
I just started this on January 1 and the beta is nearly done. Crazy. Like you, I’m using a platform I had no experience with, the Hugo site generator. This also includes time for me to try (and reject) Jekyll.
I mean after you look and only see 80 lines of code for logic that can not only analyze pixels of an image but make a list of unique colors based on multiple attributes such as hue, brightness and saturation is amazing given I’ve never written any rust till today and this was less than 3 hours to complete. Then host it for free without any needs for a backend service or any external services to do this and its all client side
I get what I pay for? I will never write code again the same and it was only $20.
My life will forever be impacted by this and it has a basic level monthly subscription
I have no idea what the world will look like in the next year and feel like all of the best and worst things are all going to collide at the same time.
Sorry I was referring specifically to the CS degree portion of your message. Not debating whether you can output working programs using Chat GPT. I can also output working programs by copying someone’s solution from Stack Overflow, but I would never consider it the equivalent of a CS degree.
For sure, I’m only able to create this amazing website with good UX, pristine data, few bugs, in many languages and for zero hosting fees because I have a CS degree, 20 years of work experience, and have lived on two different continents.
It’s just the tiny incremental cost of 2 weeks’ dev time that looks magical.
I think there’s definitely bars to competition as well as value to good education.
I took a look at this in Finnish, and I have quite a bit of feedback. I’m not sure how strongly it should be given since you mentioned that this is not beta yet, so take from this what you will. I looked from the viewpoint of stumbling upon these guides when searching online. I’m writing this with the assumption that you plan on using ChatGPT for the content generation. There are several issues:
Falsehoods in the code examples and text. All the Gleam examples are wrong (since there’s not much Gleam for ChatGPT to know about), and not in a way that it’s just old – the examples are syntax that has to my knowledge never existed in Gleam. Several examples in Elixir are also wrong, e.g. claiming that strings in Elixir are objects (and thus you can call "FOO".downcase), and that strings can be concatenated with + but it’s slow, and that’s why Elixir uses <>. In the introduction it said that “other languages using the Erlang VM are Ruby on Rails and Phoenix Framework”. The page about date parsing teaches how to use a nonexistent function. For a popular language such as JavaScript the code examples seemed to be quite accurate.
Bad translations. When I think about it from a technological point of view, I’m actually surprised how well it works. But when I think about it as if I was a reader of a programming tutorial, it’s not good enough. Constantly having the wrong inflections makes it more difficult to understand the text, and sometimes it changes the meaning of the sentences. Several different words are used for the same concept such as “string” (which only has one acceptable translation). As an example, the name of the Elixir guide for upcasing a string is Merkitsevä nauha, which actually means “meaningful ribbon”. One page states “People who place bets may wonder what HTML analyzing means”. I don’t think the tech for automated translation is there yet; this would need a thorough review by human volunteers which might take the same amount of time as a translation in the first place.
Disconnect between body text / footer links and code examples. For example in a C++ example, the footer linked to chrono, but the code example stored a date as three separate int variables and didn’t use the library at all. All in all the code example was quite useless. (The code also imported ctime but didn’t use it.)
Hallucinated docs links. E.g. this page has all the links lead to 404 or not the content you’re looking for. Elixir doc links pointed to 1.12 in some places that I saw.
Fluff. Every guide seems to have an introduction and a “deep dive” that are mostly worthless. Every one of them says something like “X is very important to learn” or “Historically X has been part of the computer science since the 1960s”, which reads very odd and is just unnecessary when it’s something like “HTML analyzing has been an integral part of creating and analyzing websites since the beginning.” (It then continues: “Before the invention of JavaScript, JavaScript could only be used to analyze static data.”) I think these introductions and conclusions should be dropped because they don’t offer anything of value.
Since you said this is nearly beta, I hope there is some way to address the above concerns, especially just the amount of misinformation. At the least I think each page should be manually reviewed before publishing.
The site itself looks nice and easy to navigate.
And this is my experience with the current LLMs. The output looks correct from further away. It’s very wordy and knows how to be convincing and confident. But when you get into the details like you need to do when you’re learning from a guide, it starts to fall down and needs manual attention.
Now on the other hand, I see you’ve both quickly made websites that work using LLMs. So there’s plenty of promise there.
I totally agree, I gave it a few tries when I was trying to integrate zig builder in elixir. The code it displayed, and the overall explanations it gave were abysmal, a lot of them completely wrong and even some of them containing code that cannot be compiled in zig. The biggest problem I have with this is that there is no distinction between answers that can be potentially wrong and some sure ones, hence I can believe only what is written in an official source and at this point why bother and not read the official documentation.
I think using LLMs for code generation is just not worth the effort, if you are writing repetitive code that could be easily generated by a bot it means the problem lies elsewhere, most probably at the automation level in the codebase.
It would be cool to build an open source LLM and fine tune it for elixir.
If you have a specialized use case, and enough data and time to fine-tune a specialized model, you can achieve competitive or better performance than proprietary models.
Are there any attempts? Haven’t looked but curious, it seems logical to me that you’d have specialized by programming language LLM’s.
I’m currently using openai GPT v4 and its day and night from v3.
I think most of us devs have no idea just how powerful open AI v4 is and it going to disrupt everything over the next few months if its not already.
Honest to god, give it a try if you have not yet. I have been blowing it off cause I’ve tried many versions of gpt from huggyface and so forth but non have come close to what I just seen over the last day on V4. Its just that big of a diff.
That said I’ve only been playing with rust with it so far and I suspect due to the strong type is why its been easier than my past experience with dynamic langs
Crazy. Like you, I’m using a platform I had no experience with, the 
