My prediction is that it will be used to write lots of code that the author doesn’t understand, and that folks will continue to insist that a fancy autocomplete is “sentient”.
al2o3cr
dmitriid
There’s a cut-off on data it was trained on. It was 2021 for GPT 3, no idea for GPT4
Coupled with the fact that it always tries to generate a plausible-looking answer, expect it to generate a lot of garbage, especially for something reasonably new or unpopular. I tried to ask it to generate something with Ash Framework, and it generate actual correct code… using Ash modules that have never been in existence and have never even been discussed.
I expect GPT4 with it’s better generative model to be even worse in this regard. [1]
Then there’s the problem of choice. If you have many ways of doing something, you will get wildly different results for the same input.
Just today I was playing with it (again, v3, not v4, but I expect the outcome to be the same). The query I had was “using latest Java, and the best libraries, write a Telegram bot integrating with OpenAI”. Running this query three times gave me implementations with OkHttp, unrest and httpclient. It looked plausible, but I didn’t check it.
So, if you’re willing to babysit it like you would a junior dev, and code review every step of the way, and use rather outdated libraries for some common boilerplate code… then it’s pretty good.
[1] This is true for programming and significantly worse for anything outside programming. Basically, as long as it’s popular, non-technical, and in English, the results will often blow your mind. Since the modern world (especially online world) is becoming so predominantly English-speaking, LLMs have an insanely large trove of data to pull from. For anything else the results they produce will very quickly become invalid garbage even if there’s at least some data on the subject available in English. E.g. try asking it about relatively non-popular non-English literature. Same goes for programming languages and technical topics.
2
n0gg1n
dmitriid
The issue of outdated info still remains. Quick googling shows that even GPT-4 is rained on data up to September 2021. So it’s useless for any info in the past 2.5 years which includes some of the biggest announcements and releases in Elixir: the vast majority of Nx, Livebook, etc.
1
fceruti
I guess there’s plenty that can be done without AI.
Just basic ad-hoc documentation and examples would be amazing!
1
regex.sh
True, outdated training data is a challenge, but GPT-4 can still be useful in many situations. Combining AI with ad-hoc documentation, examples, and community-driven content could provide a more comprehensive solution for developers. Excited to see GPT-4’s evolution and its impact on the developer community.
1
Hmaddocks
My experience so far has been that GPT has been fine, or even quite good, for more “mainstream” languages, JavaScript, Python, maybe Rust, but useless for Elixir. The Elixir code it produces is almost always garbage. I don’t mean bad code but, not Elixir. It usually writes weird hybrid of Elixir and Ruby.
My expectation of 4 is pretty much the same unless they have used a much broader training set.
2