Your posts reads like āmany people tried it hence it must be goodā. Not sure whereās the actual argument.
1 Like
The point is: huuu-maans also produce bad code.
Where you able to quantify how much co-pilot helped your productivity? I expect you to have used it because you made this thread.
Yes, humans also produce bad code. They can read, understand and correct it however, unlike ChatGPT and Copilot.
Very interested in you sharing your experience after you tell one of both: āplease fix the bug in this codeā. Would be super cool if they actually could, and if they can Iāll absolutely be integrating them in my workflow.
1 Like
Sure, it has to be a dialog.
Have you ever tried that with chatGPT?
It is not yet good enough (if only because there is no IDE-integration) to be usable, but we see where this is going.
Who is āweā? 
Not to drag this sub-thread forever so I am stopping discussing the [supposed] quality of ChatGPT and Copilot after this comment ā but somebody in the 1970s tried a few things with LISP, liked them a lot, and then 5000 articles later government and corps invested literal hundreds of billions on AI that never got created, leading to the so-called āAI winterā. Skynet never got created, too.
Tone the hype down. 
Iād love to be wrong btw. So far I see zero indications for it. Weāll see though.
3 Likes
agreed.
weāll see.
Company I work for maybe year before corona pandemic hit had three backend programmers who used C# me included. We all tried using Visual Studio IntelliCode back then for 2 months. We three all agreed back then that it was actually hurting productivity. Of course that was many years ago and that AI product was very new back then. @dmitriid reply makes me believe these tools havenāt made much progress since then. Back then when we started using it there was lot of talk from Microsoftās direction that it will greatly increase production. Iāve stopped believing these sale speeches because after using product myself it wasnāt at all good what they claimed it be. We all three programmers of course where senior programmers and our codebase used coding style we had decided on plus we used tools that automatically validates it. Example I started programming with C# in 2003.
They have really progressed since then, and I find especially IntelliJ tools quite useful (near-painless refactoring, finding duplicates, rewriting chunks of code to new standards etc.)
Iāve found Copilot to be a very minor improvement over the current state of the art in IDEs in those rare cases when it actually worked 
Thereās probably an area where they could be somewhat of a leap eventually: boilerplate, data structure conversions etc. But they are not there yet.
===
Though I just remembered. We had a short straightforward bash script that we needed to convert to something windows-specific like BAT or PowerShell. ChatGPT produced garbage trying to convert it to BAT, and was only a small error away from a perfect conversion to PowerShell.
So code rewriting is maybe within the realm of possibility for some subset of code.
1 Like
I do believe AI will eventually write a lot of code for developers. However, a few months ago I had a run with a few of them and quickly realised they are ānot there yetā.
Although it is nice to see them write the License text including author name in the codeā¦ā¦ow waitā¦
2 Likes
to reply to the original intent of OP: the only thing we can do is have more public projects with a permissive license on github. and since business isnāt going to change their codebase just to improve someone elses product (github) then thereās nothing to do.
iām still waiting to have autocomplete that would just suggest me keys from a struct, lol otherwise the generators are helpful enough imo.
i just installed copilot anyway, just to try it out. i absolutely LOVE (the idea) that it can make a simple functions in Elixir (that would take me less than a minute anyway). i want to see how practical it proves to be. of course what itāll be in 5 years is a whole other story.
1 Like
I talked chat gpt through writing me a neural network with a exponential decayed potentiation to increase the likely hood of re triggering again after a recent trigger event in elixir. It was on par with a junior ish developer in terms of what mistakes it would make but it was fast and far cheaper than my hiring someone to basic plumbing/setup work. It will be a mistake to overlook this area regardless of the immaturity of the current generation of tooling.
I was pumped by ChatGPT at first, now Iām just worried of the damage it will do. Iāve used it almost daily in the past month, for ideation, code generation (learning Elixir), asking quick facts, and it has a scary high ratio of just giving you garbage super confidently, both with code and simple conversations.
For example, I asked when a certain person won the world championship in some sport. It gave me a wrong answer. But it was so wrong, that it even gave completely fictional details about city & country of contest, athlete names, years. When pressing that the answer is not correct, it made up stuff again. This went on for 5 or 6 rounds, confidently spitting out lies. When this happens with code, especially if itās generating issues with edge-cases that you donāt realize at first, thereāll be way more time lost than won with the scaffolding.
Iāll park my initial enthusiasm and will revisit ChatGPT & co in a couple of months.
3 Likes
I believe this was written in a confusing way. The following chart from the article is a bit clearer in that those numbers mean suggestions acceptance rate. So 46% of the suggestions were accepted but this is a quantitative measure and it makes sense they are higher for Java as it has more boilerplate. But here is what I would love to know:
-
If 54% of the suggestions are rejected, does it mean I need to parse a suggestion and then discard it? Which would mean that most of the time suggestions could be slowing me down?
-
What is the time taken to accept or reject a suggestion?
-
Does Copilot tracks what happens with a suggestion? Maybe it is accepted and then it is immediately changed or removed because it was wrong?
In any case, I believe there are two separate discussions here, and they are getting mixed.
-
Are the AI tools in a state where we can consider them trustworthy or generally acceptable? To me the answer is no. Besides a huge potential copyright issue on tools like Copilot, which has made some organizations ban certain AI tools altogether, there is still a lot to improve. For example, researchers have found that code generated by OpenAIās Codex contained security vulnerabilities 40% of the time. However, the tools will improve as there is a large amount of techniques and ideas that still have to explored and potentially adopted (such as reinforcement learning with static and security analysis!).
-
That brings us to the second discussion: should we do what is necessary to get Elixir working with more AI tools? To me, the answer is a 100% yes, because it will only get better and it is not only about code completion.
15 Likes
I think weāve seen this here on the forum as well. Newbies pasting a piece of code and asking why it doesnāt work, without explaining what they were trying to do with it in the first place. Often, the code isnāt even syntactically valid.
Also, I remember reading about somebody asking how to transpose a matrix in Elixir. ChatGPT insisted that there is a transpose function in the standard library. It even generated a working implementation when asked to show the source code.
4 Likes
My last comment on ChatGPT will be this:
I asked a dev on my team to build a pretty simple feature using Phoenix channels. We sketched out the basic architecture, they said they understood, and off they went. Some very long time later they came back with a working prototype, but it did not adhere to our design. Specifically, it was pushing data from one channel to the client, which was sending it back up to a different channel, rather than sending it from one process to another directly as we planned. When I asked what was up with that, they said they couldnāt figure out how to get that piece working, and they asked ChatGPT which told them it was not possible. So there was a whole suite of javascript functions responsible only for listening for different data events and sending them right back. This was despite having the relevant documentation at hand. But why fuss with sometimes confusing documentation when you can just ask an elaborate magic 8 ball?
My first experience with āAIā programming in the wild and sadly, yes, probably not my last.
6 Likes
I canāt wait to write āWrite me a state machine in Elixir which can keep track of daily routinesā and then āWrite me a LiveView frontend for the backend in the visual style of other viewsā and knowing the code is excellent.
This way I would only have to create really unique things that require human capabilities. No more plumbing, just āsnippets on steroidsā.
But we are not there yet.
Ps. We better learn how to āaskā best as the quality of the input largely influences the quality of the outcome. For reference: geek vs noob search results
1 Like
I havenāt used this to generate code, but Iāve asked a few Elixir-related questions using https://phind.com/ and have had great results. The responses all have references, so Iāve found it to be a pretty great āshortcut to the right docsā in a couple cases where it wasnāt clear where to go first.
5 Likes
After reading the posts, it seems that there are mixed opinions, the only question that was not addressed: can a matrix of computed weights (witch such technologies are) that is not capable of understanding concepts be capable of solving a problem? the short answer is no.
If such a technology is capable of replacing people, then it means that the work they do could be automated long ago, AI was never needed for this, all this scaffolding and plumbing problem you are talking about so often seems to be more of an organizational problem than related to the limits of engineering.
As for my last five cents, products like copilot and chatgpt will always be biased, because they are commercial products and their main scope is to make profit, not your life easier, this is true now more than ever, where consumerism is more important than innovation.
3 Likes
Thatās crazy. I understand using ChatGPT to test its capabilities and play with them, but there are actually developers using it to help them code?
IMHO most people in this thread underestimate how much AI will disrupt our space, just like artists and designers thought for the longest time that AI would never be a problem for them. Nothing to really worry about for the next few years, but I dread a little being forced to use this kind of tool in order to stay competitive in the future.
2 Likes
