Get AI code generation tools to create correct Elixir code, or else šŸ˜±

I think the point is developers can use AI to help them not code. Thatā€™s the selling point. It sounds good, because people hear it and think ā€œmore productivity, less workā€, which is not untrue. Of course, like any tool, there are dangers to being too reliant on it. But in the case of AI, I think the very words we use to describe it, starting with ā€œintelligence,ā€ give the false impression that if you have this tool, you donā€™t need to know what youā€™re doing. And that is something we should all be very suspicious of, concerned about even, and not really because we fear for our jobs.

2 Likes

Your greatest asset is transforming ā€œI want a buttonā€ into a functional requirement including things like ā€œwhich backend services I need to talk to for auth and which are deprecated based on an email I saw the team that no longer exists sent out three months ago. Oh, and does legal need to be involved with thisā€. And then coding it.

Most people in this thread greatly overestimate the usefulness of a toddler, sorry, ā€œAIā€.

Some repetitive tasks will be replaced, poorly. The actual task of building a functioning app? Transferring labor to cheaper places overseas is still the greatest threat in the forseeable future.

1 Like

In my experience ChatGPT is terrible at generating Elixir even with examples of the right way to do things. It will make up BIF that exist for Python but not in Elixir for example.

chatGPT is a toddler now. But this will change.
I see what AI already does to my field (buildings).
See a glimpse of whats already possible here.

This is moving fast. 99% in the branch are not prepared and will cease to be competitive, especially for larger projects. I already see that. It is gonna be a massacre.

2 Likes

I already see that. It is gonna be a massacre.

Of course it wonā€™t. Even in the video you link at 20:49: ā€œA lot of people are worried that itā€™s gonna take over my job and then Iā€™m not gonna have the work, but youā€™re still are involved in the process. Itā€™s just helping you do your job better and fasterā€.

Which is painfully obvious even with al the amazing generative tools that are shown in the video.

Because there are a hundred million small and large decisions that have to be made that the AI cannot do until the person who knowns about these details tells it.

ā€” long story ā€”

I donā€™t know that much about construction, so Iā€™ll give you an example from my area. I work for Spotify. Letā€™s say youā€™re tasked with implementing a ā€œskip to next songā€ button. A surprising amount of people today will say that ā€œyup, your job is gone, the AI can do thatā€. Whereas a skip button will depend:

  • on auth state
  • type of account (premium/ad-supported)
    due to licensing on some devices (only some devices, and often not even all devices from the same brand) you cannot skip more than a certain number of songs, and that should result in a valid message to the user (for example, in a voice saying ā€œyou cannotā€ on smart speakers)
  • current player states (end of queue, end of playlist, depending on quite a few options within the player and the device)
  • media availability due to licensing, country, external and internal providers. You need fallbacks for that.
  • partner integrations (e.g. running on various brands of smart TVs will have their own slightly different issues with UI, API calls, displaying data on screen etc.)
  • and about 100 more things that are getting more and more specific

All of this, of course needs to conform to brand guidelines and to platform guidelines and behaviours (the look and feel on smart TVs with navigation via remote will be different from mobile devices).

All of this, of course, needs to be retrieved from or sent to any number of internal Spotify services (even if some of them are not documented you can at least look at the source code) and partner-provided APIs (good luck finding docs for some of the devices).

Thereā€™s also A/B testing, metrics, experiments, white/balck-listing of clients models and platforms across various services for certain things etc.

And thatā€™s just for skip button.

ā€” end long storyā€”

The absolute primitive stuff like ā€œgive me a media control strip using CSSā€ will definitely work, and will even give you working code (tried it with ChatGPT after Iā€™ve already implemented it for an internal project). That you still need to spend several hours on because it conflicts with the rest of the UI and still needs to be hooked up to everything. Edit: Copilot was no better, on this project. It worked maybe twice, and only once it actually produced a valid result (which did save a few seconds of tedious work).

There will be no massacre.

3 Likes

exactly. I do not say, that we will be replaced, but we will work differently. We will work in dialog with a bot.
The massacre Iā€™m talking about, the victims will not be people but companies. Those that do not adapt.
You (working in tech) can not imagine how conservative and backwards the building industry (at least in Germany) is. Iā€™m seeing the first companies going down right now because of that. Not yet because of AI, but because of bad processes. It will only get more interesting.

3 Likes

ME: imagine you are working for spotify, you are tasked with implementing a skip button. On which parameters will the implementation depend?

chatGPT: As an AI language model, I can give you some general parameters that may be considered when implementing a skip button in Spotify, but please note that the actual implementation decisions made by Spotify may vary depending on their specific goals, priorities, and technical constraints.

Here are some possible parameters that might be considered when implementing a skip button in Spotify:

  1. User behavior and preferences: Spotify may analyze user data, such as skip rate and reasons for skipping songs, to determine the most effective way to implement the skip button. For example, if users are skipping songs because they donā€™t like the genre, Spotify may want to implement a skip button that allows users to skip to a different genre.
  2. Licensing agreements: Spotifyā€™s licensing agreements with music labels and artists may dictate the number of skips allowed per hour or other limitations on the skip feature.
  3. User interface and user experience: The skip buttonā€™s location, size, and appearance could affect its usability and user satisfaction. Spotify may test different designs and interfaces to determine the optimal placement and design for the skip button.
  4. Technical limitations: Spotify may need to consider technical limitations, such as the amount of bandwidth required to stream songs and the amount of memory required to store user preferences and song metadata.
  5. Business considerations: Spotify may consider the impact of the skip button on its business, such as the effect on advertising revenue or the impact on user retention and engagement.

Overall, the implementation of a skip button in Spotify would likely depend on a combination of user behavior and preferences, licensing agreements, user interface and user experience considerations, technical limitations, and business considerations.

Yup. Thatā€™s a high-level overview that you can get for any product out there once youā€™ve worked for a few years.

Now. Thereā€™s the very small tiny insignificant step left to make it work: implementation.

:wink:

1 Like

reporting back on the current state of things after having used Copilot for a bunch of days.

itā€™s interesting that the first thing it did was help me refactor a function. it did it so well! it know the context, what i was trying to do, seemingly even why iā€™m refactoring in the first place.

alas that is the only example iā€™ve gotten that actually saved time.

most of the time it takes more time to use than just writing the code myself, because:

  1. i need to switch into AI mode in my brain and figure out how to order it around. this takes extra time.
  2. the generated code i need to read and think if it makes sense or not, then adjust the prompt and read another set of guesstimates.
  3. sometimes (rarely) when the code does make sense i insert it and then spend another 10minutes debugging it, because thereā€™s a random end statement somewhere extra or the on: part of the join: was missing (too obvious for me to check) or thereā€™s some other obvious mistake Iā€™ve never made typing it myselfā€¦

all in all i dislike it a lot, but am afraid to uninstall because of FOMO :smiley: iā€™m pretty sure this will be a major helper in the future, but right now itā€™s more trouble than itā€™s worth.

3 Likes

Can a collection of globs of lipids and proteins each incapable of understanding anything whatsoever design a db schema?

1 Like

Careful what you wish for :upside_down_face:

Tools like CoPilot and ChatGPT are in their infancy, imagine where theyā€™ll be 5 or 10 years from now. At the very least, is it safe to say they will probably be able to replace junior programmers?

Where will they be 20 years from now? Exponentially more advanced?

A few lawsuits have already launched - itā€™d probably be worth seeing how things pan out (or rather, the arguments presented) as they could affect you or people like you personally, before forming any kind of concrete opinionā€¦

A couple months ago I tried to make ChatGPT implement a rather simple algorithm in Python:

  1. In a data sheet with float numbers, find each row for which col1(i) > col1(i-1) and col1(i) > col1(i+1).
  2. Find each of those for which there is col2(j) > col2(i) and col3(j) < col3(i).
    These were explained in human terms that are not available on the Internet, or are extremely hard to find. It managed to do the first task relatively fast, but even after three hours of trying to implement the second task in a hand-holding way, basically explaining what code to write line-by-line it still couldnā€™t do it. Itā€™s as if it had severe dementia and was forgetting the previous concept straight upon learning the next one. So, Iā€™m quite skeptical about its capabilities outside of what could be solved with regular functions/macros anyways. I also do not think it will see any significant improvement in the future because IMO the low-hanging fruit of expanding the sample sizes for training are already grabbed and whatā€™s in front of these predictive models is abstract thinking - something only available to humans. Maybe the prediction itself could be abstracted away and there is some ā€œlearning meta-patternā€ behind it? If so, it would be a breakthrough much more interesting than token generation.
2 Likes

Well summed, thatā€™s my take as well. People love to anthromorphize technology and introduce mysticism where it absolutely doesnā€™t belong, ignoring huge problems that have to be solved.

That the low-hanging fruit has been picked doesnā€™t mean that the high-hanging fruit will be picked in the next 100 years, or ever. Thatā€™s a crucial element that people keep missing.

ā€œImagine where it will be in 5-10 yearsā€. Itā€™ll be in the history books as a failed experiment is my bet. Or in the best case scenario itā€™ll be described as one the first misguided attempts to achieve a true AI.

3 Likes

Agree hard on unnecessary anthropomorphizing :grinning:. My favorite analogy is that if the water in a river makes a rock move, it doesnā€™t mean the rock has actual locomotion the way animals do. The same way with ā€œAIā€ and ā€œthinkingā€.

But I donā€™t think it will be in the history books as a failed experiment. I think the part of it that could be deservingly called something like ā€œanalytical computationā€ might be of use in some areas. And the Sillicon Valley overhyped marketing side of it ā€“ rather not :grin: .

2 Likes

Yeah, it will very likely be filed under ā€œthis is how we imagined AI at the timeā€, agreed.

The sad thing is that these idiots showed that you can collect data from random sources and generate something that can be made money from. If this continues on, a lot of sites (especially the ones hosting news, stories or with a lot of data) will become closed by nature, as you donā€™t want the article you spent resources to be used by some machine learning spewing some youtube videos (witch by the way already is happening).

1 Like

What I tell my daughter who has started using ChatGPT to generate outlines and brainstorm writing ideas: it wonā€™t replace us in the sense that car operators didnā€™t replace horse riders. It isnā€™t that we will be replaced just that we are raising the bar on what can be accomplished in a certain timeline.

How are we going to get to a Star Trek future unless we get a lot more efficient with education and technology? :slight_smile:

Call me an eternal optimist but I think it is more about increased efficiency. The real issue is how wealth is distributed as the means of production becomes ever more concentrated and I donā€™t see that trend reversing any time soon.

Copilot canā€™t create good code, when it does not have good context.
After a while you know, when proposed code has the chance to be useful, when I know that it can only be garbage, I donā€™t even look at it.

So I normally start to give it some good context. For example I write the moduledoc together with copilot.
Really explaining to it whats going on. Modeling the data-structures used. It is a very good rubber duck. Then I (we) write down an example. Now I see if it understood whats going on. If thats the case, copilot writes at least 80% of the example on its own. It uses the data-structures defined before.

If thats done, I (we) can start coding.
I write the function-header (wording has to be like in the docs). The proposal for the implementation is - most of the time - now useful.

A good and easy way to see how helpful good context is for copilot is refactoring. Take some none-dry code, and see how easy you can DRY it with the help of copilot.

It also helps when you use well-known patterns and data-structures. I currently build sth using libgraph, and it very much helps copilot when I tell it that this and that is a vertex and they connect because of foobar.

Everybody writing this off, or thinking that the same can be done with some snippets is making a mistake imo.

3 Likes

It can be though. Code generation for common patterns has been a thing since forever. IDEA has had simplify, re-write, find duplicates, extract etc. as a part of it tooling also since forever. Also for popular languages :smiley:

So if Copilot is marginally better than existing tools in some situations, great. But herein lies the main problem:

After a while you know, when proposed code has the chance to be useful, when I know that it can only be garbage, I donā€™t even look at it.

It produces too much garbage to be useful for many people. And even ā€œthis code has a chance of being usefulā€ is additional load because you end up reviewing and fixing code instead of writing it (or simply googling it :slight_smile: )

2 Likes

Its not useful for people, that do not invest the time to learn to work with it.
Everybody I know, that invested some time, thinks different.
Iā€™d never want to miss it.
Your loss.

GPT4 just showed, that the potential of transformers is not yet exhausted.

I can now sketch sth on paper and get a good starting point for anything. wow.

1 Like