Our jobs are safe for now (ChatGPT)

[tfwright]

@max-au

What I am wondering about, would any model (not necessarily GPT) get to a state where we can use it to safely refactor an existing system, with the goal of simplification, cleanup and removal of deprecated stuff?

Since there is no univocal concept of “simple” I think the answer to this has to be no.

[al2o3cr]

Personally, I don’t find the Tetris example convincing - the log shows the generator produced Tetris without specific details about what Tetris is. There are undoubtedly many implementations of Tetris in the training set.

In a situation with more-novel input, ChatGPT hallucinated most of the functions it tried to use:

ChatGPT generated code for eval_string alternative

Was talking with ChatGPT, is the following code a valid/good replacement for Code.eval_string? def create_function(name, args, code) do ast = Macro.escape(code, [:strict]) code = Macro.to_string(ast) {name, Code.string_to_quoted(code)} end This code would run in a template engine of sorts to load function schema/definitions from a json file. defmodule Engine do def start(schema_files) do functions = schema_files |> Enum.map(&parse_schema_file/1) |> Enum.map(&crea…

[dimitarvp]

@max-au

What I am wondering about, would any model (not necessarily GPT) get to a state where we can use it to safely refactor an existing system, with the goal of simplification, cleanup and removal of deprecated stuff?

I’d get half-dead drunk the day this becomes a reality.

But I have a feeling I am safe from going to hospital from drinking

Because that thing is like, I don’t know, 50% of the way to a general / true AI with self-consciousness and self-optimization subroutines.

[dimitarvp]

@tfwright

Since there is no univocal concept of “simple” I think the answer to this has to be no.

I am more optimistic. A true AI would (IMO) be borne out of a hyper-self-optimization algorithm. Being able to minimize / simplify / optimize code – including its own – would be its core tenet.

[al2o3cr]

@max-au

What I am wondering about, would any model (not necessarily GPT) get to a state where we can use it to safely refactor an existing system, with the goal of simplification, cleanup and removal of deprecated stuff?

Some kinds of cleanup (especially around deprecations) can already be automated - for instance, the React community has built an impressive arsenal of code-manipulation tools to deal with API changes:

GitHub - reactjs/react-codemod: React codemod scripts

There’s no AI here, tho, just lots of complicated AST-juggling.

[stevensonmt]

Tangentially related but I was playing with ChatGPT for my day job and asked it to write a letter explaining why a certain medication was needed for a certain disease and to include citations from the medical literature. It provided a reasonably good response for the first part, but the citation it assigned a quote to was to an article on a semi-related topic that never mentioned the medicine in question or the disease in question. So I guess other jobs are still safe too.

[D4no0]

@max-au

Test quality is the best way to figure out seniour from a juniour engineer. In the latter case, if there are tests at all, they are likely trivial “sequence of actions” checking expected outcomes. On the contrary, experienced developers go all the way to property-based tests, modeling (down to temporal models sometimes).

I hardly doubt that, this is more related to a conclusion that senior engineers understand better what they are testing the system for. While it is true that writing tests on some systems is harder than the system itself, a good balance should be always kept. If you are using 80% of your time to write tests instead of creating something new, on a system where you can allow certain errors, then you are not an efficient developer, no matter how many fancy testing techniques you know and can apply.