Expectations for GPT-4?

I’ve been using a ChatGPT plugin for Neovim and have found it to be useful for generating basic Elixir code - genservers, filters, and reducers - better than starting from a blank screen.

Not sure what GPT4 will bring.

Do you know whether this plugin uploads your source code remotely in order to make suggestions or is it completely locally?

I believe the plugin I linked to calls out to the open-ai api using curl.

Maybe you’ll be lucky enough to have that code regurgitated back at you in GPT-4.

My expectations are that many thousands of forum topics (not here in particular but likely here as well) and tweets will appear where people passionately claim that this program “deeply understands” stuff.

In all seriousness, I appreciate these tools’ ability to produce boilerplate so you don’t have to go check how to make a new empty GenServer or such. Beyond that, I have no expectations.

Would love to be proven wrong. Some of my past customers have allowed me to keep old versions of their code – provided I don’t distribute them and only use them for educational purposes, of course – and I’d be curious if I can run some of the “AI” tools on them and have them tell me if there’s a bug in this or that logic. If so, then and only then I’d be actually excited.

Not sure what my own expectations are but was interesting reading Musk’s in this story on DT the other day:

His quote in bold:

On Friday morning, Elon Musk responded to a comment on his favorite website about OpenAI, the research laboratory-turned-company that he co-founded along with a number of other tech luminaries in 2015. Following up on a tweet by finance writer Genevieve Roch-Decter, Musk wrote “OpenAI ws created as an open source… non-profit company to serve as a counterweight to Google, but now it has become a closed source, maximum-profit company effectively controlled by Microsoft. Not what I intended at all.”

My expectation (and probably beyond v4) is that inside VSCode, while I’m working on a module, there’s a side tab, that is giving me all sorts of information based on cursor position:

• [compute] This function is log O(n), n being number of products
• [compute] This function calls a database 3 times.
• [compute] This function call these external services: …
• [docs] Here’s the docs for all the functions you are calling.
• [docs] Here’s the docs for things you might need.
• [popular examples] You are using Enum.reduce, here is how it’s used in popular projects.
• [broad examples] Seems like you are trying to ***, here are some examples of how to do it:

Not necessarily AI-driven, but wishful-driven:

• [local refs] This function is being called by:
  – directly —
  – indirectly: —
  (and for each one of those, probable type clashes)
• [local examples] YourProject.make_things_happen is being used like this:

That’s what I would call a true copilot.

I haven’t been looking into AIs much, but to me the first thing they will be able to automatically do (and better than humans) will be generating comprehensive test suites from a codebase.

I have tested it and I have got back “template” or “boilerplate” code. Not bad at all.
For example, i asked “simple Erlang nif in rust for elixir” and GPT4 gave me an answer that I need Rust, rustler, create a rustler module with mix, (simple) Rust code, an Elixir module and iex code to test. All with links and code, easily to be copied.

So overall not bad and much better than GPT-3.5.

Atm it is not a “danger” for developers: Architecture, special requirements etc. is still with us. But as a first code example within a minute (with ChatGPT plus) it is a good help. And easier than to go through several links in Github and blogs.

Hmm, this would be a dream tool for developers
But I think GPT-4 can’t do that because of tokenization, on small projects (it’s somewhat possible but anything beyond 40ish files is out of GPT4-32k context capability)

Overall, sounds like an interesting project to work on
And regarding space-time complexity well halting problem but I’ve seen GPT-3.5 have a good guess on Elixir code and big O so maybe an estimation giver