DaAnalyst
Questions re experience in using LLM's on non-trivial (non-"flat"), frameworked, large user base projects in production
Would be very grateful if someone (even if just a couple of people) who have already used LLM-driven code generation/debugging/design assistance on a large front-end or full stack project in production could share their experience of how useful it is effectively.
Environment expectations:
- The person needs to consider themself a super-senior (15+ years of hand-on production experience) and very experienced in the entire stack used (as in otherwise capable of doing everything by themselves without any LLM assistance).
- The project is in the ballpark of at least 50,000+ (human written) LoC.
- The project is at least relatively complex with a proprietary design of the front-end components and it’s leveraging at least two frameworks (at least one server-side, one client-side; e.g.LiveView + Vue or whatever else), preferably more than two, even if proprietary.
- The project has/had to address several architectural (non-functional) challenges otherwise not supported out-of-the-box by the libraries/frameworks used.
- The project has a fair share of integration b/w at least two languages (e.g. Elixir + JS) either through frameworks or otherwise (i.e. there’s plenty of state management both server-side and client-side as well as exchange between them).
The answers I’m interested fall in the following groups:
- How granular and articulated (as in well-designed) does the “code” in English have to be to achieve the code match to requirements/quality/no bloat comparable to that of super-senior human’s? For granularity I’m referring to what’s addressed in this article by OpenAI: https://openai.com/index/harness-engineering/
- How good is the LLM of choice in leveraging the entire stack and “deciding” on where to apply changes when there’s a change in requirements (e.g. the backend vs in Elixir server-side front-end vs. JS client-side vs CSS/Tailwind)?
- What are your (currently) “definitive” conclusions on which tasks it can be given to solve virtually autonomously (and at what expense in terms of writing detailed behavioral specifications) vs. the types of tasks it’s better to not even consider letting it deal with?
- How beneficial really (quality/time/requirements match -wise) is letting the LLM directly access the code base (and change it) relative to querying it for copy-paste snippets/suggestion in a prompt from where it can’t see the project code base?
- Anything else worth mentioning.
Thank you
Edit: Last but not least, what’s the expected cost of using the LLM of choice 8/hrs a day for the said tasks without hitting the limit?
Most Liked
al2o3cr
Last Post!
DaAnalyst
Popular in Questions
Hi,
I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum.
...
New
lets say i have a sample like
a = 20; b = 10;
if (a > b) do
{:ok, "a"}
end
if (a < b) do
{:ok, b}
end
if (a == b) do
{:ok, "equa...
New
Hi everyone!
I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Hello!
Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: )
Hello all, this is ...
New
Hello all!
I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Other popular topics
Hi there,
I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
Lets say I have map like this fetching from my database
%{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
Hey,
Just curious what are the main benefits of Elixir compared to Clojure?
When is Elixir more useful than Clojure and vice versa?
Th...
New
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex









