DaAnalyst
Would be very grateful if someone (even if just a couple of people) who have already used LLM-driven code generation/debugging/design assistance on a large front-end or full stack project in production could share their experience of how useful it is effectively.
Environment expectations:
- The person needs to consider themself a super-senior (15+ years of hand-on production experience) and very experienced in the entire stack used (as in otherwise capable of doing everything by themselves without any LLM assistance).
- The project is in the ballpark of at least 50,000+ (human written) LoC.
- The project is at least relatively complex with a proprietary design of the front-end components and it’s leveraging at least two frameworks (at least one server-side, one client-side; e.g.LiveView + Vue or whatever else), preferably more than two, even if proprietary.
- The project has/had to address several architectural (non-functional) challenges otherwise not supported out-of-the-box by the libraries/frameworks used.
- The project has a fair share of integration b/w at least two languages (e.g. Elixir + JS) either through frameworks or otherwise (i.e. there’s plenty of state management both server-side and client-side as well as exchange between them).
The answers I’m interested fall in the following groups:
- How granular and articulated (as in well-designed) does the “code” in English have to be to achieve the code match to requirements/quality/no bloat comparable to that of super-senior human’s? For granularity I’m referring to what’s addressed in this article by OpenAI: https://openai.com/index/harness-engineering/
- How good is the LLM of choice in leveraging the entire stack and “deciding” on where to apply changes when there’s a change in requirements (e.g. the backend vs in Elixir server-side front-end vs. JS client-side vs CSS/Tailwind)?
- What are your (currently) “definitive” conclusions on which tasks it can be given to solve virtually autonomously (and at what expense in terms of writing detailed behavioral specifications) vs. the types of tasks it’s better to not even consider letting it deal with?
- How beneficial really (quality/time/requirements match -wise) is letting the LLM directly access the code base (and change it) relative to querying it for copy-paste snippets/suggestion in a prompt from where it can’t see the project code base?
- Anything else worth mentioning.
Thank you
Edit: Last but not least, what’s the expected cost of using the LLM of choice 8/hrs a day for the said tasks without hitting the limit?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 3- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
DaAnalyst
UPDATE:
Just yesterday after posting I’ve found about this ultra useful article (and all the stuff and resources it links to): advanced-context-engineering-for-coding-agents/ace-fca.md at main · humanlayer/advanced-context-engineering-for-coding-agents · GitHub
It partially (in a small part) answers my questions:
Besides the loss of privacy (the tradeoff to weigh the potential benefits against), I’m still having issues with the following:
Guaranteed cost vs unsure efficiency over time: What if the (paid-for) failures in doing the tasks correctly while still having to pay for the humans to eventually redo some of them ultimately end up in more total money/time spent than without using the agents?
The author (being obviously very experienced) draws a parallel of not versioning the prompts to not versioning Java source code once the jar’s are packaged.
I have another parallel to that: What about not versioning the models? Are we expected to believe that the new (unversioned) models will continue giving equal results for the same (versioned) prompts over time? Would that not be akin to upgrading a compiler with undeclared (but virtually guaranteed) backward compatibility issues?
The necessity to put accent on repetitive DON’Ts and NEVERs in the prompts is an incredible turn off IMO. It emphasizes the relativity of the expected result. The mere sensation that it may disobey (and it may and frequently does) is very discouraging. How many of those DON’Ts and NEVERs is even enough? Will they still be enough for the future model versions? etc, etc.
All this is why I’d really appreciate someone with no vested interest to answer the questions in my original post. I’d really like to know what’s real here and whether losing the privacy (effectively handing over the codebase to Anthropic, Google or whoever else) is worth it.
al2o3cr
This reminded me of a post from last year:
DaAnalyst
Thanks for the link!