DaAnalyst
Would be very grateful if someone (even if just a couple of people) who have already used LLM-driven code generation/debugging/design assistance on a large front-end or full stack project in production could share their experience of how useful it is effectively.
Environment expectations:
- The person needs to consider themself a super-senior (15+ years of hand-on production experience) and very experienced in the entire stack used (as in otherwise capable of doing everything by themselves without any LLM assistance).
- The project is in the ballpark of at least 50,000+ (human written) LoC.
- The project is at least relatively complex with a proprietary design of the front-end components and it’s leveraging at least two frameworks (at least one server-side, one client-side; e.g.LiveView + Vue or whatever else), preferably more than two, even if proprietary.
- The project has/had to address several architectural (non-functional) challenges otherwise not supported out-of-the-box by the libraries/frameworks used.
- The project has a fair share of integration b/w at least two languages (e.g. Elixir + JS) either through frameworks or otherwise (i.e. there’s plenty of state management both server-side and client-side as well as exchange between them).
The answers I’m interested fall in the following groups:
- How granular and articulated (as in well-designed) does the “code” in English have to be to achieve the code match to requirements/quality/no bloat comparable to that of super-senior human’s? For granularity I’m referring to what’s addressed in this article by OpenAI: https://openai.com/index/harness-engineering/
- How good is the LLM of choice in leveraging the entire stack and “deciding” on where to apply changes when there’s a change in requirements (e.g. the backend vs in Elixir server-side front-end vs. JS client-side vs CSS/Tailwind)?
- What are your (currently) “definitive” conclusions on which tasks it can be given to solve virtually autonomously (and at what expense in terms of writing detailed behavioral specifications) vs. the types of tasks it’s better to not even consider letting it deal with?
- How beneficial really (quality/time/requirements match -wise) is letting the LLM directly access the code base (and change it) relative to querying it for copy-paste snippets/suggestion in a prompt from where it can’t see the project code base?
- Anything else worth mentioning.
Thank you
Edit: Last but not least, what’s the expected cost of using the LLM of choice 8/hrs a day for the said tasks without hitting the limit?
Trending in Questions
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
Hello!
Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app.
I creat...
New
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
So my question is quite simple and i have found no conclusive answer on forum, google or AI.
Should we use :erlang.float for Integer to ...
New
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New
I’m trying to set up Emacs with elixir-ls via lsp-mode and credo via Flycheck. This should mostly be preconfigured as Flycheck picks up c...
New
Other Trending Topics
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Hobbes is a low-level distributed database for the Elixir programming language.
Hobbes provides a simple, safe, and scalable storage lay...
New
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve.
They are GUI (Emerge) and State management (S...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #blog-post
- #ai
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming










Showing Posts 1 to 3- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
DaAnalyst
UPDATE:
Just yesterday after posting I’ve found about this ultra useful article (and all the stuff and resources it links to): advanced-context-engineering-for-coding-agents/ace-fca.md at main · humanlayer/advanced-context-engineering-for-coding-agents · GitHub
It partially (in a small part) answers my questions:
Besides the loss of privacy (the tradeoff to weigh the potential benefits against), I’m still having issues with the following:
Guaranteed cost vs unsure efficiency over time: What if the (paid-for) failures in doing the tasks correctly while still having to pay for the humans to eventually redo some of them ultimately end up in more total money/time spent than without using the agents?
The author (being obviously very experienced) draws a parallel of not versioning the prompts to not versioning Java source code once the jar’s are packaged.
I have another parallel to that: What about not versioning the models? Are we expected to believe that the new (unversioned) models will continue giving equal results for the same (versioned) prompts over time? Would that not be akin to upgrading a compiler with undeclared (but virtually guaranteed) backward compatibility issues?
The necessity to put accent on repetitive DON’Ts and NEVERs in the prompts is an incredible turn off IMO. It emphasizes the relativity of the expected result. The mere sensation that it may disobey (and it may and frequently does) is very discouraging. How many of those DON’Ts and NEVERs is even enough? Will they still be enough for the future model versions? etc, etc.
All this is why I’d really appreciate someone with no vested interest to answer the questions in my original post. I’d really like to know what’s real here and whether losing the privacy (effectively handing over the codebase to Anthropic, Google or whoever else) is worth it.
al2o3cr
This reminded me of a post from last year:
DaAnalyst
Thanks for the link!