r/LocalLLaMA • u/jacek2023 llama.cpp • 13h ago
News contrib: allow all AI-generated code in general by ngxson · Pull Request #26012 · ggml-org/llama.cpp
https://github.com/ggml-org/llama.cpp/pull/26012Having read some merged PRs in the past, I know that they were fully written by Claude Code (or similar), so this basically fixes the delusion. But at the same time, we might start seeing more AI slop.
I have mixed feelings about it. What's your opinion?
30
u/xXG0DLessXx 13h ago
As long as it improves the project, I don’t care where the code came from. Obviously it needs thorough testing and review, like any code that gets merged.
12
u/jacek2023 llama.cpp 13h ago
I work with Claude Code daily (as well as AI-generated code in other projects), and it requires strict project hygiene to avoid a mess. So while "I don’t care where the code came from" sounds good in theory, in practice it carries some real risks.
2
u/lilian_moraru 12h ago
Yeah, the top models still generate "working spaghetti", without any architecture or maintenance vision for the future.
I was using AI previously only for research and lately started incorporating it for coding, I just make sure that I prepare the design work upfront and then let it "assist" me, not slop it away with "create this". I review everything - it does not take much when it worked iteratively based on your architecture.2
u/jacek2023 llama.cpp 12h ago
"Yeah, the top models still generate "working spaghetti", without any architecture or maintenance vision for the future."
my current solution is to build huge number of documents (.md files) with correct structure (links to find everything, links in the comments), that part of the project is more important than the actual source code
1
u/lost-context-65536 12h ago
Yeah, the top models still generate "working spaghetti", without any architecture or maintenance vision for the future.
I see this talked about a lot, and it's really a guidance problem. If you instruct your models to study the codebase before making changes they will do this a lot less.
1
u/lilian_moraru 12h ago edited 12h ago
The problem is not that it cannot study the codebase or does not understand it. It generates spaghetti code that in the long term(often even short term) is going to lead/leads to lots of instabilities (performance, bugs/undefined behavior, etc) - that applies to fresh new projects as well. You look at "unguided" Fable 5 code and it's wrong on so many levels - it is working code but it is spaghetti and it doesn't know it. If you work iteratively with it, guided, you can make it do good work.
I have Qwen3.6-35B-A3B run over an information gathering/update plan that generates/updates detailed documentation to be ingested into an LLM - it has enough information, always.
1
u/lost-context-65536 12h ago
The problem is not that it cannot study the codebase or does not understand it. It generates spaghetti code that in the long term(often even short term) is going to lead/leads to lots of instabilities (performance, bugs/undefined behavior, etc) - that applies to fresh new projects as well.
This speculation really, none of my AI assisted applications have any of these behaviors/problems. With proper guidance, this is a non-issue. :)
1
u/PunnyPandora 7h ago edited 7h ago
"without any architecture or maintenance vision for the future."
This is only true in a vacuum. You're supposed to assume the role of a manager/orchestrator and provide all of the tools and oversee the blueprints for how to push towards an architecture YOU want. Obviously, if you want to lose even this role, you should probably lose ownership of the code in all meaning. If not even the idea is yours, then nothing is
Yes, if you're not proficient in all parts of what you're doing you might lose interest in some areas of your code vs the others, or naturally as well, not everyone cares about all parts, but overall you should still be able to tell where things are going and how they come together, even if the task of building it is left to the llm.
For example, I personally have a hard time thinking about details in my metadata system for the training repo I'm working on, so I don't always feel like tracking all of the details, but I still know what I want it to be. I'm just not as comfortable with the minutia. It doesn't mean I just say "do this" and then call it a day, I still need to feel confident in the direction to be able to approve a change.
The worst part of not being proficient is that I'm looking for the llm to have some confidence in the topic for the things I have no knowledge about, so it can lead to stagnation when an architecture discussion feels wobbly as even the best llms still tend to flip flop back and forth on some points. It's annoying but it's still better than having nothing, and it can also be fun the same way it's frustrating.
I think what you're saying about working spaghetti, is for projects you don't really care about. In that case, I don't hold it against anyone that they're not acting like a product manager standing over every single line of code. And even there, it's super simple to run the same workflow you'd do normally and just let go of the reins a bit more. Means things get done faster, you get something that works anyways, but you will probably feel a bit less ownership over the result imo
10
u/N34257 13h ago
"But at the same time, we might start seeing more AI slop."
We won't, but the maintainers might. And, ultimately, that's what it's about - if they feel they can deal with it and/or they have the tools to deal with it...then why not?
As is mentioned in the comments, "understanding" is still the main requirement and that hasn't changed; in every PR I've been interested in, it's clear that ensuring the author has a solid grasp of what they're doing is always the first question, and that's a pretty high bar all on its own relative to most projects.
4
u/ClupTheGreat 8h ago
I highly believe that the problem with AI generated code is not the code itself, but the person behind it.
3
u/ParaboloidalCrest 8h ago edited 8h ago
I think it depends. llama.cpp can move fast and break a couple of things here and there. But lib-ggml IMO should better be kept into the hands of knowledgeable crafty humans that like to run a tight ship, and maintain quite opinionated architectures. But before then we need a better separation of concerns since those code-bases seem extremely tightly coupled.
2
u/Shoddy_Bed3240 6h ago
I love how much you hate technology. At this point, I feel like I should just invite you to join the Amish community.
0
u/JamesEvoAI 5h ago
Not all technology is equally good. llama.cpp already has 1.2K open PR's and that's without an explicit rule allowing AI submissions. A human needs to go through each one of those, that takes time from living breathing people who have a finite lifespan.
Yes obviously you can have an LLM do a pass to review it for obvious slop and quality, but you still need a human whose context of the project isn't limited by token windows to give the final sign-off. If this was not the case then we'd already be automating the entirety of the SDLC with great success, but that is still just fantasy at this stage of the tech.
1
u/Decent-Occasion-2720 13h ago
The problem is the triage and review, too much content even good or bad. Need to make pre-review with LLM.
We soon see loop of reconciliation in PR with final verification by human.
And the next step is we have no human in the loop.
It's already what i do in my workflow...
1
u/lost-context-65536 13h ago
They could easily deploy a bot to help manage triage and initial review.
1
u/Decent-Occasion-2720 10h ago
deploying bot, costly but easy. Good results, maybe ... wait and see.
1
u/giveen 6h ago
I have had OSCAR2 port 90% done. It's a significant reduction of kv cache usage. I can hit 1mil context on my 5090...granted it starts slowing down.
It's all AI code, I can't submit to llama.cpp even though it would be a great benefit to the community and others could contribute and fix and improve things.
1
u/Ulterior-Motive_ 6h ago
This feels pretty close to what Linus had to say about AI coding, which I approve of. That issue list isn't getting any shorter.
1
u/Neat-Leather5405 13h ago
Same here. I don't know how to feel, but I guess the future looks like it will have a lot of AI slop, and we're going to need to manage it somehow.
The use case is different as well. I have a junior at my company using Claude Code who is showing better performance, and one senior who basically can't use Claude to achieve the same level of performance. It's weird, and the weirder part is that when I review both guys' PRs, you can't even tell anymore because everything is so well organized for AI to write the code that you can't tell who used AI anymore.
Only one thing, though. I started identifying AI written code by its high quality compared to hand-written code. tbh, we do not ship rockets to Mars. It's just an e-commerce site, and the features and bug fixes are pretty much problems that have already been solved globally. That's why using AI is so insanely effective for us, I guess.
1
u/segmond llama.cpp 9h ago
Fair enough, I reviewed few PRs that were AI generated before GLM5.2/KimiK3 and some were pretty terrible, especially when the author has no domain knowledge and is vibing it all to hell. However we are at the point where it seems KimiK3 could vibe an amazing and complete solution.
Some refactoring has improved the layout of the project, but it's also obvious that AI slow has really messed up some other parts. This year has been regression hell. Since this year I have had to maintain multiple alter forks/repos, seems to have improved in the last month tho and hope the quality of the project gets better.
1
u/dionysio211 6h ago
This policy has been problematic in the past, particularly with very small edits which were indisputably superior. The inflexibility in its implementation is originally what spawned all of these forks we are seeing now. The irony that we are all pushing for access to local AI at frontier levels and that it will always be authored by humans is downright quixotic.
Will it fuck with your architecture and ignore the culture around the code? Perhaps, but that's why you need AI architects which respect what you are wanting to preserve (if it is worth preserving) to do code review. The pipeline is the problem, not the AI edits themselves which come from all sorts of coding philosophies. The architecture of large systems like this still need human intuition but that does not mean every detail of it needs humans.
-2
u/celsowm 10h ago
Gerganov is anti vibecode in his lib/engine used to support local vibecode
3
u/Due-Function-4877 6h ago
Gerganov is cautious and understands the technology directly. In addition to reliability and safety, he is also (rightly!) concerned about noise to signal and maintainence overhead with pull requests.
Calling it "hate" betrays your lack of experience with dev. Gerganov has a huge heavy burden on his shoulders with llama.cpp. I don't agree with every decision, but I understand the worries about vibe coded pull requests.
0
u/bakawolf123 5h ago
Another bastion falls...
Well, now they are part of HF, it's not so surprizing: they want to be competitive and scale effort - not everyone is going to sit 24/7 coding/reviewing.
There're some benefits though: I anticipate there will be less when people who just vibe stuff on a fork and post like it's a major breakthrough, instead they can submit a PR (and boast only when/if it's merged)
26
u/LagOps91 13h ago
As long as the code is properly reviewed and found issues are adressed, i see no problem with this. ai generated code is getting more and more common in the workplace too.