The article very much resonates with my experience past several months.
The project I work on has been steadily growing for years, but the amount of engineers taking care of it stayed same or even declined a bit. Most of features are isolated and left untouched for months unless something comes up.
So far, I managed growing scope by relying on tests more and more. Then I switched to exclusively developing against a simulator. Checking changes with real system become rare and more involved - when you have to check, it's usually the gnarliest parts.
Last year's, I noticed I can no longer answer questions about several features because despite working on those for a couple of months and reviewing PRs, I barely hold the details in my head soon afterwards. And this all even before coding agents penetrated deep into our process.
With agents, I noticed exactly what article talks about. Reviewing PR feels even more implicit, I have to exert deliberate effort because tacit knowledge of context didn't form yet and you have to review more than before - the stuff goes into one ear and out of another. My team mates report similar experience.
Currently, we are trying various approaches to deal with that, it it's still too early to tell. We now commit agent plans alongside code to maybe not lose insights gained during development. Tasks with vague requirements we'd implicitly understand most of previously are now a bottleneck because when you type requirements to an agent for planning immediately surface various issues you'd think of during backlog grooming. Skill MDs are often tacit knowledge dumps we previously kept distributed in less formal ways. Agents are forcing us to up our process game and discipline, real people benefit from that too. As article mentioned, I am looking forward to tools picking some of that slack.
One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate. It's as if the concept was alien to them or they could comprehend that other people handle that at different capacity than them.
> One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate.
Engineering managers in my experience (even in ones with deep technical backgrounds) often miss the trees for the forest. The best ones go to bat for you, especially once verifying that they can do something to unblock or support you. But that’s still different than being in the terminal or IDE all day.
Offloading cognitive load is pretty much their entire role.
I think that recording dialog with the agent (prompt, the agent's plan, and agent's report after implementation) will become increasingly important in the future.
You will also add a markdown file to the changelog directory named with the current date and time `date -u +"%Y-%m-%dT%H-%M-%SZ"`, record the prompt, and a brief summary of what changes you made, this should be the same summary you gave the developer in the chat.
From that I get the prompt and the summary for each change. It's not perfect but it at least adds some context around the commit.
Isn’t the commit message a better place to add what and why? You might need to feed some info that the agent doesn’t have access to “we are developing feature X this change will such and such to blah blah”. The agent will write a pretty good commit message most of the times. Why do you need a markdown file? Are releasing new versions of the software for third parties?
Agree, but current agents don't help with that. I use Copilot, and you can't even dump it preserving complete context, including images, tool call results and subagent outputs. And even if you could, you'd immediately blow up the context trying to ingest that. This needs some supporting tooling, like in today's submission where agent accesses terabytes of CI logs via ClickHouse.
I've had some luck creating tiny skills that produce summaries. E.g. a current TASK.md is generated from a milestone in PLAN.md, and when work is checked in STATUS.md and README.md are regenerated as needed. AGENTS.md is minimal and shrinking as I spread instructions out to the tools.
Part of my CI process when creating skills involves setting token caps and comparing usage rates with and without the skill.
"The system they built feels slightly foreign even as it functions correctly." This is exactly the same issue that engineers who become managers have. You are further away from the code; your understanding is less grounded, it feels disconnected.
When software engineers become agent herders their day-to-day starts to resemble more that of a manager than that of an engineer.
exactly, as a manager and a sometimes a developer, "vibe-coding" has been looking more and more as my day job (in a good way, it's good to not have to do all the dirty work for your pet projects) and it's all about having the same discipline in term of:
* thinking about the big picture
* knowing how you can verify that the code match the big picture.
In both case, somtimes you are happily surprised, sometimes you discover that the things you told 3 times the one writing code to do was still not done.
My team has experienced this over the past 6 months for sure.
The core of the article is “ AI-assisted development potentially short-circuits this replenishment mechanism. If new engineers can generate working modifications without developing deep comprehension, they never form the tacit knowledge that would traditionally accumulate. The organization loses knowledge not just through attrition but through insufficient formation.”
But is it possible this phenomenon is transient?
Isn’t part of the presumed value add of LLM coding agents in the meta-realm around coding; e.g. that well-structured human+LLM generated code (green field in particular) will be organized in such a way that the human will not have to develop deep comprehension until needed (e.g. for bug fix/optimization) and then only for a working set of the code, with the LLM bringing the person up to speed on the working set in question and also providing the architectural context to frame the working set properly?
In my view with current LLMs: they still produce far too much bloat and unclean solutions when not targeting them at very specific issues/features, making LLMs essentially a requirement for any debugging or features for the lifecycle of the product/service.
The whole premise of the post, that coders remember what and why they wrote things from 6 months ago, is flawed.
We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."
I don't remember exactly what I wrote and how the logic works, but I generally remember the broad flow of how things tie together, which makes it easier to drop in on some aspect and understand where it is code-wise.
There's code structure but then there's also code philosophy.
The worst code bases I have to deal with have either no philosophy or a dozen competing and incompatible philosophies.
The best are (obviously) written in my battle tested and ultra refined philosophy developed over the last ~25 years.
But I'm perfectly happy to be working in code bases written even with philosophies that I violently disagree with. Just as long as the singular (or at least compatible) philosophy has a certain maturity and consistency to it.
It feels like that at first, especially as I get older. But I still think it comes back to me a lot quicker if I once understood it than if I was learning it from scratch. Possibly just because I know how I think.
These feelings aren't mutually exclusive. I'm often like "I have no memory of this place" while my name stares at me from git blame, but that doesn't mean my intuition of how it's structured isn't highly likely to be right in such cases.
I definitely understand my own code better than what other people wrote, even from 10 years ago. I often see code and think "this makes sense to do it this way". Turns out I wrote it years ago.
I find this to be the case if it was something I was deeply involved with.
Other times, I can make a small change to something that doesn't require much time, and once it's tested and committed, I quickly lose any memory of even having done it.
Yeah I did pour a lot of sweat and thinking into that codebase all those years ago.
When I do a drive-by edit, I probably don't remember it in a week.
Which is why the "cognitive debt" from the article is relevant, IMHO. If I just thoroughly review the plan and quickly scan the resulting code, will that have a strong enough imprint on my mind over time?
I would like to think "yes", my gut is telling me "no". IMHO the LLMs are now "good enough" for coding. These are hard questions we'll have to grapple with this in the next year or two (in context of AI-assisted software development).
I juggle between various codebases regularly, some written by me and some not, often come back to things after not even months but years, and in my experience there's very little difference in coming back to a codebase after 6 months or after a week.
The hard part is to gain familiarity with the project's coding style and high level structure (the "intuition" of where to expect what you're looking for) and this is something that comes back to you with relative ease if you had already put that effort in the past - like a song you used to have memorized in the past, but couldn't recall it now after all these years until you heard the first verse somewhere. And of course, memorizing songs you wrote yourself is much easier, it just kinda happens on its own.
I still remember the core architecture of code I wrote 20 years ago at my first job. I can visualize the main classes and how they interact even though I haven’t touched it since then.
Meanwhile some stuff Claude wrote for me last week I barely remember what it even did at a high level.
If I’m learning for the first time, I think it matters to hand code something. The struggle internalizes critical thinking. How else am I supposed to have “taste”? :)
I don’t know if this becomes prod code, but I often feel the need to create like a Jupyter notebook to create a solution step by step to ensure I understand.
Of course I don’t need to understand most silly things in my codebase. But some things I need to reason about carefully.
Almost anything I write in Python I start in jupyter just so I can roll it around and see how it feels- which determines how I build it out and to some degree, how easy it is to fix issues later on.
Harder here doesn’t mean slower. Reading and understanding your own code is way faster than writing and testing it, but it’s not easy.
AI tools don’t prevent people from understanding the code they are producing as it wouldn’t actually take that much time, but there’s a natural tendency to avoid hard work. Of course AI code is generally terrible making the process even more painful, but you where just looking at the context that created it so you have a leg up.
Certainly AI tools don't prevent anything per se, that's management's job. Deadlines and other forms of time pressure being what they are it's trivial to construct a narrative where developers are producing (and shipping) code significantly faster than the resulting codebase can be fully comprehended.
This is also an area where AI can help. Don't just tell it to write your code. Before you get going, have it give you an architectural overview of certain parts you're rusty on, have it summarize changes that have happened since you were familiar, have it look at the bigger picture of what you're about to do and have it critique your design. If you're going to have it help you write code, don't have it ONLY help you write code. Have it help you with all the cognitive load.
It’s hard to keep the minutiae in your memory over a long period of time - but I certainly remember the high level details. Patterns, types, interfaces, APIs, architectural decisions. This is why I write comments and have thorough tests - the documentation of the minutiae is critical and gives guardrails when refactoring.
I absolutely feel the cognitive debt with our codebase at work now. It’s not so much that we are churning out features faster with ai (although that is certainly happening) - but we are tackling much more complex work that previously we would have said No to.
OP talks about the increased frequency of such events happening, and not that this is a new problem.
For example, handwritten code also tended to be reviewed manually by each other member of the team, so the probability of someone recalling was higher than say, LLM generated code that was also LLM reviewed.
In the past, it was also an optimistic assumption that your engineers would still be working for you in a year's time? You need some kind of documentation / instructive testing anyway. And maybe more than one person who understands each bit of the system (bus factor).
Oh come on, that is complete nonsense. I can reunderstand complicated code I wrote a year ago far, far faster than complicated code someone else wrote. Especially if I also wrote tests, accompanying notes, and docs. If you can't understand your old code when you come back to it... including looking through your comments and docs and tests... I'm going to say you're doing it wrong. Maybe it takes a while, but it shouldn't be that hard.
Anyone pretending gen-ai code is understood as well as pre-gen-ai, handwritten code is totally kidding themselves.
Now, whether the trade off is still worth it is debatable, but that's a different question.
The trade-off is worth it in my opinion when you are in a time crunch to deliver a demo, or are asked to test out an idea for a new feature (also in a time crunch).
The hope being that if the feature were to be kept or the demo fleshed out, developers would need to shape and refactor the project as per newly discovered requirements, or start from scratch having hopefully learnt from the agentic rush.
To me, it always boils down to LLMs being probabilistic models which can do more of the same that has been done thousands of times, but also exhibit emergent reasoning-like properties that allow them to combine patterns sometimes. It's not actual reasoning, it's a facsimile of reasoning. The bigger the models, the better the RLHF and fine-tuning, the more useful they become but my intuition is that they'll always (LLMs) asymptotically try to approach actual reasoning without being able to get there.
So the notion of no-human-brain-in-the-loop programming is to me, a fool's errand. I do obviously hope I am right here, but we'll see. Ultimately you need accountability and for accountability you need human understanding. Trying to move fast without waiting for comprehension to catch up (which would most likely result in alternate, better approaches to solving the problem at hand) increases entropy and pushes problems further down the road.
Also the article is AI written itself or AI assisted - there’s a tendency in AI text to bloviate and expound on irrelevant stuff so as to lose the plot
AI spec docs and documentation also have this documentation problem
A lot of bug fixing relies on some mental model about the code. It manifests as rapid "Oh 100% I know what's causing" -heureka moments. With generated code, that part's gone for good. The "black box written by a black box" is spot on on, you're completely dependent on any LLM to maintain the codebase. Right now it's not a vendor lock thing but I worry it's going to be a monopoly thing. There's going to be 2-3 big companies at most, and with the bubble eventually bursting and investor money dying, running agents might get a lot more expensive. Who's going to propose the rewrite of thousands of LLM-generated features especially after the art of programming dies along with current seniors who burn out or retire.
I’m very confused by this statement. I routinely answer questions about why we wrote the code we wrote 6 months ago and expect other people to do the same. In my mind that skill is one of the key differences between good and bad developers. Is it really so rare?
I have been laboriously going through the process of adding documentation and comments in code explaining the purpose and all the interfaces we expect and adding tests for the purpose of making it easier for claude to work with it but it also makes it easier for me to work with it.
Claude often makes a hash of our legacy code and then i go look at what we had there before it started and think “i don’t even know what i was thinking, why is this even here?”
Not to disagree with anything the article talks about but to add some perspective...
The complaint about "code nobody understands" because of accumulating cognitive debt also happened with hand-written code. E.g. some stories:
- from https://devblogs.microsoft.com/oldnewthing/20121218-00/?p=58... : >Two of us tried to debug the program to figure out what was going on, but given that this was code written several years earlier by an outside company, and that nobody at Microsoft ever understood how the code worked (much less still understood it), and that most of the code was completely uncommented, we simply couldn’t figure out why the collision detector was not working. Heck, we couldn’t even find the collision detector! We had several million lines of code still to port, so we couldn’t afford to spend days studying the code trying to figure out what obscure floating point rounding error was causing collision detection to fail. We just made the executive decision right there to drop Pinball from the product.
This underlines the argument of the OP no?
The argument presented is that the situation where nobody knows how and why a piece of code is written will happen more often and appear faster with AI.
> The engineer who pauses to deeply understand what they built falls behind in velocity metrics.
This is the most insidious part. It's not even that bad code gets deployed. That can be fixed and hopefully (by definition) the market weeds that out.
The problem is that the market doesn't seem to operate like that, and instead the engineer who cares loses their job because they're not hitting the metrics.
Of course, there are counter examples but there's a disconnect between the production of something and the selling of it with almost opposing goals. Given unlimited money and time, many engineers, arts, etc will write and rewrite something to perfection. Constraints are needed because the world doesn't operate in a vacuum and unless we all live in a utopia, we have to compete for customers and resources.
Constraints often result in better results. Think of Duke Nukem Forever and how long it took them to release a nothingburger.
I just watched a show called the Knight of the Seven Kingdoms and the showerunners were given a limited budget compared to their cousin shows and it resulted in a better product.
> Given unlimited money and time, many engineers, arts, etc will write and rewrite something to perfection
This is a common trope, but in my experience many engineers I met know that's not how a business runs. Dealing with the constraints and weighing them out is one of the essential skills of any engineer. Knowing when a product is just good enough is one of the things that make you senior.
The usual person that believe in the trope is the one that wants something with short-term budget, but with long-term quality. Lot of engineers know the triangle of budget-scope-time. But a lot of managers want to have the cake and eat it too. When they ask to reduce time and the budget and the engineer ask what to reduce in the scope, they get all vague and shifty.
I think we might as well just go all in at this point: "LGTM, LLM". The industry always overshoots and then self-corrects later. Therefore, maybe the right thing to do is help it get to a more sane equilibrium is to forget about the code altogether and focus on other ways to constrain it / ensure correctness and/or determine better ways to know when comprehension is needed vs optional.
What I don't like is the impossible middle ground where people are asked to 20X their output while taking full responsibility for 100% of the code at the same time. That is the kind of magical thinking that I am certain the market will eventually delete. You have to either give up on comprehension or accept a modest, 20% productivity boost at best.
The productivity boost entirely depends on the way the software was written.
Brownfield legacy projects with god classes and millions of lines of code which need to behave coherently across multiple channels- without actually having anything linking them from the written code? That shit is not even gonna get a 20% boost, you'll almost always be quicker on your own - what you do get is a fatigue bonus, by which I mean you'll invest yourself less for the same amount of output, while getting slightly slower because nobody I've ever interacted with is able to keep such code bases in their mind sufficiently to branch out to multiple agents.
On projects that have been architected to be owned by an LLM? Modular modilith with hints linking all channels together etc? Yeah, you're gonna get a massive productivity boost and you also will be using your brain a shitton actually reasoning things out how you'll get the LLM to be able to actually work on the project beyond silly weekends toy project scope (100k-MM LOC)
But let's be real here, most employees are working with codebases like the former.
And I'm still learning how to do the second. While I've significantly improved since I've started one year ago, I wouldn't consider myself a master at it yet. I continue to try things out and frequently try things that I ultimately decide to revert or (best case) discard before merge to main simply because I ... Notice significant impediments modifying/adding features with a given architecture.
Seriously, this is currently bleeding Edge. Things have not even begun to settle yet.
We're way too early for the industry to normalize around llms yet
“Seriously, this is currently bleeding Edge. Things have not even begun to settle yet.
We're way too early for the industry to normalize around llms yet”
That’s the exciting part isn’t it? This stuff is completely uncharted water. The interesting thing is there is huge forests of low hanging fruit to harvest too.
The interesting thing is this tickles all the same things “actual programming” does. It’s just the programming language changed to precise English and while the fundamental CS constrains are the same there is a new layer of constraints. Some, like speed of inference, I suspect will be solved (which will be incredible) but others like context management and size will always exist for LLMs. Understanding how little adjustments to what you prompt them can have dramatic changes to their output will never go away. Knowing what to put into context is a hugely important skill.
While I too am only seeing a boost on the order of 20% so far, I think there are more creative applications of LLM beyond writing code, that can unlock multiples of net productivity in delivering product end to end. People are discovering these today and blogging about them, but the noise about dark factories and agents supervising agents supervising agents, etc, is drowning out their voices.
Every one of us is a pioneer if we choose to be. We have only scratched the surface as an industry.
Yes exactly, of all the uses cases for LLMs "writing code" is easily my least favorite. Theres so many other cool things for "stochastic contextual orchestrators"
The problem with this is when something breaks and your manager says “why haven’t you figured it out yet” as you spend hours digging into the 200 PRs of vibe slop that landed in the past day.
Now you could say that expectation has to change but I don’t see how—the people paying you expect you to produce working software. And we’ve always been biased in favor of short term shipping over longer term maintainability.
Richard Gabriel wrote a famous essay Worse Is Better (https://www.dreamsongs.com/WorseIsBetter.html). The MIT approach vs the New Jersey approach does not necessarily apply to the discussion of the merits of coding agent, but the essay's philosophy seems relevant. AI coding sometimes sacrifices correctness or cleanness for simplicity, but it will win and win big as long as the produced code works per its users' standards.
Also, the essay notes that once a "worse" system is established, it can be incrementally improved. Following that argument, we can say that as long as the AI code runs, it creates a footprint. Once the software has users and VC funding, developers can go back and incrementally improve or refactor the AI's mess, to a satisfying degree.
I hope people can ask themselves why the goal is "winning" and "winning big", and not making a product that you are proud of. It shouldn't be about VC funding and making money, shouldn't we all be making software to make the world a little bit better? I realize we live in an unfortunate reality surrounded by capitalism, but giving in to that seems shortsighted and dismissive of actual problems.
“Winning” is just the the subjective word I quickly picked. It certainly can be another one, such as the success due to a great product as you mentioned
You might lose context of a specific project over time, but not of the language itself. When you're no longer involved with the project's implementation or the programming language itself, what remains?
> When an engineer writes code manually, two parallel processes occur. The first is production: characters appear in files, tests get written, systems change. The second is absorption: mental models form, edge cases become intuitive, architectural relationships solidify into understanding.
That absorption only takes place in the mind of that individual, unfortunately. That doesn't help when they no longer work there or are on vacation.
The ideal situation is the solo open source project. You wrote all 200K lines of code yourself, and will maintain them until death. :)
It used to take years, decades, or centuries before a system could grow and evolve to be so complex and unwieldy, and so full of internal contradictions, that the whole thing becomes an incomprehensible tangle of hairballs. An example is the patchwork system of international, national, regional, and local laws we have at present, which has grown and evolved over centuries.
The organizational memory and on-call debugging sections allude to this, but there are significant effects on other parts of the organization. For example, if I work in product support and a customers asks about a products behavior - it becomes much more challenging to find answers if documentation is sparse (or ai written), engineers don’t immediately know the basics of the code they wrote, etc. Even if documentation is great and engineers can discuss their code, the pace of shipping updates can be a huge challenge for other teams to keep up with.
Management will demand that free time goes to more features. Thats the problem. Time spent understand the feature (either while writing it or documenting it) is not valued, only time spent making it. So when making and understanding are decoupled, management will demand you spend all your time making, rather than understanding. They'll just tell you to have the llm make the docs
With the free time gained from the advent of fast food people "should" have started exercising more, but they didn't. As disciplined as you yourself may be, the typical person is going to use AI to expend minimal effort and go home at 4:55 pm.
But to offer a counter argument, would the same thing not have happened with the rise of high level languages? The machine code was abstracted away from engineers and they lost understanding of it, only knowing what the high level code is supposed to do. But that turned out fine. Would llms abstracting the code away so engineers only understand the functionality (specs, tests) also be fine for the same reason? Why didnt cognitive debt rise in with high level languages?
A counter counter argument is that compilers are deterministic so understanding the procedure of the high level language meant you understood the procedure that mattered of the machine code, and the stuff abstracted away wasnt necessary to the codes operation. But llms are probabilistic so understanding the functionality does not mean understanding the procedure of the code in the ways that matters. But id love to hear other peoples thoughts on that
> would the same thing not have happened with the rise of high level languages?
Any argument that attempts to frame LLMs as analogous to compilers is too flawed to bother pursuing. It's not that compilers are deterministic (an LLM can also be deterministic if you have control over the seed), it's that the compiler as a translator from a high level language to machine code is a deductive logical process, whereas an LLM is inherently inductive rather than deductive. That's not to say that LLMs can't be useful as a way of generating high level code that is then fed into a compiler (an inductive process as a pipeline into a deductive process), but these are fundamentally different sorts of things, in the same way that math is fundamentally different from music (despite the fact that you can apply math to music in plenty of ways).
“Programs must be written for people to read, and only incidentally for machines to execute." — Harold Abelson
The purpose of high level languages is to make the structure of the code and data structures more explicit so it better captures the “actual” program model, which is in the mind of the programmer. Structured programming, type systems, modules, etc. are there to provide solid abstractions in which to express that model.
None of that applies to giving an LLM a feature idea in English and letting it run. (Though all of it is helpful for keeping an LLM from going completely off the rails.)
The code for the machine to execute is opcodes realized by electric signals. But it's not a good tool to solve problems even when directly translated to assembly. All the other languages were invented for better human affordance and can map to assembly (and opcodes) with logical rules.
The last part is what matters. There's no such clear rules in LLMs behavior. Yes you can get to behave roughly like a rule, but there's no clear cut demarcation between what's in and what's not.
It did not turn out fine. Fortunately no one took it seriously, and at least seniors still have an intuitive model of how the hardware works in their head. You don't have to "see" the whole assembly language when writing high level code, just know enough about how it goes at lower levels that you don't shoot yourself in the foot.
When that's missing, due to lack of knowledge or perhaps time constraints, you end up on accidentally quadratic or they name a CVE after you.
I think it won't be too different once we see a few upgrades that are going to be required for reliability (and scaling up the AI assisted engineering process):
- deterministic agents, where the model guarantees the same output with a seed
- much faster coding agents, which will allow us to "compile" or "execute" natural language without noticing the llm
- maybe just running the whole thing locally so privacy and reliability are not an issue
We're not there yet, but once we have that then I agree there won't be too much of a difference between using a high level language and plain text.
There's going to be a massive shift in programming education though, because knowing an actual programming language won't matter any more than knowing assembly does today.
I also was having a similar thought, and think you wrote the answer I could not put my finger on.
Compilers are deterministic, AI is a stochastic process, it doesn't always converge exactly to the same answer. Here's the main difference
Yes my hot take is that the real risk isn't skill atrophy... it's failing to develop the new skill of using AI. It's all abstraction layers anyway and people always lament the next abstraction up.
0/1s → assembly → C → high-level languages → frameworks → AI → product
The engineer keeps moving up the abstraction chain with less and less understanding of the layers below. The better solution would be creating better verification, testing, and determinism at the AI layer. Surely we'll see the equivalent of high-level languages and frameworks for AI soon.
I wrote a SaaS project over the weekend. I was amazed at how fast Claude implemented features. 1 sentence turned into a TDD that looked right to me and features worked
but now 3 weeks later I only have the outlines of how it works and regaining the context on the system sounds painful
In projects I hand wrote I could probably still locate major files and recall system architectures after years being away
I have been in a big company for 4 years, and following the zillions of projets going on here and there, how they interact [nicely or not] has become a job in itself.
Very disturbing as I thought my technical skills would help me clarify the global picture. And that is exactly the contrary that is happening.
I was at a company with one (complex) product and joined a company 10x large with 50x as many products - there is zero chance anyone could understand the global picture, though some of us are expected to somewhat grasp it. Quite the challenge, would be truly impossible with llms
The way that people interact inside of knowledge companies to get things done is itself the fabric of how it operates. A recent SaaS CEO piece here calls is the 'language games'.
Author from the other thread here. I'm surprised to see so many similarities, but in good faith I'll assume that it's just a coincidence because many devs start to notice the upcoming problems.
I appreciate your good faith but I tried to copy-paste the first ~7k character of this (not yours!) article in an AI detector (gptzero) and it's "highly confident that this text was AI generated" with a probability of 100%
Oh yes, this is blatantly AI-written. Whether this is ironic and discredits the claims is up to you.
But the opening at least makes a good point: AI-written code is right now something of an embezzlement or theft or fraud. Like junk food, it presents all the appearance of coming with correlates like "an engineer now understands the problem-space and has built tacit knowledge and a theory of how the domain/code work"... But it doesn't. The human engineer takes all the credit (and promotions), having put in a tenth or less the usual work, but evades the blame or tech debt.
This is why vibecoding is the inevitable endpoint. The current code market is a lemon market, and we see the exact same kind of dynamics of 'buyers' withdrawing from 'purchases' as they see all these shiny code bases which are secretly lemons, and sellers increasingly get frustrated they can't get a 'fair price' (like anyone to look at their 'Show HN' posts).
We must understand that the source code is no more valuable than a compiled binary, and that the responsible human doesn't understand it and has to trust it, in the same way that we now understand source code but don't understand and have to trust the compiled binary. Today, I understand my Haskell code, but I have no idea what is going on in the compiled binary (nor do I claim to); tomorrow, I will understand my prompt/spec, but I will have no idea what is going on in the 'source code' (nor will anyone expect me to or admire me for 'writing all that code' or be surprised if some seriously mistaken assumption is hardwired into the generated code). Just as no one cares about a binary you compiled today, no one will care about your source code. (Unless there is something special about it, of course. 'I compiled Firefox for my Linux' - OK, congrats? 'I compiled Firefox for my toaster' - OK now I'm interested.)
So, no one will care about the codebase for your 'Show HN' project; you have to instead get them to care about your proposal or your testsuite. If it's any good, they can simply pass it to their own LLMs to 'compile'; no need to trust your crummy dubious already-obsolete 'source code'...
> The second is absorption: mental models form, edge cases become intuitive, architectural relationships solidify into understanding. ... . The friction of implementation creates space for reasoning.
> This gap between output velocity and comprehension velocity is cognitive debt.
I have felt that lack of absorption during the last months, adding doomscroolling to the equation, I have felt how my thinking is disappearing.
I tried to speculatively expand that idea in this post
> A species that cannot follow the reasoning of its own systems does not supervise them; it simply inhabits them until they stop working.
I feel this idea is closely related to additive bias. People are scared of breaking things, so the safest way is to just add another tiny part to an already complex system. As cognitive debt accumulates faster, this additive bias just becomes stronger imo.
I know the topic won't probably be about this -and I'll be reading the article next-, just wanted to share that this title perfectly reminded me the feeling of attempting the speed reading technique explained in this old gem of a video (minute 20:15)
BOOKSTORES: How to Read More Books in the Golden Age of Content
You get the same when your company employs a lot of low to medium skilled offshore devs. Every morning or every week you get a huge pile of code that sort of works but there is simply no way to review it in a meaningful way. It's just too much. Thats how I feel working with Claude Code. It cranks out a lot of code really quickly but how do I know it's not creating subtle problems?
Just to make sure it's somewhere in these comments: the fundamental issue is people trying to measure something they don't understand. That is not new. The article gives an interesting exploration of how things break down in a new way when people focus too much on metrics instead of (IMO) the more robust approach of getting people who care to try to make something that feels quality. We're building crap, yes, but I blame the people who spend their time measuring "velocity" like it's a well defined term, not the coding tools being used to play the game.
It reminds me of Clay Christensen’s book How to Measure Your Life. In one of his talks, he talked about how companies get killed because they optimized for the wrong/short-term metrics. What we are seeing with AI could be a supercharged flavor of Innovator’s Dilemma, where organizations optimize a pre-existing set of success metrics while missing the bigger picture because some previous assumptions no longer hold.
I really like the article. It’s not trying to sell fear (which does sell); it doesn’t paint the leaderships as clueless. Nobody knows what is going to happen in the future. The article might be wrong on a few things. But it doesn’t matter. It points out a few assumptions that people might be missing and that is great.
As s.o. who had to browse/navigate/understand all these people's code:
- legacy guys
- super 10x guys who say no to u all the time
- students
- even more legacy
- open source
I got to a point where I honestly care so little about all these guys' damn architectural decisions, which to me - a practitioner, scientist, researcher and academics teacher - made similarly very little sense.
Really, top coders, and veteran Java enterprise copy-pasters, I care so little about your damn code, it is very wrong most of times. I care very little about architectural decisions most of the opensource people took, as they very often come from weird backgrounds and these decisions do not match mine. Needless to say - they often know their architectural decisions are already wrong 10 years later (a great example is the QGIS crowd in this regard). I don't care about somebody's greatly designed ProC code. Neither do I care if Twitter was doing 1000 of API calls, which it seems to have been doing in reality, as even though I despise the Elon guy - well, his new X is arguably faster and more stable.
I don't care about how great your docker scales, if you need to scale to 1m VMs and back again, there is a fair chance you're Google, so I don't care about you either, as you are not the good guys anymore.
Likewise, I very much would bet 99% of visitors here don't really care what architectural decisions YC took when they decided to showcase Algolia's search. Very little interest in this.
The whole idea that there is a right way to do architecture or code is in total and direct contradiction with the history of computing, which has a good record of many successful projects not having great architecture (MySpace for example) and great projects that did not fly, even though they were top notch.
What I care is about is people and what people they are. Are they fakers? Are they smart? Are they in love with their code, or they simply see it as a tool. Are they smart enough to make a step back. Are they calm enough, are they inspiring. And of course - am I getting paid to do it.
So this massive outcry is super misplaced, and you know what - I don't care if you created your code with Claude or by threading it one char at a time, because eventually it's going to be me, with close to little knowledge, that will be forced to untangle this wonderful mess of yours.
And, no, you cannot teach people how to code. You can show them the way, and they learn their approach to it. Leave 5 people alone in 5 rooms, you'll get 5 architectures, perhaps all of them very solid.
Before AI, we discuss on how to solve a problem with teammates.
Even if we didn't remember exactly what we wrote 6mo ago, we at least remembered the general idea.
After AI, that understanding often disappears, to the point where we can't even direct the AI to fix the problem because we don't know what's wrong.
Also AI often changes the code in the context of current problem. So, we might get more bugs when fixing one.
The solution is ironically, LLMs. You can construct a set of Claude skills to walk you through a code review, or understand code (anything, even course) fast.
This complexity to understanding compression will be a big market going forward.
There is a treasure trove of interesting stuff if you have the LLM look at the chat history as well.
I’ve been building skills and agents to help walk my projects history and chat to answer specific questions or find nuggets. For example I’ve got an agent that lets me ask “find me instances in this project where I demonstrated engineer vs product tradeoffs”. It will mine the chat history, git logs, documentation files I wrote looking to answer that.
Someday soon I’ll publish my techniques. It isn’t magic, the hard part is shaking the bias out of your mining agents so they don’t get loaded with “pre-discovered” wisdom and do nothing but confirm what the orchestrator told them they should find.
> When circumstances eventually require that understanding, when something breaks in an unexpected way or requirements change in a way that demands architectural reasoning, the organization discovers the deficit.
Maybe it's because I work in such a small team on a still-starting project, but even with the chaos of LLM-generated code, I can't imagine such a case as above that the LLMs couldn't also address.
This reminds me again of _Programming as Theory Building_[1] by Peter Naur. With agents fast generating the code, we lose the time for building the theory in our heads.
Let’s go back to terms and thinking from 5 years ago. It’s called rushing. People are rushing now and they’re making mistakes. Some are big and systematic where they don’t pause to reflect on all the consequences and some are more local which are just bad coding bugs.
I have a view that we are shifting from the traditional form of Engineering into a more AI guided form, where may be we are not learning as much about the code but about how we can produce that code with correct instructions and high level design.
It's like how we might not know how sewing is done but we know how to put instructions in a loom to produce it.
I also agree it is still important to read that code and understand how it works, may be take a moment to see what is happening but we are learning something entirely different here.
I think stronger determinism could dramatically improve the situation here. Right now, I don't know if the same model within the same hour will produce consistent output given identical prompts and low temperature.
I have no clue what my compiler is emitting every time I hit F5. I don't need to comprehend IL or ASM because I have a ~deterministic way to produce this output from a stable representation.
Writing a codebase as natural language is definitely feasible, but how we're going about it right now is not going to support this. A vast majority of LLM coding is coming out of ad-hoc human in the loop or stochastic agent swarms. If we want to avoid the comprehension gap we need something closer to a compiler & linker that operates over a bucket of version-controlled natural language documents.
More code written probably does means less understanding per line (or per a more germane metric), statistically speaking. More dilute understanding probably does lead to more failures and longer recovery times. This feels like something better addressed as an end-to-end actuarial problem though, rather than trying to design metrics for something elusive like understanding.
This seems very similar to the situation of a new employee dropped into a large codebase of varying quality. It seems like similar techniques will get you out of the mess?
Also, you can ask the coding agent for help at understanding it, unlike the old days when whoever wrote it is long gone.
Only the coding agent will only give you plausible answers, not necessarily correct ones. So read the code. Oh, but you now can't because all you know is how to ask coding agents.
Forgive me if I'm stating the obvious, but, it is completely plausible and possible to run a separate review of what ai just created, explaining what decisions where made and why, how they affect the existing system and going forward. This review can have a critique section over core failure modes that you have found in ai, or discrepancies unique to your setup. It can even be further condensed from verbose 2 page document into the core relevant explanation, for future references. - I think sometimes SWE's have an ego about needing to understand it entirely self-sufficiently, and so hold back on just asking relentless questions, like a child. 'But why?' 'but why?' 'but why?' until it is revealed, is a valid method in today's environment.
Good engineering has always been about minimizing the amount of effort it takes for someone to understand and modify your code. This is the motivation for good abstractions & interfaces, consistent design principles, single-responsibility methods without side-effects, and all of the things we consider "clean code".
These are more important than ever, because we don't have the crutch of "Teammate x wrote this and they are intimately familiar with it" which previously let us paper over bad abstractions and messy code.
This is felt more viscerally today because some people (especially at smaller/newer companies) have never had to work this way, and because AI gives us more opportunity to ignore it
Like it or not, the most important part of our jobs is now reviewing code, not writing it. And "shelfed" ideas will now look like unmerged PRs instead of unwritten code
I just recently started using Claude Code seriously and I am surprised how well it understands even complex codebases and can explain not only what was done but also why. Really impressive and very useful. You just have to pray that it didn't hallucinate something
This happened to me yesterday.
I give a junior engineer a project. He turns it around really quickly with Cursor. I review the code, get him to fix some things (again turned around really quickly with Cursor) and he merges it. I then try a couple test cases and the system does the wrong thing on the second one I try. I ask him to fix it. He puts into cursor a prompt like "fix this for xyz case" and submits a PR. But when I look at the PR, it's clearly wrong. The model completely misunderstood the code. So I leave a detailed comment explaining exactly what the code does.
He's moving so fast that he's not bothering to learn how the system actually works. He just implicitly trusts what the model tells him. I'm trying to get him to do end-to-end manual testing using the system itself (log into the web app in a local or staging environment and go through the actions that the user would go through), he just has the AI generate tests and trusts the output. So he completely misses things that would be clear if you learned the system at a deep level and could see how the individual project you're working on fit in with the larger system
I see this with all the junior engineers on my team. They've never learned how to use a debugger and don't care to learn. They just ask the model. Sometimes they think critically about the system and the best way to do something, but not always. They often aren't looking that critically at the model's output.
Senior engineers must become more comfortable giving quick, broad feedback that matches the minimal time put into the PR. "This doesn't fit how the system works; please research and write a more detailed prompt and redo this" is the advice they need. It feels taboo to do it to a significant diff, but diff size no longer has much correlation to thought or effort in these situations.
Management where I work is currently touting a youtube video from some influencer about the levels of AI development, one of the later ones being "you'll care that it works, not how".
We are all supposed to be advancing through these levels. Moving at a pace where you actually understand the system you're responsible for is now considered a performance issue. But also, we're "still held responsible for quality".
Needless to say I'm dusting off my resume, but I'm sure plenty of other companies are following the same playbook.
Just read every line of the generated code and make sure it is as clear and good as possible. If you can't understand it when it's new you won't tomorrow, either. This verification process places a natural limit on the rate at which you can safely generate code. I suppose you could reduce that to spot checks and achieve probabilistic correctness but I would not venture there for things that matter.
Because lines of code interact with each other. Understanding what one line does in isolation does not always show the rough edges that are found when code interacts. The challenge is seeing the forest instead of individual trees.
Code has become cheaper to produce than to perceive.
Which means fixes can go in faster than it would require to first grok it
What’s missing in literally every single one of these conversations is testing
Literally all you have to do is implement test driven development and you solve like 99.9% of these issues
Even if you don’t go fully TDD which I’m not a fan of necessarily having an extensive testing suite the covers edge cases is necessary no matter what you do but it’s a need to have in a case where your code velocity is high
This is true for a company full of juniors pumping out code like early days of Facebook let’s say which allowed for their mono repot to grow insanely but it took major factors every few years but it didn’t really matter because they had their resources to do it
There's another gap, actually. Imprecise specification.
When you need to implement something yourself, you have to make decisions when faced with the reality of turning ideas into code.
An AI agent sometimes surfaces these; and sometimes it just makes a choice.
The risk is tests just embed these decisions as policy in code, without there have been proper consideration.
Often there's a core ambiguity in a conception somewhere, and because of the limited context of an AI, it can implement things one way and then another for the next feature, without actually hitting the inconsistency.
I agree but this is captured in the higher order evaluation layer
So a poor specification is perfectly technically implemented won’t actually do what the intended goal is because the user has not correctly specified the goal and so ultimately it will fail at the highest level task if poorly specified
But that won’t necessarily reveal itself in code it would reveal itself in failure for other people to adopt or the user let’s say to adopt that tool for their workflow
The project I work on has been steadily growing for years, but the amount of engineers taking care of it stayed same or even declined a bit. Most of features are isolated and left untouched for months unless something comes up.
So far, I managed growing scope by relying on tests more and more. Then I switched to exclusively developing against a simulator. Checking changes with real system become rare and more involved - when you have to check, it's usually the gnarliest parts.
Last year's, I noticed I can no longer answer questions about several features because despite working on those for a couple of months and reviewing PRs, I barely hold the details in my head soon afterwards. And this all even before coding agents penetrated deep into our process.
With agents, I noticed exactly what article talks about. Reviewing PR feels even more implicit, I have to exert deliberate effort because tacit knowledge of context didn't form yet and you have to review more than before - the stuff goes into one ear and out of another. My team mates report similar experience.
Currently, we are trying various approaches to deal with that, it it's still too early to tell. We now commit agent plans alongside code to maybe not lose insights gained during development. Tasks with vague requirements we'd implicitly understand most of previously are now a bottleneck because when you type requirements to an agent for planning immediately surface various issues you'd think of during backlog grooming. Skill MDs are often tacit knowledge dumps we previously kept distributed in less formal ways. Agents are forcing us to up our process game and discipline, real people benefit from that too. As article mentioned, I am looking forward to tools picking some of that slack.
One other thing that surprised me was that my eng manager was seemingly oblivious to my ongoing complains about growing cognitive load and confusion rate. It's as if the concept was alien to them or they could comprehend that other people handle that at different capacity than them.
Engineering managers in my experience (even in ones with deep technical backgrounds) often miss the trees for the forest. The best ones go to bat for you, especially once verifying that they can do something to unblock or support you. But that’s still different than being in the terminal or IDE all day.
Offloading cognitive load is pretty much their entire role.
You will also add a markdown file to the changelog directory named with the current date and time `date -u +"%Y-%m-%dT%H-%M-%SZ"`, record the prompt, and a brief summary of what changes you made, this should be the same summary you gave the developer in the chat.
From that I get the prompt and the summary for each change. It's not perfect but it at least adds some context around the commit.
Part of my CI process when creating skills involves setting token caps and comparing usage rates with and without the skill.
When software engineers become agent herders their day-to-day starts to resemble more that of a manager than that of an engineer.
* thinking about the big picture * knowing how you can verify that the code match the big picture.
In both case, somtimes you are happily surprised, sometimes you discover that the things you told 3 times the one writing code to do was still not done.
The core of the article is “ AI-assisted development potentially short-circuits this replenishment mechanism. If new engineers can generate working modifications without developing deep comprehension, they never form the tacit knowledge that would traditionally accumulate. The organization loses knowledge not just through attrition but through insufficient formation.”
But is it possible this phenomenon is transient?
Isn’t part of the presumed value add of LLM coding agents in the meta-realm around coding; e.g. that well-structured human+LLM generated code (green field in particular) will be organized in such a way that the human will not have to develop deep comprehension until needed (e.g. for bug fix/optimization) and then only for a working set of the code, with the LLM bringing the person up to speed on the working set in question and also providing the architectural context to frame the working set properly?
We've always had the problem that understanding while writing code is easier than understanding code you've written. This is why, in the pre-AI era, Joel Spolsky wrote: "It's harder to read code than to write it."
The worst code bases I have to deal with have either no philosophy or a dozen competing and incompatible philosophies.
The best are (obviously) written in my battle tested and ultra refined philosophy developed over the last ~25 years.
But I'm perfectly happy to be working in code bases written even with philosophies that I violently disagree with. Just as long as the singular (or at least compatible) philosophy has a certain maturity and consistency to it.
I didn't remember every line but I still had a very good grasp of how and why it's put together.
(edit: and no, I don't have some extra good memory)
Other times, I can make a small change to something that doesn't require much time, and once it's tested and committed, I quickly lose any memory of even having done it.
When I do a drive-by edit, I probably don't remember it in a week.
Which is why the "cognitive debt" from the article is relevant, IMHO. If I just thoroughly review the plan and quickly scan the resulting code, will that have a strong enough imprint on my mind over time?
I would like to think "yes", my gut is telling me "no". IMHO the LLMs are now "good enough" for coding. These are hard questions we'll have to grapple with this in the next year or two (in context of AI-assisted software development).
The hard part is to gain familiarity with the project's coding style and high level structure (the "intuition" of where to expect what you're looking for) and this is something that comes back to you with relative ease if you had already put that effort in the past - like a song you used to have memorized in the past, but couldn't recall it now after all these years until you heard the first verse somewhere. And of course, memorizing songs you wrote yourself is much easier, it just kinda happens on its own.
Meanwhile some stuff Claude wrote for me last week I barely remember what it even did at a high level.
I don’t know if this becomes prod code, but I often feel the need to create like a Jupyter notebook to create a solution step by step to ensure I understand.
Of course I don’t need to understand most silly things in my codebase. But some things I need to reason about carefully.
With llm-first coding, this experience is lost
AI tools don’t prevent people from understanding the code they are producing as it wouldn’t actually take that much time, but there’s a natural tendency to avoid hard work. Of course AI code is generally terrible making the process even more painful, but you where just looking at the context that created it so you have a leg up.
I absolutely feel the cognitive debt with our codebase at work now. It’s not so much that we are churning out features faster with ai (although that is certainly happening) - but we are tackling much more complex work that previously we would have said No to.
For example, handwritten code also tended to be reviewed manually by each other member of the team, so the probability of someone recalling was higher than say, LLM generated code that was also LLM reviewed.
Anyone pretending gen-ai code is understood as well as pre-gen-ai, handwritten code is totally kidding themselves.
Now, whether the trade off is still worth it is debatable, but that's a different question.
The hope being that if the feature were to be kept or the demo fleshed out, developers would need to shape and refactor the project as per newly discovered requirements, or start from scratch having hopefully learnt from the agentic rush.
To me, it always boils down to LLMs being probabilistic models which can do more of the same that has been done thousands of times, but also exhibit emergent reasoning-like properties that allow them to combine patterns sometimes. It's not actual reasoning, it's a facsimile of reasoning. The bigger the models, the better the RLHF and fine-tuning, the more useful they become but my intuition is that they'll always (LLMs) asymptotically try to approach actual reasoning without being able to get there.
So the notion of no-human-brain-in-the-loop programming is to me, a fool's errand. I do obviously hope I am right here, but we'll see. Ultimately you need accountability and for accountability you need human understanding. Trying to move fast without waiting for comprehension to catch up (which would most likely result in alternate, better approaches to solving the problem at hand) increases entropy and pushes problems further down the road.
AI spec docs and documentation also have this documentation problem
Claude often makes a hash of our legacy code and then i go look at what we had there before it started and think “i don’t even know what i was thinking, why is this even here?”
The complaint about "code nobody understands" because of accumulating cognitive debt also happened with hand-written code. E.g. some stories:
- from https://devblogs.microsoft.com/oldnewthing/20121218-00/?p=58... : >Two of us tried to debug the program to figure out what was going on, but given that this was code written several years earlier by an outside company, and that nobody at Microsoft ever understood how the code worked (much less still understood it), and that most of the code was completely uncommented, we simply couldn’t figure out why the collision detector was not working. Heck, we couldn’t even find the collision detector! We had several million lines of code still to port, so we couldn’t afford to spend days studying the code trying to figure out what obscure floating point rounding error was causing collision detection to fail. We just made the executive decision right there to drop Pinball from the product.
- and another about the Oracle RDBMS codebase from https://news.ycombinator.com/item?id=18442941
(That hn thread is big and there are more top-level comments that talk about other ball-of-spaghetti projects besides Oracle.)
This is the most insidious part. It's not even that bad code gets deployed. That can be fixed and hopefully (by definition) the market weeds that out.
The problem is that the market doesn't seem to operate like that, and instead the engineer who cares loses their job because they're not hitting the metrics.
Constraints often result in better results. Think of Duke Nukem Forever and how long it took them to release a nothingburger.
I just watched a show called the Knight of the Seven Kingdoms and the showerunners were given a limited budget compared to their cousin shows and it resulted in a better product.
Sometimes those metrics keep things on the rails
This is a common trope, but in my experience many engineers I met know that's not how a business runs. Dealing with the constraints and weighing them out is one of the essential skills of any engineer. Knowing when a product is just good enough is one of the things that make you senior.
What I don't like is the impossible middle ground where people are asked to 20X their output while taking full responsibility for 100% of the code at the same time. That is the kind of magical thinking that I am certain the market will eventually delete. You have to either give up on comprehension or accept a modest, 20% productivity boost at best.
https://www.benguttmann.com/blog/double-it-or-cut-it-in-half...
Brownfield legacy projects with god classes and millions of lines of code which need to behave coherently across multiple channels- without actually having anything linking them from the written code? That shit is not even gonna get a 20% boost, you'll almost always be quicker on your own - what you do get is a fatigue bonus, by which I mean you'll invest yourself less for the same amount of output, while getting slightly slower because nobody I've ever interacted with is able to keep such code bases in their mind sufficiently to branch out to multiple agents.
On projects that have been architected to be owned by an LLM? Modular modilith with hints linking all channels together etc? Yeah, you're gonna get a massive productivity boost and you also will be using your brain a shitton actually reasoning things out how you'll get the LLM to be able to actually work on the project beyond silly weekends toy project scope (100k-MM LOC)
But let's be real here, most employees are working with codebases like the former.
And I'm still learning how to do the second. While I've significantly improved since I've started one year ago, I wouldn't consider myself a master at it yet. I continue to try things out and frequently try things that I ultimately decide to revert or (best case) discard before merge to main simply because I ... Notice significant impediments modifying/adding features with a given architecture.
Seriously, this is currently bleeding Edge. Things have not even begun to settle yet.
We're way too early for the industry to normalize around llms yet
That’s the exciting part isn’t it? This stuff is completely uncharted water. The interesting thing is there is huge forests of low hanging fruit to harvest too.
The interesting thing is this tickles all the same things “actual programming” does. It’s just the programming language changed to precise English and while the fundamental CS constrains are the same there is a new layer of constraints. Some, like speed of inference, I suspect will be solved (which will be incredible) but others like context management and size will always exist for LLMs. Understanding how little adjustments to what you prompt them can have dramatic changes to their output will never go away. Knowing what to put into context is a hugely important skill.
Every one of us is a pioneer if we choose to be. We have only scratched the surface as an industry.
Now you could say that expectation has to change but I don’t see how—the people paying you expect you to produce working software. And we’ve always been biased in favor of short term shipping over longer term maintainability.
Also, the essay notes that once a "worse" system is established, it can be incrementally improved. Following that argument, we can say that as long as the AI code runs, it creates a footprint. Once the software has users and VC funding, developers can go back and incrementally improve or refactor the AI's mess, to a satisfying degree.
I will fully admit that AI writes better code than me and does it faster.
That absorption only takes place in the mind of that individual, unfortunately. That doesn't help when they no longer work there or are on vacation.
The ideal situation is the solo open source project. You wrote all 200K lines of code yourself, and will maintain them until death. :)
Now, it can take only a few days or weeks.
But to offer a counter argument, would the same thing not have happened with the rise of high level languages? The machine code was abstracted away from engineers and they lost understanding of it, only knowing what the high level code is supposed to do. But that turned out fine. Would llms abstracting the code away so engineers only understand the functionality (specs, tests) also be fine for the same reason? Why didnt cognitive debt rise in with high level languages?
A counter counter argument is that compilers are deterministic so understanding the procedure of the high level language meant you understood the procedure that mattered of the machine code, and the stuff abstracted away wasnt necessary to the codes operation. But llms are probabilistic so understanding the functionality does not mean understanding the procedure of the code in the ways that matters. But id love to hear other peoples thoughts on that
Any argument that attempts to frame LLMs as analogous to compilers is too flawed to bother pursuing. It's not that compilers are deterministic (an LLM can also be deterministic if you have control over the seed), it's that the compiler as a translator from a high level language to machine code is a deductive logical process, whereas an LLM is inherently inductive rather than deductive. That's not to say that LLMs can't be useful as a way of generating high level code that is then fed into a compiler (an inductive process as a pipeline into a deductive process), but these are fundamentally different sorts of things, in the same way that math is fundamentally different from music (despite the fact that you can apply math to music in plenty of ways).
The purpose of high level languages is to make the structure of the code and data structures more explicit so it better captures the “actual” program model, which is in the mind of the programmer. Structured programming, type systems, modules, etc. are there to provide solid abstractions in which to express that model.
None of that applies to giving an LLM a feature idea in English and letting it run. (Though all of it is helpful for keeping an LLM from going completely off the rails.)
The last part is what matters. There's no such clear rules in LLMs behavior. Yes you can get to behave roughly like a rule, but there's no clear cut demarcation between what's in and what's not.
It did not turn out fine. Fortunately no one took it seriously, and at least seniors still have an intuitive model of how the hardware works in their head. You don't have to "see" the whole assembly language when writing high level code, just know enough about how it goes at lower levels that you don't shoot yourself in the foot.
When that's missing, due to lack of knowledge or perhaps time constraints, you end up on accidentally quadratic or they name a CVE after you.
There's going to be a massive shift in programming education though, because knowing an actual programming language won't matter any more than knowing assembly does today.
0/1s → assembly → C → high-level languages → frameworks → AI → product
The engineer keeps moving up the abstraction chain with less and less understanding of the layers below. The better solution would be creating better verification, testing, and determinism at the AI layer. Surely we'll see the equivalent of high-level languages and frameworks for AI soon.
I wrote a SaaS project over the weekend. I was amazed at how fast Claude implemented features. 1 sentence turned into a TDD that looked right to me and features worked
but now 3 weeks later I only have the outlines of how it works and regaining the context on the system sounds painful
In projects I hand wrote I could probably still locate major files and recall system architectures after years being away
One could argue with its stance, but I took it as a given (the equation for cognitive debt touches on science).
It feels entirely logical to view LLMs/coding agents as an almost final step in the short-term focus the overall system has been thriving on.
Very disturbing as I thought my technical skills would help me clarify the global picture. And that is exactly the contrary that is happening.
https://ionanalytics.com/wp-content/uploads/2026/02/The_Wron...
"The right amount of AI is not zero. And it’s not maximum."
I'm seriously considering taking a break from this site if this is the direction that everyone wants to go in.
But the opening at least makes a good point: AI-written code is right now something of an embezzlement or theft or fraud. Like junk food, it presents all the appearance of coming with correlates like "an engineer now understands the problem-space and has built tacit knowledge and a theory of how the domain/code work"... But it doesn't. The human engineer takes all the credit (and promotions), having put in a tenth or less the usual work, but evades the blame or tech debt.
This is why vibecoding is the inevitable endpoint. The current code market is a lemon market, and we see the exact same kind of dynamics of 'buyers' withdrawing from 'purchases' as they see all these shiny code bases which are secretly lemons, and sellers increasingly get frustrated they can't get a 'fair price' (like anyone to look at their 'Show HN' posts).
We must understand that the source code is no more valuable than a compiled binary, and that the responsible human doesn't understand it and has to trust it, in the same way that we now understand source code but don't understand and have to trust the compiled binary. Today, I understand my Haskell code, but I have no idea what is going on in the compiled binary (nor do I claim to); tomorrow, I will understand my prompt/spec, but I will have no idea what is going on in the 'source code' (nor will anyone expect me to or admire me for 'writing all that code' or be surprised if some seriously mistaken assumption is hardwired into the generated code). Just as no one cares about a binary you compiled today, no one will care about your source code. (Unless there is something special about it, of course. 'I compiled Firefox for my Linux' - OK, congrats? 'I compiled Firefox for my toaster' - OK now I'm interested.)
So, no one will care about the codebase for your 'Show HN' project; you have to instead get them to care about your proposal or your testsuite. If it's any good, they can simply pass it to their own LLMs to 'compile'; no need to trust your crummy dubious already-obsolete 'source code'...
> This gap between output velocity and comprehension velocity is cognitive debt.
I have felt that lack of absorption during the last months, adding doomscroolling to the equation, I have felt how my thinking is disappearing.
I tried to speculatively expand that idea in this post
https://news.ycombinator.com/item?id=47186004
> A species that cannot follow the reasoning of its own systems does not supervise them; it simply inhabits them until they stop working.
I feel this idea is closely related to additive bias. People are scared of breaking things, so the safest way is to just add another tiny part to an already complex system. As cognitive debt accumulates faster, this additive bias just becomes stronger imo.
BOOKSTORES: How to Read More Books in the Golden Age of Content
https://m.youtube.com/watch?v=lIW5jBrrsS0&t=1215s
I really like the article. It’s not trying to sell fear (which does sell); it doesn’t paint the leaderships as clueless. Nobody knows what is going to happen in the future. The article might be wrong on a few things. But it doesn’t matter. It points out a few assumptions that people might be missing and that is great.
- legacy guys - super 10x guys who say no to u all the time - students - even more legacy - open source
I got to a point where I honestly care so little about all these guys' damn architectural decisions, which to me - a practitioner, scientist, researcher and academics teacher - made similarly very little sense.
Really, top coders, and veteran Java enterprise copy-pasters, I care so little about your damn code, it is very wrong most of times. I care very little about architectural decisions most of the opensource people took, as they very often come from weird backgrounds and these decisions do not match mine. Needless to say - they often know their architectural decisions are already wrong 10 years later (a great example is the QGIS crowd in this regard). I don't care about somebody's greatly designed ProC code. Neither do I care if Twitter was doing 1000 of API calls, which it seems to have been doing in reality, as even though I despise the Elon guy - well, his new X is arguably faster and more stable.
I don't care about how great your docker scales, if you need to scale to 1m VMs and back again, there is a fair chance you're Google, so I don't care about you either, as you are not the good guys anymore.
Likewise, I very much would bet 99% of visitors here don't really care what architectural decisions YC took when they decided to showcase Algolia's search. Very little interest in this.
The whole idea that there is a right way to do architecture or code is in total and direct contradiction with the history of computing, which has a good record of many successful projects not having great architecture (MySpace for example) and great projects that did not fly, even though they were top notch.
What I care is about is people and what people they are. Are they fakers? Are they smart? Are they in love with their code, or they simply see it as a tool. Are they smart enough to make a step back. Are they calm enough, are they inspiring. And of course - am I getting paid to do it.
So this massive outcry is super misplaced, and you know what - I don't care if you created your code with Claude or by threading it one char at a time, because eventually it's going to be me, with close to little knowledge, that will be forced to untangle this wonderful mess of yours.
And, no, you cannot teach people how to code. You can show them the way, and they learn their approach to it. Leave 5 people alone in 5 rooms, you'll get 5 architectures, perhaps all of them very solid.
After AI, that understanding often disappears, to the point where we can't even direct the AI to fix the problem because we don't know what's wrong.
Also AI often changes the code in the context of current problem. So, we might get more bugs when fixing one.
This complexity to understanding compression will be a big market going forward.
I’ve been building skills and agents to help walk my projects history and chat to answer specific questions or find nuggets. For example I’ve got an agent that lets me ask “find me instances in this project where I demonstrated engineer vs product tradeoffs”. It will mine the chat history, git logs, documentation files I wrote looking to answer that.
Someday soon I’ll publish my techniques. It isn’t magic, the hard part is shaking the bias out of your mining agents so they don’t get loaded with “pre-discovered” wisdom and do nothing but confirm what the orchestrator told them they should find.
Maybe it's because I work in such a small team on a still-starting project, but even with the chaos of LLM-generated code, I can't imagine such a case as above that the LLMs couldn't also address.
Great read though and I appreciated the article.
[1] https://pages.cs.wisc.edu/~remzi/Naur.pdf
It's like how we might not know how sewing is done but we know how to put instructions in a loom to produce it. I also agree it is still important to read that code and understand how it works, may be take a moment to see what is happening but we are learning something entirely different here.
We shouldnt be giving it up just for some mild convenience (which seems so far overall really mild). The gain simply doesnt match the long term loss.
I have no clue what my compiler is emitting every time I hit F5. I don't need to comprehend IL or ASM because I have a ~deterministic way to produce this output from a stable representation.
Writing a codebase as natural language is definitely feasible, but how we're going about it right now is not going to support this. A vast majority of LLM coding is coming out of ad-hoc human in the loop or stochastic agent swarms. If we want to avoid the comprehension gap we need something closer to a compiler & linker that operates over a bucket of version-controlled natural language documents.
Also, you can ask the coding agent for help at understanding it, unlike the old days when whoever wrote it is long gone.
These are more important than ever, because we don't have the crutch of "Teammate x wrote this and they are intimately familiar with it" which previously let us paper over bad abstractions and messy code.
This is felt more viscerally today because some people (especially at smaller/newer companies) have never had to work this way, and because AI gives us more opportunity to ignore it
Like it or not, the most important part of our jobs is now reviewing code, not writing it. And "shelfed" ideas will now look like unmerged PRs instead of unwritten code
He's moving so fast that he's not bothering to learn how the system actually works. He just implicitly trusts what the model tells him. I'm trying to get him to do end-to-end manual testing using the system itself (log into the web app in a local or staging environment and go through the actions that the user would go through), he just has the AI generate tests and trusts the output. So he completely misses things that would be clear if you learned the system at a deep level and could see how the individual project you're working on fit in with the larger system
I see this with all the junior engineers on my team. They've never learned how to use a debugger and don't care to learn. They just ask the model. Sometimes they think critically about the system and the best way to do something, but not always. They often aren't looking that critically at the model's output.
Can we get rules against this or something at this point? It's every other post.
We are all supposed to be advancing through these levels. Moving at a pace where you actually understand the system you're responsible for is now considered a performance issue. But also, we're "still held responsible for quality".
Needless to say I'm dusting off my resume, but I'm sure plenty of other companies are following the same playbook.
I propose a new paradigm: programmer experience, PX.
So, code generated by AI ideally would follow the rules of PX. Whatever those may turn out to be.
Which means fixes can go in faster than it would require to first grok it
What’s missing in literally every single one of these conversations is testing
Literally all you have to do is implement test driven development and you solve like 99.9% of these issues
Even if you don’t go fully TDD which I’m not a fan of necessarily having an extensive testing suite the covers edge cases is necessary no matter what you do but it’s a need to have in a case where your code velocity is high
This is true for a company full of juniors pumping out code like early days of Facebook let’s say which allowed for their mono repot to grow insanely but it took major factors every few years but it didn’t really matter because they had their resources to do it
When you need to implement something yourself, you have to make decisions when faced with the reality of turning ideas into code.
An AI agent sometimes surfaces these; and sometimes it just makes a choice.
The risk is tests just embed these decisions as policy in code, without there have been proper consideration.
Often there's a core ambiguity in a conception somewhere, and because of the limited context of an AI, it can implement things one way and then another for the next feature, without actually hitting the inconsistency.
So a poor specification is perfectly technically implemented won’t actually do what the intended goal is because the user has not correctly specified the goal and so ultimately it will fail at the highest level task if poorly specified
But that won’t necessarily reveal itself in code it would reveal itself in failure for other people to adopt or the user let’s say to adopt that tool for their workflow