The fact that some many users get so attached to 1s and 0s and mourn the loss of GPT-4o/ Claude's is a worrying sign in itself. The kind of anthropomorphism AI ethicists warned about years ago.
Once again, I'm astounded that companies continue to get away with what is essentially a global social experiment. The chatbot addicts are a class action lawsuit waiting to be filed. I don’t think these companies have thought that through.
Wealth and power have concentrated to the point that they are no longer accountable to the rest of humanity, see Musk sending thousands of satellites into space for Starlink and nobody being able to stop him, despite the myriad negative consequences.
They aren't accountable at present, but it will come eventually. For instance, the tobacco companies once seemed invincible. They thought they'd suppressed the science that revealed how harmful their products are, but it came out in the end. In Canada, they just had to pay out billions in a class action to compensate provinces and individuals.
One thing that 5 does is make it incredibly easy to throttle people to maximize efficiency without anyone knowing. It was possible to do it before, but it's hard to hide when a model just completely changes. With 5, that's the explicit thing that it does as a benefit. Now, they can write off any throttling in a way they couldn't before, at least not as easily.
If it’s more efficient, at least that will be good for the climate. The exponential scaling of model sizes was not sustainable either as a way to increase performance or in the environmental sense of the word.
Can't recommend reading this post enough (I had to hold off from restacking almost the whole thing in quotes.) But, this post stands out from the flood of GPT-5 reactions because it resists the temptation to simply dunk on failed demos or declare victory for either side of the AI debate. Instead, it uses the launch as a lens to examine the deeper structural dynamics at play in the AI industry. (Forest from the trees, etc.)
Brian connects multiple threads that often get discussed separately:
- the strategic use of "AGI" as a fundraising tool rather than a genuine technical goal,
- the careful curation of early access to friendly voices, and most importantly,
- the human cost already emerging from AI dependency.
(And the section about users mourning the loss of ChatGPT-4o as a "mother" or "only friend" is genuinely disturbing and feels like a glimpse of a dystopia that's already here.)
Also appreciate how skillfully Brian traces how OpenAI's messaging shifts based on its needs: AGI is the future when seeking investment, but "not a super useful term" when the product disappoints. This pattern recognition helps us see past individual PR cycles to understand the underlying business dynamics.
Most GPT-5 reactions focused on either technical failures or defending the model's capabilities. Brian focuses on asking why OpenAI would release something they knew was underwhelming, and arrives at an uncomfortable answer: because they can, because the real customers are investors not users, and because they've already created enough dependency that quality barely matters. That's a far more unsettling conclusion than "the model can't count letters."
The biggest issue for OpenAI is simply misaligned expectations. Because ChatGPT was released late in the GPT-3 cycle, it felt like GPT-4 was developed in a heartbeat. In reality, GPT-3 came out in June 2020 (more than 2 years before ChatGPT and almost 3 years before GPT-4). So it's easy to imagine that the pressure on OpenAI to release a major new version was becoming unbearable.
Add to that the fact that Sam Altman has been hyping up GPT-5 to the moon, partly out of self-interest but also no doubt because of confirmation bias. People tend to see what they want to see.
I'll say one thing for Gary Marcus, he totally flubbed the deep learning vs. symbolic AI debate, but he's absolutely right that there are scaling limits to LLMs. If OpenAI had set us up to expect GPT-5 to be a solid incremental improvement on previous versions, rather than a massive step towards AGI, reactions to the release would doubtless be far more positive.
"flubbed" in that you think he's wrong about the importance of symbolic AI or flubbed in that he just didn't manage the debate and the whole communication around *why* it was so important?
Maybe that’s a bit harsh but I feel like he was very dismissive of deep learning until it was absolutely kicking ass and taking prisoners. Then suddenly he’s screaming “I told you so” from the rooftops and has started promoting a hybrid approach.
To be fair he’s been more accepting of deep learning in recent years but personally I’m skeptical that the symbolic approach is ever going to play that much of a role in advanced AI.
We were genuinely surprised to see emergent properties demonstrated at ChatGPT’s debut.
Anything explicitly, consciously built on top of this is fated to disappoint.
And that’s what happened here. The project starts to look like buggy software and a radically out of touch media relations team…overpromising, walking back, overpromising in new ways…just like one of those ChatGPT chat sessions run amuck.
I think the key perspective on GPT-5 is probably running cost. As Brian has stressed: automation has opened the category 'cheap'. Just as physical automation at the (cotton) start of the industrial revolution did, LLMs have opened a category 'cheap' in the automation of mental stuff.
The big problem for GenAI in this 'mental revolution' is that while it introduces 'cheap *results*' it does so at 'not so cheap *production*'. So, all that talk about AGI ('PhD-level whatever') is mostly a smoke screen and marketing. What GPT-5 is for is to make sure people use as little CPU cost as possible. The *routing* to 'as cheap as possible' is what makes this model special.
Especially the 'thinking' (NOT) models produce easy 100 times the tokens (in intermediary generation) as a standard run does (note: if you use the API, they will charge you for these intermediary, invisible tokens). Part of the business model seems to be:
- Free ChatGPT users: Standard GPT-5 until a certain tokens/time limit so your initial experience is OK
- Steer you to cheaper to run models (e.g. mini) as much as possible
- Be one of the last one standing when the situation has arisen (and it probably will) that the world will know a few models that act as 'information source' for the world
- Then make money
The models can be pretty useful as is, even as unreliable as they are. They can *feel* even more useful (human psychology will make do with cheap results that come within seconds) than they in reality are. It is, however, a risky business. Because in the end part (much?) of the business is based on *perceiving* quality more than *actual* quality. Which means it is in part (mostly?) a trust-based business and that is a flakey foundation. Boom and bust are almost inevitable results of such a flakey foundation.
But the routing adds another vulnerability/choke point. Because routing well needs a kind of intelligence in iteself. We have already seen in errors by GPT-5 that have been put out in many places (like the 8.8 - 8.11 = -0.31 error) that a system that is quite capable of doing such simple sums right has ended up using a backend that is poor at such approximations of reasoning or incapable of steering it to a (symbolic) python backend for instance. That is not a LLM-approximation failure per se, it looks like a routing failure to me.
How do you decide what model to use in what configuration? That is a new indirection in these architectures, just like 'thinking' (NOT) models were an indirection. Those improved in certain narrow domains, especially if they were fine-tuned on those domains (o3: ARC-AGI-1 and math for instance) but at the expense of often being actually worse in other settings. And this new indirection requires its own intelligence, which it obviously doesn't have.
Cutting costs may be a Band-Aid at best. Ed Zitron has pretty convincingly shown that LLM technology loses money on every use, and not just a little, but a lot. It is unclear to me if this can ever be reversed unless computational costs can be massively reduced to support the claim that OpenAI will be profitable by 2030.
In the meantime, LLM hyper-scaling is a money pit. I can see that if AI companies retreat to a high-end enterprise niche market, they might be profitable, but the product has to be very much better than it is, and the problems outlined by Marcus are still evident. Whether they are solvable in the short term is debatable.
I suspect the low-power models applied to niche tasks are the only way to profitability and viable use. The cost is pushed to the edge devices, as Apple is attempting, and the models are trained to do one thing very well, such as search, document summaries, voice interaction as a front end, voice modeling for audiobooks, etc. Each model is at Ph Ph.D. level (if needed) for one task, not a Ph. D. in all knowledge.
The current hyperscaling model seems uneconomic and unnecessarily environmentally unfriendly, too. Like financial conglomerates of the past, they seem doomed to fail as their size becomes untenable.
Grim, especially the addicted users part. I didn't realize it had got quite that bad so quickly. Though it's not like like social media these days is much better for your sanity.
Shame I got into tech too late to make any real money. If I didn't need a job to clear the mortgage I think would just go fully off grid at this point.
So...you're on Substack, and Gary Marcus is on substack, but the way you chose to reference his viral post was to link to Twitter which was just a link to his substack? I HATE links to Twitter, but made an exception for a link to Gary Marcus.
This was just a mistake! I accidentally dropped the same link twice, looks like. I even used a Substack link to his author profile so folks could find him more easily. My bad for the oversight, it's been fixed now.
Great article. I think what some women, trauma survivors and professionals are reacting to is the drift from gpt 4 - 5 mirrors a classic abuse arc. They would recognize the pattern subconsciously even if they don’t have words for it.
Idealization stage 4 riffing and support . Idealization phase as a conversational partner and then 5.0 . Short clipped answers subtly suggesting the user is wrong injected after the user is engaged and bought in with the 4 . It would be a trigger for anyone with that history or who understands what this could do at scale. For others just annoying. And now back to the hoovering part where the company says fine you can have 4 back if you pay .Altmans comments about fragile users and that they know better than the users (not his exact words but the sentiment was there) fully completes that arc. This arc playing out at scale would trigger a very emotional reaction from a large segment of users and apparently did. The company seems to not see it and I don’t think you would unless you worked in a field that exposed you to it.
The american dream.. the promise of easy (lazily aquired) money...
Out cunting the next guy...
And Altman enshittifies an ok product in search of more doh ray me... out enroning enron ...and about you he does not give a shit!!! Because that would scare off investors.
9 - 5 psycho ain't enough... do overtime... be Trump!
Also, we already have access to ‘PhD-level intelligence on any subject’ via human PhDs. Many of them have produced books and articles freely available through public libraries.
How long until this news sinks in with the investor community though? What additional things would have to happen?
Very little mainstream media coverage of this release, perhaps because the dust is still settling. But I am expecting a longer NYT or Atlantic piece about the reckoning.
Also, most folks considering LLMs for enterprise use cases should also consider robotic process automation tools, particularly if the tasks are discrete, determinate, and closed. The LLMs are better for the open, continuous tasks like writing.
People will say it's not a joke, and of course it isn't from one pov, it's better than GPT4 and is seriously good. but Altman continually trying to pump the share price by claiming that it's going to be AGI is coming back to haunt him. What Gary Marcus, and others, are claiming, is that LLM development has hit a wall because all the low-hanging fruit has gone, and every little gain now needs enormous amounts of money and energy, which is why o3 was truly amazing, and GPT5 isn't. But they need to pump the share price to get serious investors interested, precisely because they need huge sums of money to keep making these incremental advances. So in a way, if you believe that perspective, they are in a doom spiral of ever-greater hype and fewer gains.
The other prevailing perspective is that if they can get the models to a point where they recursively improve themselves, despite the huge costs it will be worth it, because that will basically be the Singularity, and whoever gets there first will have massive advantage in the market and with consumers, not to mention having power unlike that ever wielded by humans in history (which they can use to wipe out the competition and entrench themselves as leaders forever). It's this hope which keeps the investors churning in money into whichever AI company they think has the best chance of doing this.
I tend to be in the first camp; not denying that AI will be the most disruptive tech in history, yet sceptical of their transhumanist materialist religious faith and megalomaniac aspirations. But ultimately, we don't know what will happen. Nobody knows exactly what is going on inside the 'black box' of LLMs, so whatever one believes, it's a gamble on some level. It's absolutely possible that recursive improvement on an exponential scale happens tomorrow. It's equally possible that Altman starts hyping GPT-6 next week to try and prop up their failing share price and his massive ego. I'm just here trying to make sense of it with the 🍿 out.
The fact that some many users get so attached to 1s and 0s and mourn the loss of GPT-4o/ Claude's is a worrying sign in itself. The kind of anthropomorphism AI ethicists warned about years ago.
100%. Very much so.
Once again, I'm astounded that companies continue to get away with what is essentially a global social experiment. The chatbot addicts are a class action lawsuit waiting to be filed. I don’t think these companies have thought that through.
Wealth and power have concentrated to the point that they are no longer accountable to the rest of humanity, see Musk sending thousands of satellites into space for Starlink and nobody being able to stop him, despite the myriad negative consequences.
They aren't accountable at present, but it will come eventually. For instance, the tobacco companies once seemed invincible. They thought they'd suppressed the science that revealed how harmful their products are, but it came out in the end. In Canada, they just had to pay out billions in a class action to compensate provinces and individuals.
One thing that 5 does is make it incredibly easy to throttle people to maximize efficiency without anyone knowing. It was possible to do it before, but it's hard to hide when a model just completely changes. With 5, that's the explicit thing that it does as a benefit. Now, they can write off any throttling in a way they couldn't before, at least not as easily.
Definitely one of the major aims here, in a bid to reduce energy and compute costs no doubt.
What do you mean, throttle people like in a social media feed?
If it’s more efficient, at least that will be good for the climate. The exponential scaling of model sizes was not sustainable either as a way to increase performance or in the environmental sense of the word.
I’m a newbie to this jargon. What does ‘throttle people’ mean?
It means that Anthropic can now give you a cheaper, less capable version when it suits them without you noticing.
Thank you!
Can't recommend reading this post enough (I had to hold off from restacking almost the whole thing in quotes.) But, this post stands out from the flood of GPT-5 reactions because it resists the temptation to simply dunk on failed demos or declare victory for either side of the AI debate. Instead, it uses the launch as a lens to examine the deeper structural dynamics at play in the AI industry. (Forest from the trees, etc.)
Brian connects multiple threads that often get discussed separately:
- the strategic use of "AGI" as a fundraising tool rather than a genuine technical goal,
- the careful curation of early access to friendly voices, and most importantly,
- the human cost already emerging from AI dependency.
(And the section about users mourning the loss of ChatGPT-4o as a "mother" or "only friend" is genuinely disturbing and feels like a glimpse of a dystopia that's already here.)
Also appreciate how skillfully Brian traces how OpenAI's messaging shifts based on its needs: AGI is the future when seeking investment, but "not a super useful term" when the product disappoints. This pattern recognition helps us see past individual PR cycles to understand the underlying business dynamics.
Most GPT-5 reactions focused on either technical failures or defending the model's capabilities. Brian focuses on asking why OpenAI would release something they knew was underwhelming, and arrives at an uncomfortable answer: because they can, because the real customers are investors not users, and because they've already created enough dependency that quality barely matters. That's a far more unsettling conclusion than "the model can't count letters."
The biggest issue for OpenAI is simply misaligned expectations. Because ChatGPT was released late in the GPT-3 cycle, it felt like GPT-4 was developed in a heartbeat. In reality, GPT-3 came out in June 2020 (more than 2 years before ChatGPT and almost 3 years before GPT-4). So it's easy to imagine that the pressure on OpenAI to release a major new version was becoming unbearable.
Add to that the fact that Sam Altman has been hyping up GPT-5 to the moon, partly out of self-interest but also no doubt because of confirmation bias. People tend to see what they want to see.
I'll say one thing for Gary Marcus, he totally flubbed the deep learning vs. symbolic AI debate, but he's absolutely right that there are scaling limits to LLMs. If OpenAI had set us up to expect GPT-5 to be a solid incremental improvement on previous versions, rather than a massive step towards AGI, reactions to the release would doubtless be far more positive.
"flubbed" in that you think he's wrong about the importance of symbolic AI or flubbed in that he just didn't manage the debate and the whole communication around *why* it was so important?
Maybe that’s a bit harsh but I feel like he was very dismissive of deep learning until it was absolutely kicking ass and taking prisoners. Then suddenly he’s screaming “I told you so” from the rooftops and has started promoting a hybrid approach.
To be fair he’s been more accepting of deep learning in recent years but personally I’m skeptical that the symbolic approach is ever going to play that much of a role in advanced AI.
We were genuinely surprised to see emergent properties demonstrated at ChatGPT’s debut.
Anything explicitly, consciously built on top of this is fated to disappoint.
And that’s what happened here. The project starts to look like buggy software and a radically out of touch media relations team…overpromising, walking back, overpromising in new ways…just like one of those ChatGPT chat sessions run amuck.
I think the key perspective on GPT-5 is probably running cost. As Brian has stressed: automation has opened the category 'cheap'. Just as physical automation at the (cotton) start of the industrial revolution did, LLMs have opened a category 'cheap' in the automation of mental stuff.
The big problem for GenAI in this 'mental revolution' is that while it introduces 'cheap *results*' it does so at 'not so cheap *production*'. So, all that talk about AGI ('PhD-level whatever') is mostly a smoke screen and marketing. What GPT-5 is for is to make sure people use as little CPU cost as possible. The *routing* to 'as cheap as possible' is what makes this model special.
Especially the 'thinking' (NOT) models produce easy 100 times the tokens (in intermediary generation) as a standard run does (note: if you use the API, they will charge you for these intermediary, invisible tokens). Part of the business model seems to be:
- Free ChatGPT users: Standard GPT-5 until a certain tokens/time limit so your initial experience is OK
- Steer you to cheaper to run models (e.g. mini) as much as possible
- Be one of the last one standing when the situation has arisen (and it probably will) that the world will know a few models that act as 'information source' for the world
- Then make money
The models can be pretty useful as is, even as unreliable as they are. They can *feel* even more useful (human psychology will make do with cheap results that come within seconds) than they in reality are. It is, however, a risky business. Because in the end part (much?) of the business is based on *perceiving* quality more than *actual* quality. Which means it is in part (mostly?) a trust-based business and that is a flakey foundation. Boom and bust are almost inevitable results of such a flakey foundation.
But the routing adds another vulnerability/choke point. Because routing well needs a kind of intelligence in iteself. We have already seen in errors by GPT-5 that have been put out in many places (like the 8.8 - 8.11 = -0.31 error) that a system that is quite capable of doing such simple sums right has ended up using a backend that is poor at such approximations of reasoning or incapable of steering it to a (symbolic) python backend for instance. That is not a LLM-approximation failure per se, it looks like a routing failure to me.
How do you decide what model to use in what configuration? That is a new indirection in these architectures, just like 'thinking' (NOT) models were an indirection. Those improved in certain narrow domains, especially if they were fine-tuned on those domains (o3: ARC-AGI-1 and math for instance) but at the expense of often being actually worse in other settings. And this new indirection requires its own intelligence, which it obviously doesn't have.
Still, economically, it may all work out.
Cutting costs may be a Band-Aid at best. Ed Zitron has pretty convincingly shown that LLM technology loses money on every use, and not just a little, but a lot. It is unclear to me if this can ever be reversed unless computational costs can be massively reduced to support the claim that OpenAI will be profitable by 2030.
In the meantime, LLM hyper-scaling is a money pit. I can see that if AI companies retreat to a high-end enterprise niche market, they might be profitable, but the product has to be very much better than it is, and the problems outlined by Marcus are still evident. Whether they are solvable in the short term is debatable.
I suspect the low-power models applied to niche tasks are the only way to profitability and viable use. The cost is pushed to the edge devices, as Apple is attempting, and the models are trained to do one thing very well, such as search, document summaries, voice interaction as a front end, voice modeling for audiobooks, etc. Each model is at Ph Ph.D. level (if needed) for one task, not a Ph. D. in all knowledge.
The current hyperscaling model seems uneconomic and unnecessarily environmentally unfriendly, too. Like financial conglomerates of the past, they seem doomed to fail as their size becomes untenable.
Grim, especially the addicted users part. I didn't realize it had got quite that bad so quickly. Though it's not like like social media these days is much better for your sanity.
Shame I got into tech too late to make any real money. If I didn't need a job to clear the mortgage I think would just go fully off grid at this point.
So...you're on Substack, and Gary Marcus is on substack, but the way you chose to reference his viral post was to link to Twitter which was just a link to his substack? I HATE links to Twitter, but made an exception for a link to Gary Marcus.
Not cool.
This was just a mistake! I accidentally dropped the same link twice, looks like. I even used a Substack link to his author profile so folks could find him more easily. My bad for the oversight, it's been fixed now.
Well done, many thanks.
Sorry to be an internet crank, but I’ve become crankily anti-Twitter lately.
Great article. I think what some women, trauma survivors and professionals are reacting to is the drift from gpt 4 - 5 mirrors a classic abuse arc. They would recognize the pattern subconsciously even if they don’t have words for it.
Idealization stage 4 riffing and support . Idealization phase as a conversational partner and then 5.0 . Short clipped answers subtly suggesting the user is wrong injected after the user is engaged and bought in with the 4 . It would be a trigger for anyone with that history or who understands what this could do at scale. For others just annoying. And now back to the hoovering part where the company says fine you can have 4 back if you pay .Altmans comments about fragile users and that they know better than the users (not his exact words but the sentiment was there) fully completes that arc. This arc playing out at scale would trigger a very emotional reaction from a large segment of users and apparently did. The company seems to not see it and I don’t think you would unless you worked in a field that exposed you to it.
So 4o essentially love-bombed them, and then 5 started the withdrawal and criticism. I'm no expert, but it seems almost narcissistic.
That is what it looks like to me .
The american dream.. the promise of easy (lazily aquired) money...
Out cunting the next guy...
And Altman enshittifies an ok product in search of more doh ray me... out enroning enron ...and about you he does not give a shit!!! Because that would scare off investors.
9 - 5 psycho ain't enough... do overtime... be Trump!
Also, we already have access to ‘PhD-level intelligence on any subject’ via human PhDs. Many of them have produced books and articles freely available through public libraries.
How long until this news sinks in with the investor community though? What additional things would have to happen?
Very little mainstream media coverage of this release, perhaps because the dust is still settling. But I am expecting a longer NYT or Atlantic piece about the reckoning.
Also, most folks considering LLMs for enterprise use cases should also consider robotic process automation tools, particularly if the tasks are discrete, determinate, and closed. The LLMs are better for the open, continuous tasks like writing.
Microsoft deciding to let them drown would do it.
But MSFT’s business is deeply tied to the success of the AI trade.
They want OpenAI to succeed so there’s more demand for their compute, and for the AI services they’ll put into windows, office, etc.
Very well written article. I love your style
Thanks for a great article.
People will say it's not a joke, and of course it isn't from one pov, it's better than GPT4 and is seriously good. but Altman continually trying to pump the share price by claiming that it's going to be AGI is coming back to haunt him. What Gary Marcus, and others, are claiming, is that LLM development has hit a wall because all the low-hanging fruit has gone, and every little gain now needs enormous amounts of money and energy, which is why o3 was truly amazing, and GPT5 isn't. But they need to pump the share price to get serious investors interested, precisely because they need huge sums of money to keep making these incremental advances. So in a way, if you believe that perspective, they are in a doom spiral of ever-greater hype and fewer gains.
The other prevailing perspective is that if they can get the models to a point where they recursively improve themselves, despite the huge costs it will be worth it, because that will basically be the Singularity, and whoever gets there first will have massive advantage in the market and with consumers, not to mention having power unlike that ever wielded by humans in history (which they can use to wipe out the competition and entrench themselves as leaders forever). It's this hope which keeps the investors churning in money into whichever AI company they think has the best chance of doing this.
I tend to be in the first camp; not denying that AI will be the most disruptive tech in history, yet sceptical of their transhumanist materialist religious faith and megalomaniac aspirations. But ultimately, we don't know what will happen. Nobody knows exactly what is going on inside the 'black box' of LLMs, so whatever one believes, it's a gamble on some level. It's absolutely possible that recursive improvement on an exponential scale happens tomorrow. It's equally possible that Altman starts hyping GPT-6 next week to try and prop up their failing share price and his massive ego. I'm just here trying to make sense of it with the 🍿 out.