The Hard Part of AI Isn't Intelligence Anymore
The real challenge is no longer making AI capable. It is making AI reliable, affordable, measurable, and useful in the messy world of production.
For the last few years, the AI industry has been obsessed with one question:
How intelligent can these models become?
Can they write software? Can they reason? Can they use tools? Can they operate autonomously? Can they replace repetitive human work?
The answer, increasingly, is yes.
Frontier models can write and debug code, search the web, analyze documents, generate images and audio, operate tools, and execute multi-step tasks. Agentic coding systems can now work across repositories and run tests rather than simply autocomplete a line of code.
But something interesting is happening as these systems move from demos and experiments into real companies.
The difficult problem is changing.
The question is becoming less:
“Can AI do this?”
and more:
“Can we operate AI doing this reliably, economically, safely and repeatedly?”
Recent events at Microsoft, Uber, Amazon, Starbucks and Pizza Hut provide very different examples of the same underlying problem. Some involve cost overruns. Others involve operational failures or integration problems. And some are allegations contained in ongoing litigation rather than established findings.
Taken together, however, they point toward a larger engineering challenge:
AI capability is advancing faster than many organizations' ability to deploy and control it.
The AI Capability Problem Is Becoming the AI Deployment Problem
The first generation of enterprise AI projects was relatively easy to understand.
Take a model. Give it a prompt. Get an answer. Maybe build a chatbot around it.
The new generation is fundamentally different.
Every additional capability introduces another potential source of:
A model can be extraordinarily capable and the overall system can still be a bad production system.
That distinction is becoming increasingly important.
Microsoft: When an AI Coding Tool Becomes Too Popular
Microsoft is an especially interesting example because it isn't an outsider struggling to adopt AI.
Microsoft is one of the world's largest AI companies.
Yet in May 2026, The Verge reported that Microsoft was preparing to phase out most internal Claude Code licenses in parts of the company, with the Experiences + Devices organization moving developers toward GitHub Copilot CLI. The decision was described as both strategic and financial.
That distinction matters.
It would be inaccurate to summarize the story as:
“Microsoft stopped using Claude because Claude was bad.”
Microsoft continues to use Anthropic models in other products and services. The reported decision was primarily about consolidating its internal coding workflow around GitHub's tooling, with financial considerations also involved.
But there is an important lesson here.
AI usage inside an organization is no longer just a software-license decision.
Agentic coding tools can consume significant amounts of model inference. The more developers use them, the more unpredictable consumption can become.
Uber: The Token Bill Can Arrive Faster Than Finance Expects
Uber provides an even clearer example.
According to reporting from TechCrunch, Claude Code spread across roughly 5,000 Uber engineers much faster than the company's financial models anticipated.
Uber reportedly exhausted its 2026 AI budget by April.
The company subsequently introduced internal spending controls, including a reported $1,500 monthly cap per employee and agentic coding tool, with exceptions available in some circumstances.
This creates an important distinction between traditional enterprise software and agentic AI.
Traditional Software
Agentic AI System
And the variables aren't necessarily stable.
An engineer might use an AI coding agent for one small task.
The agent might:
The user sees one task.
The infrastructure sees dozens of operations.
This is one reason cheaper models do not automatically mean cheaper AI systems.
If the cost per operation falls by 50% while the number of operations increases by 10×, the total bill can still increase dramatically.
Amazon: When a Cheap Software Mistake Becomes an Expensive AI Mistake
Amazon provides an even more striking example.
The Financial Times reported in July 2026 that Amazon engineers had identified several cases where AI projects generated unexpectedly large costs because of deployment mistakes and insufficient spending controls.
One reported case involved approximately $1.8 million spent on an unsuccessful attempt to use Anthropic's Claude Sonnet to match author information with Amazon ecommerce listings.
The reported spending represented an 860% cost overrun against the project's budget and reportedly took around five months to detect.
Other reported incidents included approximately $541,000 in unexpected costs related to financial-auditing tools and around $134,000 in an AI logistics project.
Amazon pushed back against interpreting isolated incidents as representative of its overall AI operations, saying it is experimenting and learning how to drive cost efficiencies.
That caveat is important.
The lesson isn't:
“Amazon can't control AI costs.”
The more useful lesson is:
Traditional engineering mistakes can have very different financial consequences when computation involves expensive model inference.
A bug in conventional software might waste CPU cycles.
An agentic loop can repeatedly invoke a large model, process large contexts and call external systems.
The architecture of the mistake has changed.
Starbucks: The Model Can Work and the System Can Still Fail
Cost is only one side of the problem.
Starbucks demonstrates another:
Operational reliability.
Starbucks introduced an AI-powered Automated Counting system designed to automate inventory counting for items such as milk and beverage components.
The system used camera and LIDAR data to identify products on shelves.
The promise was straightforward:
Scan the shelf → identify products → count inventory → improve replenishment.
But Reuters reported that the system frequently miscounted or mislabeled products, including confusing similar milk products or failing to identify items.
After roughly nine months of deployment across North American stores, Starbucks retired the system and returned these inventory categories to the same manual counting process used for other inventory.
There is an important nuance here.
Starbucks had previously said the tool improved product availability. The eventual decision to retire it was described by the company as an effort to standardize inventory counting and improve consistency and execution at scale.
This isn't necessarily evidence that computer vision itself was incapable of solving the problem.
It demonstrates something more important:
A model can perform adequately in a controlled evaluation and still fail when embedded inside a messy operational workflow.
A warehouse isn't a benchmark.
A coffee shop isn't a benchmark.
A restaurant isn't a benchmark.
Real-world environments contain:
Production doesn't care how impressive the demo looked.
It cares whether the system works at 8:30 AM on a busy Monday when everything is slightly different from the test environment.
Pizza Hut: Integration Can Become the Failure Point
The Pizza Hut case takes the problem one step further.
A Pizza Hut franchisee, Chaac Pizza Northeast, which operates approximately 111 restaurants, filed a lawsuit alleging that Pizza Hut's Dragontail AI delivery-management system caused cascading operational problems, slower orders and problems involving third-party delivery integration.
The franchisee alleges approximately $100 million in lost business and enterprise value.
Important: the $100 million figure is an allegation in the lawsuit, not an established court finding.
According to Restaurant Dive, Pizza Hut said it was reviewing the claim and would respond through the appropriate legal channels.
According to the complaint, the system changed how kitchen, point-of-sale and third-party delivery operations interacted.
The franchisee alleges that the resulting workflow created longer delivery times and other operational problems.
This is precisely why AI deployment is fundamentally different from an AI demo.
You aren't deploying a model.
You're deploying a socio-technical system.
A failure anywhere in that chain can destroy the value created by the model.
The Pattern Isn't “AI Doesn't Work”
This is where the discussion needs to become more nuanced.
The evidence does not prove that AI doesn't work.
In fact, there is evidence that AI can produce substantial value.
A 2026 Microsoft-affiliated research study examining the rollout of Claude Code and GitHub Copilot CLI reported increased merged pull requests among developers who adopted command-line coding agents, while also noting that a merged pull request is not the same thing as delivered business value.
The study is available on arXiv.
So the problem isn't:
AI = useless.
It is closer to:
AI = highly capable technology whose economic and operational behavior becomes much harder to predict when deployed at scale.
That is a completely different problem.
The Production Evaluation Gap
Traditional software testing asks relatively clear questions:
- Does the API return the correct response?
- Does the function produce the expected output?
- Does the database transaction succeed?
- Does the application crash?
AI systems introduce probabilistic behavior.
Now we need to ask:
- Is the answer factually correct?
- Is it sufficiently complete?
- Did the agent use the right tool?
- Did it take unnecessary actions?
- Did it expose sensitive information?
- Did it consume too many tokens?
- Did it follow the business rules?
- Did it know when to ask a human?
- Did it behave differently after a model update?
And agentic systems add another dimension:
Did the AI do the right thing while getting there?
An agent could eventually produce the correct answer after making ten unnecessary tool calls.
Technically successful.
Economically terrible.
Another could produce the correct answer 99 times and make a catastrophic mistake on the 100th.
Average accuracy might look excellent.
The production risk might still be unacceptable.
This means AI evaluation needs to move beyond:
“Does the model answer correctly?”
toward:
“Does the complete system behave correctly under realistic operating conditions?”
The Hidden Cost Isn't Just Tokens
When companies talk about AI costs, they often focus on inference.
But the real enterprise cost is much larger.
Consider a production AI application:
The model may eventually become a relatively small part of the total system cost.
This is similar to what happened with cloud computing.
Nobody serious about cloud economics calculates only the price of a single CPU hour.
The real cost involves storage, network traffic, databases, observability, backups, engineering and operational overhead.
AI is moving in the same direction.
The Agentic AI Multiplier
Agentic AI makes this even more interesting.
A conventional chatbot might generate one response.
An agent can create a chain of actions.
The system isn't simply generating text anymore.
It is consuming compute to reason and act.
This creates a new engineering discipline:
AI FinOps
Companies need to understand:
The important metric may eventually become:
Cost per successful business outcome
rather than:
Cost per million tokens.
The Human-Replacement Paradox
This leads to another uncomfortable question.
The AI industry has frequently presented automation as a way to reduce human labor.
But reliable AI automation itself requires humans.
You need people to:
So the question isn't simply:
“How many humans can AI replace?”
A more useful question is:
“How much human work is required to make an AI system trustworthy enough to replace another human workflow?”
Sometimes the answer will be extremely favorable.
Sometimes it won't.
And that is why ROI needs to be measured rather than assumed.
Perhaps the Biggest Signal Is Coming From the AI Companies Themselves
There is a fascinating development happening underneath all of this.
The companies building frontier models are increasingly investing in AI implementation and deployment.
Anthropic announced a new enterprise AI services company with Blackstone, Hellman & Friedman and Goldman Sachs. The company is intended to work directly with businesses to identify where Claude can create value, build customized solutions and support long-term deployments.
The resulting venture, called Ode, has been described as a $1.5 billion AI implementation company.
OpenAI has also created its own deployment-focused business.
AWS committed $1 billion toward a forward-deployed engineering organization.
Microsoft launched Microsoft Frontier Company with $2.5 billion in funding to help customers select and integrate AI technologies, including models from outside Microsoft.
TechCrunch reported on this broader shift toward AI implementation.
Think about what this means.
The companies selling the intelligence are increasingly building businesses around helping customers deploy the intelligence.
That is a major signal.
The frontier model may be the product.
But implementation may be the bottleneck.
Enterprise AI Is Entering Its Systems Engineering Phase
The first phase of generative AI was largely about:
Model capability.
Then came:
Application development.
Now we are entering:
Systems engineering.
The winning architecture won't simply be:
It will look more like:
This is where a lot of enterprise AI engineering will happen.
Not in inventing another chatbot.
In building systems that make probabilistic intelligence behave predictably enough for production.
The Data Suggests This Isn't an Isolated Phenomenon
S&P Global's research provides another useful piece of context.
Its Voice of the Enterprise research found that the proportion of organizations abandoning most of their AI initiatives before production had increased from 17% to 42%.
The research also found that organizations reported abandoning an average of 46% of AI proof-of-concept projects before production.
These numbers should not be interpreted as saying that 42% of all AI projects universally fail. They come from a survey and concern organizations' reported AI initiatives.
But they are nevertheless a useful indicator of the growing gap between experimentation and production.
And that distinction is critical.
A proof of concept asks:
“Can we make this work?”
Production asks:
“Can we make this work every day, at scale, within budget, safely, with measurable business value?”
Those are completely different engineering problems.
What an AI-Ready Enterprise Actually Looks Like
An AI-ready organization isn't simply one where everyone has access to ChatGPT, Claude or Gemini.
It is an organization capable of answering basic operational questions.
Cost
- How much does each workflow cost?
- What happens if usage increases 10×?
- What prevents an agent from entering an expensive loop?
Reliability
- What happens when the model is wrong?
- What happens when an API fails?
- What happens when the model provider changes behavior?
Evaluation
- How do we know the system is improving?
- How do we test it after every model or prompt change?
Security
- What can the agent access?
- What can it modify?
- Which actions require human approval?
Observability
- Can we see what the agent did?
- Which tools did it call?
- How many tokens did it consume?
- Why did it fail?
Recovery
- Can the system roll back?
- Can a human take control?
- Can the organization switch models?
Business Value
- Did the system actually improve revenue?
- Did it improve productivity or customer satisfaction?
- Did it improve delivery time, quality, or operational cost?
If the answer to these questions is unknown, deploying AI at scale becomes an experiment disguised as production.
The Real AI Maturity Curve
Stage 1
"Can AI do it?"
Stage 2
"Can AI do it reliably?"
Stage 3
"Can AI do it economically?"
Stage 4
"Can AI do it safely at scale?"
Stage 5
"Can AI produce measurable business value?"
Stage 6
"Can we operate and continuously improve it?"
Many organizations are still moving from Stage 1 to Stage 2.
The industry narrative, however, sometimes talks as if everyone is already at Stage 5.
That mismatch is where many of today's problems originate.
So, Is AI Overhyped?
That isn't the conclusion I would draw.
If anything, these examples demonstrate the opposite.
AI is powerful enough to create new classes of engineering and economic problems.
- Uber didn't spend heavily because AI was irrelevant.
- Microsoft didn't need to manage coding-agent adoption because nobody was using it.
- Amazon didn't encounter unexpected inference spending because AI had no computational impact.
- Starbucks didn't need to retire an automated inventory system because AI had no role in operations.
The problem is precisely that these technologies are capable enough to become deeply embedded in real systems.
The industry is therefore moving beyond:
“Can AI do something impressive?”
and toward:
“Can organizations build the infrastructure required to control something this powerful?”
The Next Competitive Advantage May Be Predictability
For years, AI competition has been framed around intelligence.
- Who has the smartest model?
- Who has the largest context window?
- Who has the best benchmark scores?
- Who can reason the best?
Those questions will continue to matter.
But another competition is emerging.
Who can make AI predictable?
- Predictable in cost.
- Predictable in latency.
- Predictable in behavior.
- Predictable in failure.
- Predictable in security.
- Predictable in evaluation.
- Predictable in business outcomes.
That may become just as important as raw model intelligence.
Because enterprises don't ultimately buy intelligence.
They buy outcomes.
Conclusion: We Solved Part of the AI Problem
The AI industry has made extraordinary progress.
Models that seemed impossible a few years ago are now accessible through APIs and developer tools.
- Agents can write code.
- AI can interact with enterprise systems.
- Computer vision can count inventory.
- Models can reason over large bodies of information.
The capability is real.
But capability is only the beginning.
The difficult engineering problem is now what happens around the model.
- Can we control its cost?
- Can we evaluate it?
- Can we observe it?
- Can we integrate it?
- Can we secure it?
- Can we recover when it fails?
- Can we measure whether it actually created value?
And perhaps most importantly:
Can we trust it enough to put it in charge of something that matters?
The answer won't be the same for every use case.
- Some applications will deliver extraordinary returns.
- Some will fail.
- Some will need humans permanently in the loop.
- Some will become almost completely autonomous.
That is exactly why the next stage of AI shouldn't be about blindly deploying more intelligence.
It should be about engineering the systems around that intelligence.
Because the hardest part of AI may no longer be making the machine smarter.
It may be making its intelligence predictable enough for the real world.
References and Further Reading
-
Microsoft and Claude Code
The Verge : Microsoft starts canceling Claude Code licenses -
Uber and AI spending
TechCrunch : Uber caps employee AI spending after blowing through budget in four months -
Amazon and unexpected AI costs
Financial Times : Amazon finds cases of AI causing runaway spending on tech projects -
Starbucks Automated Counting
Reuters : Starbucks scraps AI inventory tool across North America -
Pizza Hut / Dragontail lawsuit
Restaurant Dive : Pizza Hut franchisee lawsuit over AI delivery system -
AI coding-agent productivity research
arXiv : Adoption and Impact of Command-Line AI Coding Agents -
Enterprise AI project abandonment
S&P Global : Generative AI shows rapid growth but yields mixed results -
Anthropic's enterprise AI services
Anthropic : Building a new enterprise AI services company -
AI implementation and deployment
TechCrunch : The next AI business may be implementation, not models -
Microsoft Frontier Company
Reuters : Microsoft launches firm to help companies adopt AI
A note on the evidence
The examples above should not be interpreted as evidence that AI as a technology is failing. They represent different types of problems: unexpected costs, adoption economics, workflow failures, integration issues and projects that did not progress successfully to production.
Some claims are based on company statements or reporting from people familiar with the situations; the Pizza Hut $100 million figure is specifically an allegation in ongoing litigation.
The broader argument of this article is therefore not “AI doesn't work.”
It is:
As AI becomes more capable, the engineering challenge is moving from demonstrating intelligence to operating that intelligence reliably, economically and safely at scale.
