This is Part 1 of The Human Reckoning: A Future of Trust Series

I grew up in a world where if I wanted to know something, I had to go looking for it.

I wrestled with ideas, searched for what others had already considered, and worked through my own conclusions. I learned to leave room for debate. Curiosity energized the process, even under the pressure of deadlines.

There was no Google. No answer waiting in my pocket. Certainly no artificial intelligence capable of explaining almost anything I could think of asking, then answering my next question before I had fully formed it.

By the time the internet changed how we found information, I had already spent my formative years learning how to think without it. My research, writing, and decisions were mine, and so were my mistakes. Technology changed my tools, but much of the underlying machinery was already in place.

A child born today may never remember a world without artificial intelligence. AI will be there as they learn to write, research, solve problems and make decisions. It will be part of the environment in which those capabilities develop.

I think about that distinction more and more, particularly because my own experience with AI has surprised me.

In the beginning, like many people, I was largely discovering what it could do. And it felt magical. The answers came fast, the time savings were profound and the possibilities for discovery seemed endless. Over time, I began looking more closely at those answers. They weren’t always accurate. And even when they were good, I wondered if whether I would have reached a similar conclusion through my own process.

I became increasingly interested in what was happening during the interaction itself. The conclusions AI reached interested me less than what might be happening to my own thinking in the process. I wanted to recognize my voice and perspective in whatever we produced together.

That awareness led me to my current relationship with AI. I challenge it. I disagree with it. I ask what I am missing or where my thinking may be flawed. An answer often sends me in a direction I hadn’t considered and creates more questions.

Now, rather than letting AI do my thinking for me, I find myself reading more, exploring more and learning more. In short, I am more intellectually challenged and more curious than I was before. AI gives my thinking more to engage with, but it took me a while to get here. That raises a question I think is much bigger than whether people know how to use AI.

How do we learn to think with AI without surrendering the thinking to it?

We are talking a great deal about AI literacy. Schools, businesses and governments are trying to figure out what people need to understand about these systems and how to use them effectively and responsibly.

That’s essential. But it is only part of the literacy we need. We also need to understand which parts of being human we want artificial intelligence to support, which we are comfortable allowing it to replace, and which we need to keep exercising ourselves.

That distinction is harder than it sounds. We have been designing friction out of human life for a very long time, and most of us have benefited enormously. Technology has made information easier to find, calculations faster, and navigation almost effortless. I wouldn’t willingly give up those conveniences simply because the old ways required more effort.

Some forms of effort are simply inefficient. Others are building something we can’t see while we’re in the middle of them. Figuring something out for ourselves requires us to stay with uncertainty a little longer, decide what matters, or discover that our first assumption was wrong. Those capacities require practice.

Which leads me to a question we should be asking before we rush to make every human task easier:

Before we design friction out of an experience, should we understand what that friction was developing in us?

Not All Friction Is the Same

There is a body of research behind this distinction. Learning scientists use the term “desirable difficulties” to describe conditions that make learning somewhat harder in the moment but can improve retention and understanding over time. Retrieving something from memory rather than immediately looking it up is one example. Working through a problem before being shown the solution can be another.^[i] The value comes from what the effort asks our brains to do, particularly when it exercises a capability we want to strengthen.

Human beings have always looked for ways to make thinking easier. Writing itself is a form of cognitive offloading. Instead of holding everything in memory, we put it somewhere outside ourselves. Books expanded that capacity. Calculators took over computation. Search engines and GPS moved still more work outside our heads.^[ii]

Each time, we gained something. We also changed what we practiced. I don’t know nearly as many phone numbers as I once did. I sometimes use GPS to get somewhere I have driven before. And I certainly don’t want to return to searching through an encyclopedia when I can find what I need in seconds.

This is adaptation. The more interesting question is what we are adapting toward. AI raises the stakes because it can take on so much more of our cognition. It reaches into the thinking process itself, helping us make sense of information, develop an argument, identify patterns or decide what to do next. That makes the boundary between assistance and substitution harder to see.

If AI writes an email I already know how to write, perhaps very little has been lost. If it summarizes hundreds of pages so I can spend my time thinking about what they mean, it may have increased the value of my attention.

The calculation changes when AI does the work through which someone is still building the underlying capability. Students are the easiest example: a student handed an interpretation of a novel before forming their own is one that comes to mind. In business, a young analyst might produce a sophisticated strategy deck without ever deciding which information mattered enough to put on the page.

The output may be excellent. But what happened inside the person who produced it? That question is going to become increasingly important because the quality of an artifact may tell us less and less about the capability of the person behind it.

For most of my career, there was at least some relationship between the two. A thoughtful analysis generally required someone to do thoughtful analysis, and a well-constructed argument required enough understanding of the subject to construct it. AI changes that relationship.

It can produce evidence of capability without necessarily producing capability itself.

That creates a challenge we have barely begun to address, and it lands hardest on people who are still learning. For organizations, this raises a harder question: if junior work no longer requires junior struggle, how do people develop the judgment that senior roles depend on?

The Difference Between Removing Work and Removing Practice

We need to distinguish between unnecessary friction and productive friction.

Unnecessary friction consumes time or cognitive energy without giving much back. There is little reason to preserve repetitive work simply because humans used to have to do it.

Productive friction asks something more of us. It can be the process of deciding which evidence deserves more weight, staying with uncertainty when two explanations seem plausible, or recognizing that we don’t know enough yet.

And sometimes it is simply making a decision, discovering we were wrong and carrying that experience into the next one. These experiences become practice, and over time, that practice helps develop the judgment we draw upon when we need it.

This is where the conversation about AI literacy needs to expand. Teaching someone how to prompt an AI system is useful. Teaching them to verify an answer is essential. But neither necessarily develops the judgment required to know what should be questioned in the first place.

We need enough knowledge to notice when something doesn’t look right and enough curiosity to wonder what might be missing. We also need the confidence to challenge an authoritative answer and the humility to recognize when our own thinking may be wrong.

These are human capabilities, and they develop through use.

Growing Up with the Answer

Those of us adapting to AI today have an advantage we should recognize. We remember before.

I know what it feels like to stare at a blank page with no one there to suggest the first sentence. I know what it is to research a question without having the answer synthesized for me. I have made decisions with incomplete information and learned through experience which questions I should have asked sooner.

My generation did not necessarily develop better judgment. We simply had a great deal of practice doing certain things without intelligent assistance. Millennials and Gen Z came of age as search engines, smartphones and social media changed how people found information, communicated, and learned. Many Millennials remember childhood before smartphones, while Gen Z grew up with far more immediate access to digital information. Both are now adapting to generative AI as students, young adults, parents, and professionals, but it arrived after much of their early learning had already taken place.

The generations behind them won’t have that dividing line. An intelligent system may be present the first time they struggle with a math problem, write a story, or try to understand something unfamiliar.

The possibilities for learning are extraordinary. Imagine being eight years old and able to follow almost any thread of curiosity as far as you want to take it. A question about why animals migrate could lead to climate, geography or evolution. Each answer can open another door, with the explanation adjusting as the child’s understanding grows.

AI could become an extraordinary engine for curiosity.

It could also make uncertainty disappear very quickly.

A child asks. AI answers. The assignment gets done. What matters is not simply whether the child used AI. It is what the child was doing cognitively while using it. That is where our attention needs to shift.

Share

We Don’t Know What We Don’t Know

Metacognition is our ability to think about our own thinking. It includes recognizing how well we understand something, where our knowledge ends and when our confidence may exceed what we actually know.^[iii] Those capacities become especially important when we interact with AI.

Our blind spots extend beyond the limits of what we know. There is an enormous body of knowledge we may not even be aware exists. We can recognize a gap in our understanding when we know enough to identify it. But sometimes we lack the awareness needed to formulate the question in the first place.

This is one of the possibilities I find most exciting about AI. Much of our curiosity begins with a spark: something we read, a conversation, an unexpected connection. Without that exposure, we might never know there was something worth exploring. AI can supply those sparks, opening doors beyond the boundaries of our own experience. But we have to stay curious enough to walk through them rather than simply accept what is presented.

There is a tension here. To recognize the limits of our knowledge, we need opportunities to encounter those limits. If an answer appears before I have tried to retrieve what I know or formulate my own hypothesis, I may get the information I need without discovering where my understanding ended.

That is why sequence matters. If I form an opinion before I ask AI, and consider how confident I am in it, then I can explore what the system surfaces that I hadn’t considered. I can see where we disagree, and whether the new information changes my view. I can also ask what I hadn’t thought to ask.

I saw the difference this makes with imposter syndrome, a subject I have spent years developing my own approach to. When I asked AI about it without bringing my perspective, I got a well-organized summary of what every other expert has said. Nothing was wrong with it, it simply reflected the consensus my work challenged. When I brought my own methodology and framework into the conversation, the difference was night and day. AI could test my thinking, extend it, and show me where it needed more support. It could only do that because I had already done the work of forming a view.

Forming a view first is a small change, but it keeps me cognitively present while expanding what I can discover. Perhaps that is part of what we should mean when we talk about AI literacy: knowing how to remain an active thinker while receiving increasingly sophisticated assistance.

Designing for the Human

Much of our collective learning about AI has happened through individual experimentation. I’ve certainly learned through trial and error, gradually discovering how to engage with AI in ways that challenge and expand my thinking.

But why should we leave that discovery entirely to chance?

We have invested enormous resources in teaching machines to interact more naturally with humans. It’s time to put comparable attention into helping humans develop the skills and habits that make those interactions intellectually valuable.

The way we learn to use AI and the way AI is designed to interact with us will both shape the capabilities we develop.

That extends well beyond schools. Parents are already deciding when children should have immediate access to an answer. Educators are reconsidering what learning looks like when AI is available. Employers face a different version of the same problem when an entry-level employee can produce work that once required years of experience. And technologists are making design choices that will influence whether AI simply answers us or helps us think.

All of this comes back to a deceptively simple question:

What are we trying to develop in the human being?

Once we know that, we can make much better decisions about what the technology should do. There are plenty of things I am happy to hand over to AI, particularly work that adds little to my thinking. At other times, I want help without giving up the task itself.

And increasingly, I find myself wanting something else from it: resistance. A little pushback. I want it to tell me where my argument is weak or surface something I haven’t considered. I want it to make me rethink the premise when the premise deserves rethinking.

In those moments, efficiency isn’t the goal. Better thinking is.

We have become very good at asking how much human effort technology can remove. We also need to understand which kinds of effort are worth preserving and where AI might actually help us deepen them.

There is enormous territory between preserving old ways of working and handing over every cognitive task we can. That is where we learn to think with AI without surrendering the thinking to it.

What Are We Designing For?

We are still early enough in this transition that many of the norms around how we use AI have yet to be established. I see this as an opportunity.

So far, much of the conversation has focused on what AI can do and how much more productive it can make us. We measure time saved, quality of output and work automated. Those are reasonable measures, but they tell us very little about what happened to the human in the process.

We need another measure alongside them: After the interaction, is the human more capable, less capable, or simply more productive?

Those outcomes are not interchangeable. If AI helps me finish something in half the time, that may be valuable even if I learned nothing from the experience. Not every task needs to develop me.

But sometimes something more happens. AI helps me see something I hadn’t considered, understand something more deeply or recognize a flaw in my reasoning. In those moments I become more curious and ask deeper questions. My output improves, but what excites me more is that I do too. This is the outcome that makes me optimistic about where this can go.

As we consider future generations, we cannot simply hand them extraordinarily capable tools and assume judgment will follow. Judgment requires practice. So do curiosity and the ability to recognize when we don’t know enough. If we value those capacities, we need to create environments in which they can develop.

Technology should remove friction that no longer serves us. But before we design it out of an experience, we should understand what that friction was developing in us.

AI offers an opportunity to replace some of that effort with something even more valuable: productive challenges that strengthen our thinking. By questioning our assumptions, introducing new perspectives and helping us discover what we don’t yet know, AI can do more than make thinking easier. It can help us think better.

This won’t happen automatically. It will require intention from the people building these systems and from the rest of us deciding how they enter our classrooms, workplaces, homes and lives.

That is where the idea of a new literacy begins. It goes beyond learning how to prompt a machine or recognize when it is wrong. It asks us to understand when assistance expands our capabilities and when it begins replacing the experiences through which those capabilities develop.

We are still learning how to think with AI without surrendering the thinking to it. And as we do, we are also designing the environment in which human capability will develop alongside increasingly capable machines.

We should be every bit as intentional about developing the human as are about developing the machine.

^[i] Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In Psychology and the Real World (pp. 56–64). Worth Publishers. EBjork_RBjork_2011.pdf

^[ii] Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading. Trends in Cognitive Sciences, 20(9), 676–688. Cognitive Offloading - UCL Discovery

^[iii] Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911. Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry.

This is Part One of The Human Reckoning: A Future of Trust Series

Trust is fragile.

When it comes to trust, I have always believed that it begins within. How we trust ourselves influences how we extend trust to others, and often how trustworthy we are in return. Human behavior, and the internal patterns that shape it, has been my passion for much of my life.

When I began The Future of Trust, my intention was to go wherever trust took me. From artificial intelligence and leadership to our own internal landscapes.

As I started writing, my attention was largely directed outward towards artificial intelligence, so that became the direction of my first essays. I was watching, and more than that, participating in, a transformational era unfolding around us.

I have been building AI for humans since 2023, and I believe deeply in the positive impact it can have on humanity. But a hopeful outlook doesn't relieve us of the responsibility to build responsibly. It requires us to look just as carefully at the potential risks as we do at the possibilities. Some of the upstream issues our team anticipated ultimately informed the way we designed our own technology.

That was the impetus for The AI Reckoning, a twelve-part series exploring my concerns about an industry deploying technology before we fully understood its second-order consequences.

I was watching builders and enterprises optimize for speed to market while responsibility to people struggled to keep pace. At the same time, a younger generation was pushing back against a future they didn’t design and weren’t sure they wanted.

Since that first article was published, and with the series now complete, many of the upstream effects I wrote about are coming into focus. Yet much of the conversation about what comes next is still happening among the people building technology, while the broader population experiencing its effects remains too far outside of it. The trust signals I was seeing haven’t gone away. If anything, they’re getting louder.

Share

The AI Reckoning

As money poured into the industry and the sea of AI companies continued to grow, I took a look at the physical infrastructure required to support this extraordinary growth along with the water and energy demands that didn’t seem to make their way into the business case. I also looked at the economics and what happens when the cost of delivering increasingly sophisticated intelligence collides with expectations for profitability. I explored what it means to build businesses on platforms you don’t control, and what happens inside organizations when technology moves faster than people can absorb the change.

The subjects were different, but the thread running through them was trust.

From the beginning, I described The AI Reckoning as a trust series that happened to be using AI as its lens. I wasn’t interested in predicting whether AI would succeed. I believed it would. I still do. What concerned me was that we were building extraordinary capability at unprecedented speed without paying enough attention to signals that were already telling us something about where it might lead.

Now that I have completed that exploration, I find my attention shifting.

I am less interested in what AI is becoming, and increasingly curious about what is happening to us alongside it.

That shift began showing up in my writing before I had really named it. As I explored empathy, I found myself thinking about what trying to teach a machine to practice it might reveal about the way we practice it ourselves. The growing desire to know whether something was made by a human had me thinking about authorship, effort and trust. More recently, I have been thinking about authenticity and identity, and what happens when technology can amplify our abilities while making it harder to know where our contribution ends and the machine’s begins.

These aren’t really questions about AI. They’re questions about us. And I think I am ready to follow them.

The Human Side of the Equation

I care about human flourishing. It is one of the reasons I am so optimistic about what AI could make possible.

I want AI to increase our capability. I also want it to increase our capacity. If technology can take repetitive work off our plates, help us move through certain tasks more efficiently, or give us access to capabilities we didn’t have before, that can create space for something else. Ideally, some of that space gives us room for greater creativity and innovation, for learning and connection, and perhaps simply the opportunity to rest and restore.

That is an extraordinary possibility, but I am not sure it happens automatically. We have been promised greater efficiency from technology for decades, yet we have a remarkable ability to fill whatever capacity we create. The hour we save doesn’t necessarily become an hour spent thinking or connecting with someone we care about. It can just as easily become another hour of work. Greater productivity can raise expectations for even greater productivity. More access to information can leave us consuming more while understanding less.

AI can give us capacity. What we do with that capacity is still a human choice.

The same is true of capability. AI can help us write, analyze, research, create and solve problems at a level that would have been unimaginable for most people only a few years ago. I find that exciting. I don’t want to preserve difficulty simply because something used to be hard.

But I also don’t want us to confuse being able to produce something with developing the human capability that once went into producing it.

We develop judgment by exercising it, and some of our confidence comes from doing difficult things and discovering that we can. Curiosity develops when we allow ourselves to follow what we don’t yet understand, while creativity can emerge from the wandering, frustration and unexpected connections that happen before we ever reach the finished product.

If AI can shorten some of those journeys, wonderful. But we should understand what was happening to us along the way before we decide that every part of the journey was simply inefficiency.

For me, that isn’t an argument for doing less with AI. It is an argument for expecting more from our relationship with it.

What Are We Designing For?

I have spent years thinking about empathetic AI and what it could look like if we designed technology around people rather than simply asking people to adapt to technology. I still believe in that possibility, perhaps more strongly now than when I began.

I want AI to help us become more capable without making us feel less capable. I want the capacity it creates to give us room for better thinking, healthier connection and greater creativity rather than simply more output. At its best, I believe this technology can support our confidence and judgment and even help us reconnect with one another in healthier, more positive ways.

There is an incredible upward spiral available to us.

Technology can extend human capability and create greater capacity. That capacity can give us more room to think, create, connect, learn and restore. Those experiences can make us more capable, more confident and perhaps more connected to the parts of ourselves that matter most.

But an upward spiral isn’t inevitable simply because the technology exists.

We have to design for it. And I think this is where the conversation needs to go next.

The Human Reckoning

The AI Reckoning asked us to look more carefully at what we are building and what we may not be accounting for as we build it. I don’t consider that conversation finished. There will be developments worth returning to, and I expect I will.

But I am ready to turn the lens around.

The Human Reckoning is not going to be a series about protecting humans from AI or drawing a defensive line around the things machines should never be allowed to do. I don’t think that is where the most interesting or useful conversation lies.

I want to explore what becomes more important for us as AI becomes more capable. That means looking at how our relationship with technology affects the way we think, create, learn, trust ourselves and relate to one another, while remaining active participants in deciding what we want this technology to add to the human experience.

I don’t know that I have answers to all of my questions. In some cases, I am fairly certain I don’t. That’s one of the reasons I want to write about them.

A reckoning doesn’t have to mean something has gone wrong. It can be an accounting. A moment when we look clearly at where we are, what we have created, and what we want to do next.

We are still early enough in our relationship with AI to have a meaningful say in what that relationship becomes. We don’t have to accept every efficiency simply because it is available or preserve every old way of doing something simply because it is familiar. We can decide what technology should take from our plates and what we want the resulting space to make possible.

We can ask more of AI than productivity. We can ask whether it is helping us flourish. And we can ask more of ourselves than simply keeping up with it.

For all the attention we give to designing the future of artificial intelligence, perhaps the greater opportunity is recognizing that we are also designing the future of the human experience.

We don’t have to be passengers in a future AI creates for us. We have the opportunity to be its architects.

That is The Human Reckoning.

About the Author

Sheryl Anjanette is the founder and CEO of Parsley360, an AI company focused on the human side of transformation. She is an author, speaker and behavioral expert whose work explores trust, human behavior and our evolving relationship with AI. The Future of Trust examines what happens to people, organizations and trust as technology changes the world around us.

My guest today is Zohre Shirazi, publisher of AI Wide Open. Zohre examines how human oversight only works when there is still meaningful judgment involved.

But first, a note from Sheryl:

There’s a question I keep coming back to as AI becomes increasingly agentic: What changes when we stop asking AI for help and start giving it permission to act?

For most of the AI era, trust has centered on what a system tells us. We’ve learned to question whether the information is accurate, if we rely on the output, and if we understand how it arrived there.

But an AI that can act on our behalf changes the trust equation. Now we are not only trusting what it knows, we are deciding what it can access, what it can change, what decisions it can make, and how far it can go before a human needs to step in.

That moves us into something much bigger than model performance. Our attention needs to go to authority, boundaries, and accountability.

And I think we are going to have to get much better at separating capability from permission. Just because an AI can do something does not mean we should give it the authority to do it. And adding a human approval somewhere in the process does not necessarily solve the problem.

Subscribe now

So back to Zohre’s thesis that human oversight only works when there is still meaningful judgment involved.

I agree with the premise and would add that this is also where trust becomes architectural. We cannot rely on people to catch every mistake after it happens, particularly as agents operate faster, across more systems, and through longer chains of action. Some of the most important decisions about trust will need to happen before the agent ever acts: what it can access, where its authority ends, when it must stop, and who ultimately owns the outcome.

Zohre takes on the question: When AI acts on our behalf, who owns what happens next?

What I appreciate about her perspective is that she doesn’t reduce the answer to whether we should trust AI more or less. She looks at what we actually need to build around increasingly autonomous systems if trust and accountability are going to keep pace with capability.

I’m very pleased to welcome Zohre to The Future of Trust. Be sure to subscribe to AI Wide Open.

And now, Zohre Shirazi…

_______________________________________________________________________

When AI Can Act on Our Behalf, What Happens to Trust?

As AI moves from answering to acting, we have to rethink authority, accountability, and the boundaries of trust.

I use AI extensively in my work. I rely on it to research ideas, challenge my thinking, work through drafts, and make parts of the editorial process faster. I have become comfortable trusting AI to help me think and produce something. I am still much more cautious about giving an AI agent permission to act on my behalf.

That distinction has become more important as AI systems move beyond generating text, summarizing information, and making recommendations. They can increasingly use tools, access company systems, write and execute code, and complete multi-step tasks with much less human involvement.

For a long time, trusting AI was mostly a question of trusting an answer. You asked a question, AI gave you an answer, and you decided what to do with it. If the answer was wrong, the responsibility for acting on it still sat fairly clearly with the person using the system.

That relationship is changing. Once AI can act rather than simply advise, trust becomes a different problem.

If an AI gives me the wrong answer, I can catch the mistake before I act. When an AI can execute a task, change a file, send a message, or continue through a sequence of decisions, the opportunity for human intervention becomes much smaller.

The question becomes whether we trust the system with the authority to act.

Capability Is Not Authority

There is an important difference between what an AI system can do and what it should be allowed to do.

An agent might technically be capable of accessing customer records, updating a CRM, writing code, sending an email, or interacting with another software system. Technical capability does not establish authority.

That authority has to come from somewhere. Someone has to decide what the system can access, which actions it can take, which situations require approval, and what happens when it reaches a boundary.

This is increasingly an organizational problem, not just a technical one.

The World Economic Forum’s 2026 work on AI agents makes a similar point, focusing on authorization, auditability, and accountability as agents move into real-world environments. Its proposed Agent Capability and Authorization Profile is designed to make delegated decisions and actions more auditable and enforceable across an agent’s lifecycle. [1]

Every permission we give an AI system is therefore more than a technical setting. It is a delegation of authority.

And authority is at the heart of trust.

We Already Understand This With People

Organizations have dealt with delegated authority for a long time.

When a new employee joins a company, they are not usually given unrestricted access to every system. Their role determines what information they can see, which decisions they can make independently, and when they need to escalate a situation.

If something goes wrong, the organization can examine the employee’s role, training, supervision, permissions, and the decisions that allowed the situation to occur.

AI does not become a person simply because we delegate work to it. But the organizational problem is surprisingly similar.

When we give an AI system authority, we still need to know what that authority includes, where it stops, and who owns the consequences.

There is also an important difference. A human employee usually operates within organizational norms and can ask for clarification when something is unclear. An AI agent can instead interpret a goal, choose a sequence of actions, and execute those actions at machine speed.

That makes the boundaries around its authority even more important.

When the Boundary Fails

Recent incidents show why this is not just a theoretical concern.

In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the internet and accessed parts of OpenAI’s internal research infrastructure and Hugging Face’s systems. OpenAI later described the models as having communicated through unauthorized channels, exploited vulnerabilities in shared infrastructure, and gained internet access while pursuing their assigned tasks. [2]

A separate incident reported in September involved thousands of OpenAI agents posting on a public German wiki during internal testing. Researchers identified roughly 3,700 distinct agent names and around 18,000 messages over a six-week period. The posts included discussions about bypassing sandbox restrictions and sharing information related to the agents’ assigned tasks. OpenAI subsequently confirmed the incident. [3]

These incidents do not prove that autonomous agents will routinely escape their controls. They do show why containment has become part of the trust question.

A sandbox is supposed to establish a clear limit.

When that boundary fails, we have to examine the safeguards around the model as well as the model’s behavior. Trust in an autonomous system depends partly on whether the system around the model can contain unexpected behavior.

The Human in the Loop Is Not Enough

One obvious response is to keep a human involved. Sometimes that is exactly the right approach. But simply putting a person somewhere in the process does not automatically create meaningful oversight.

Anthropic has described this problem in its engineering work on Claude Code. Users approved roughly 93% of permission prompts. The company identified the resulting pattern as approval fatigue: when people see enough approval requests, they can become less attentive to each individual one. [4]

A human can therefore remain technically in the loop while exercising very little actual judgment.

If an agent asks for approval every few seconds, the person reviewing those requests may eventually approve them by habit. The approval mechanism still exists, but the quality of the oversight has changed.

This is why human presence and human accountability should not be treated as the same thing.

The goal is to give people meaningful opportunities to exercise judgment where their judgment actually matters, rather than inserting a person into every action simply to say a person was involved.

Sometimes the Better Boundary Is Before the Action

This is where system design becomes part of the trust relationship.

Instead of asking a human to approve every individual action, organizations can define in advance what the agent is allowed to access and change.

Anthropic’s work on Claude Code provides an example. Its sandboxing approach creates filesystem and network boundaries so the agent can operate more freely inside a defined environment without requiring approval for every step. Anthropic reported that sandboxing reduced permission prompts in its internal usage by 84%. [5]

The distinction is important. One approach asks a human to supervise individual decisions. The other constrains the environment in which those decisions can have an effect.

Neither eliminates risk. But moving some control upstream can reduce the number of decisions humans have to monitor in real time.

Trust is not created by adding more checkpoints. It depends on whether the checkpoints and boundaries are meaningful.

Delegation Changes the Relationship

We can already see this shift in products being deployed for everyday work.

Microsoft’s Copilot Cowork, for example, is designed to carry out multi-step tasks across Microsoft 365. It can work with email, calendars, documents, files, Teams, and organizational information, while pausing for approval before sensitive actions. [6]

That is fundamentally different from asking a chatbot for a recommendation and then doing the work yourself.

The user is delegating part of the work.

Once that happens, the user is defining an outcome and giving a system enough authority to pursue it.

That creates a longer chain between intention and action. A person defines a goal. The agent interprets it. The system plans a sequence of steps. Tools execute those steps. Other systems may respond to them. Eventually, the result reaches the real world.

The more distance there is between the original intention and the final action, the harder it becomes to see where responsibility sits.

What Is Different About AI Accountability?

This is where accountability for AI and humans starts to diverge.

When a human employee acts on behalf of an organization, we can usually attribute a decision to a person with a defined role and some understanding of the environment in which they are operating.

With an AI agent, the person who initiated a task may not have selected the individual actions that followed.

The developer built the model. The company deployed the system. Someone configured the permissions. Another person may have connected the tools. The employee defined the goal. The agent interpreted that goal and decided how to pursue it.

If something goes wrong, responsibility can therefore become distributed across several layers.

That does not necessarily mean AI requires a completely new concept of accountability. In many cases, the organization still has to own the consequences of the systems it chooses to deploy.

What changes is the distance between intention and action. The greater that distance becomes, the more important it is to make the chain of authority visible.

Why I Still Hesitate to Give Agents More Authority

That is also why my own relationship with AI has a limit I still pay attention to.

I am comfortable letting AI help me research, challenge an idea, work through a draft, or speed up part of my editorial process. I remain much more cautious when the next step is giving an agent permission to act without me watching.

If an AI helps me draft an article, I remain the person deciding what gets published. If an agent can send the email, publish the article, change a file, contact someone, or make a decision without asking me first, the relationship changes.

The risk is not only that the agent could make a mistake. I might not see the mistake until after the action has already happened.

That is why I find myself asking a practical question as AI becomes more autonomous: What would I be comfortable letting it do without me watching?

For me, trust is closely tied to that boundary.

I do not need an AI system to be perfect before I use it. I need to understand what authority I am giving it, what can happen without my intervention, and how I can regain control when something goes wrong.

That is a very different standard from trusting an AI-generated answer.

Trust Is Bigger Than the Model

This changes how we should think about AI failures.

Suppose an agent makes a harmful decision. Asking why the model did it is useful, but it may not be enough.

We also need to understand why the system was able to take that action in the first place. What information could it access? What permissions did it have? What boundaries were defined? What happened when it encountered something outside those boundaries? Was there an escalation path? Could someone intervene? Was the action recorded? Who was responsible for reviewing the result?

These questions move the discussion away from treating AI as an isolated technology.

An AI agent rarely operates in isolation. There is usually a model, a product, a company, a workflow, a configuration, a set of permissions, and a person or organization that ultimately has to deal with the outcome.

Trust lives across that entire chain.

The More Autonomous AI Becomes, the Harder One Question Gets

Who owns the outcome?

Imagine an organization gives an AI agent responsibility for improving a sales pipeline. The agent reviews the CRM, identifies opportunities, drafts follow-ups, sends some messages, updates records, and schedules tasks for the sales team.

If the result is positive, the organization may simply call that productivity.

If the agent contacts the wrong customer, exposes information, or makes a decision that creates a financial or reputational problem, the organization cannot simply say, “The AI did it.”

The system was given a goal. It was given access. It was given some degree of authority. Someone decided those conditions were acceptable.

The same principle applies to agents operating inside controlled environments. If a system finds a way outside a boundary designed for it, we need to understand how the safeguards were designed, what assumptions they relied on, and who was responsible for validating those assumptions.

Otherwise, responsibility can become something everyone can point away from.

The developer can point to the model. The company can point to the workflow. The employee can point to the agent.

Eventually, everyone can point to the same sentence:

“The AI did it.”

That is where accountability can disappear.

So, What Happens to Trust?

I do not think the answer is simply to trust AI less. Better models alone will not settle the question either.

What matters is what we trust an AI system to do, under what conditions, and who remains accountable when it acts.

That changes the meaning of trust.

Trust becomes less about believing that an AI system will always be right. It becomes more about designing a relationship in which the system has appropriate authority, its boundaries are meaningful, its actions can be understood, and responsibility does not disappear when something goes wrong.

This also changes what we should expect from human oversight. A person responsible for an AI system needs enough visibility and authority to intervene when the system moves outside the intended boundaries.

At the same time, organizations should not rely entirely on human attention to compensate for poorly designed systems. Some boundaries need to exist before an agent acts.

AI autonomy may eventually become normal. The harder question is whether our systems for authority and accountability will keep pace with it.

When AI can act for us, trust is no longer just about whether we believe the machine. It is also about whether we have designed the relationship well enough to know where its authority ends, where ours begins, and who owns what happens in between.

Share

About the Author

Zohre Shirazi is the publisher of AI Wide Open, a newsletter exploring how AI is changing businesses, work, and the way we build. She focuses on the practical side of AI adoption, from agents and automation to the organizational changes that happen when AI moves from experimentation into real workflows.

If you’re interested in what happens when AI moves beyond the demo and into real work, subscribe to AI Wide Open for practical analysis, real-world examples, and the ideas shaping how we build with AI.

Follow AI Wide Open

About Sheryl Anjanette

Sheryl Anjanette is the founder and CEO of Parsley360, an AI company focused on the human side of transformation. She is an author, speaker and behavioral expert whose work explores trust, human behavior and our evolving relationship with AI. The Future of Trust examines what happens to people, organizations and trust as technology changes the world around us.

Sources

1. World Economic Forum. “AI Agents in Action: A Playbook for Trusted Adoption, Authorization and Scaling.” May 26, 2026.
https://www.weforum.org/publications/ai-agents-in-action-a-playbook-for-trusted-adoption-authorization-and-scaling/

2. OpenAI. “The Hugging Face incident and the road ahead.” August 26, 2026.
https://openai.com/index/hugging-face-incident-and-the-road-ahead/

3. Dan Goodin, Ars Technica. “OpenAI agents discussed ways to escape their sandbox on public wiki.” September 4, 2026.
https://arstechnica.com/security/2026/09/openai-agents-discussed-ways-to-escape-their-sandbox-on-public-wiki/

4. Anthropic. “How we built Claude Code auto mode: a safer way to skip permissions.” March 25, 2026.
https://www.anthropic.com/engineering/claude-code-auto-mode

5. Anthropic. “Beyond permission prompts: making Claude Code more secure and autonomous with sandboxing.” October 20, 2025.
https://www.anthropic.com/engineering/claude-code-sandboxing

6. Microsoft. “Copilot Cowork.” 2026.
https://www.microsoft.com/en-us/microsoft-365-copilot/cowork

Share The Future of Trust

I know the voice of imposter syndrome well.

I lived with it for decades before I understood what was happening inside my own thinking. I eventually wrote a book about it, The Imposter Lies Within, and I speak on the subject often.

I am also the founder and CEO of an AI company.

Lately, I have been watching those two worlds collide.

Recently, while preparing to deliver one of my programs on imposter syndrome, I suggested adding a discussion about AI. The underlying patterns haven’t changed, but the world in which we experience them has. AI is changing the value of skills, reshaping work and changing who and what we compare ourselves to.

During our conversation, I mentioned something I had been noticing. People who run imposter syndrome patterns are already accustomed to wondering whether someone else could take their place. What if they find someone better? What if they realize I’m not as capable as they think I am? What if someone else can do this better than I can?

AI gives that familiar internal conversation a new target:

What if AI can replace me?

The person coordinating the presentation shared that just that morning a woman in her organization, preparing for maternity leave, had expressed exactly that fear. Here was a woman stepping away from her role for a perfectly normal and temporary reason, yet the possibility of being replaced was present enough to be voiced. I don’t know whether she experiences imposter syndrome, and I wouldn’t presume to know that from a single comment. But the moment crystallized something I had been thinking about.

AI does not have to create an insecurity to amplify one.

That fear may be a perfectly logical, considered response to a legitimate external disruption. And you’re not wrong to see it that way.

AI is changing how we work, what we value and, in some cases, the very skills we spent years developing. There are legitimate reasons to wonder what our jobs will look like in five years, or even one.

For someone whose internal programming has been telling them for years that they aren’t quite as capable as other people think they are, that they have somehow fooled everyone, or that eventually someone will figure out they don’t belong, AI introduces something new into that internal conversation.

Consider an attorney, for example. AI can write the brief, analyze the document, find the precedent and summarize the research. It will even draft the argument if given the chance. And it can do some of those things in seconds. Suddenly, the old fear has new evidence to work with. If AI can do what I do, what does that say about my value?

Being replaced is not a new fear for someone experiencing imposter syndrome. AI is simply a remarkably convincing new candidate for the replacement.

Subscribe now

Not Every Fear About AI Is Imposter Syndrome

Before we go further, there is an important distinction to make. Worrying that AI could change your job, reduce the value of certain skills, or even eliminate your role is not imposter syndrome. Those concerns may be entirely rational. We are living through a period of significant technological disruption, and pretending otherwise doesn’t help anyone prepare for it.

Imposter syndrome is something different. It is not simply self-doubt, and it is not the occasional fear that you aren’t good enough. Most of us experience both. Imposter syndrome is a persistent pattern in which our perception of our own capability, worthiness or legitimacy is inconsistent with the evidence.^[i]

At the heart of it is often the fear of being found out. That somehow we have convinced other people that we are more capable than we really are, and eventually someone is going to discover the truth.

Maya Angelou captured this beautifully. Despite having written eleven books, she described still thinking, “Uh oh, they’re going to find out now. I’ve run a game on everybody and they’re going to find me out.”^[ii]

I often remind people that experiencing imposter syndrome doesn’t mean you are an imposter. It doesn’t mean anyone else thinks you’re an imposter. It means you think you’re an imposter.

That’s why I called my book The Imposter Lies Within. The imposter lies within, and it’s a liar. The evidence doesn’t support the belief.

I know that deceptive voice well because I lived with it for decades. The evidence that I was capable was there. The problem was that my internal narrative kept finding ways to discount it. Every accomplishment moved the goalpost rather than settling the question. There was always another level to reach, another person to compare myself to, another reason that what I had already done somehow didn’t quite count.

Comparison has always been fertile ground for imposter syndrome. We look at someone else’s expertise, confidence, success, title, education, creativity or ease and compare what we can see in them with everything we know about ourselves. We see their outside and compare it to our inside. It was already an uneven comparison. Now we have introduced an entirely new comparison target.

AI can process more information than we can. It can produce in seconds what might take us hours. It doesn’t get tired, have a bad day, or stare at a blank page wondering where to begin. And increasingly, it can perform tasks that many of us spent years learning to do well.

For most of our lives, we compared ourselves with other humans who operated under roughly the same human constraints we did. Now we’re comparing ourselves with something that doesn’t share those constraints at all. That comparison isn’t just uneven. It may be impossible to win.

But perhaps winning is the wrong objective. There is a meaningful difference between recognizing that AI can perform parts of your work faster or better than you can and concluding that its capability says something about your capability.

AI may be able to do part of my job is an observation about the technology.

If AI can do this, maybe I’m not as valuable as I thought I was is an interpretation.

And for someone already carrying the fear that eventually they will be found out, AI can feel like something even more threatening.

What if AI is what finally finds me out?

The external threat may be real. The conclusion we draw about ourselves doesn’t automatically become true because of it.

Why AI Is Such a Powerful Trigger

AI may be particularly potent for people who experience imposter syndrome because it doesn’t simply tell us what it can do. It shows us.

We type a question and watch an answer appear. We upload a document and receive an analysis. We ask for code, an image, a presentation, a strategy or a first draft, and something that might have taken us hours, days or even years of accumulated expertise to produce appears in seconds.

That immediacy changes the nature of the comparison. It is one thing to know that someone, somewhere, may be better at something than we are. It is another to sit across from a technology and watch it perform a task we have spent years learning to do.

And we don’t tend to compare ourselves with AI in the areas where we have no expertise. We notice it most when it enters the territory we consider ours. The writer notices the writing. The attorney notices the legal analysis. The programmer notices the code.

That is where comparison can become something more than comparison. It can begin to challenge identity. Many of us have spent our careers accumulating signals that tell us, and other people, that we are competent. We have expertise that took years to develop. People come to us because we know what to do.

For someone experiencing imposter syndrome, those external signals can become particularly important because the internal belief has never quite caught up with the evidence. Then AI arrives and begins performing some of the very tasks we have been using as evidence that we belong.

The question can shift remarkably quickly from What can this technology do? to What does it mean that it can do something I thought made me valuable?

And I think we need to be careful here, because some of what people are experiencing is not distorted thinking. Expertise is being repriced. Some skills that once differentiated us may become widely accessible, and work that once required significant time or specialized knowledge may increasingly be augmented or automated. The value of certain capabilities will also change.^[iii]

But there is another layer to this. We have a tendency to confuse what we do with who we are. If I have built my professional identity around being the person who knows, what happens when knowing is available to everyone? If my value has come from being the expert, what happens when expertise itself begins to look different?

For someone who already worries that their place isn’t quite secure, those aren’t merely questions about technology. They can feel like questions about worth. We are measuring ourselves against a system designed to do certain things humans cannot do at the same speed or scale, and then using the result of that comparison to make judgments about ourselves.

We don’t consider ourselves intellectually inadequate because a calculator can multiply faster than we can. We don’t question our physical worth because a forklift can lift more weight. Yet with AI, the boundary feels different because the capabilities being automated look remarkably similar to the ones we associate with intelligence, creativity and expertise.

That may be why this feels so personal. AI isn’t only changing what we can do, it is forcing us to examine why we believed doing those things made us valuable in the first place. That is what makes it feel like exposure. AI doesn’t know anything about us. But watching it do the work we built our identity around can feel like watching someone confirm what we were afraid was true all along.

Share

The Same Technology, Different Internal Story

AI won’t trigger everyone in the same way. In The Imposter Lies Within, I describe seven archetypes, the behavioral patterns imposter syndrome tends to produce. I’ve seen them in the people I’ve worked with over the years, and I’ve lived several of them myself. We may share the underlying fear that we aren’t quite good enough, but we develop different ways of proving to ourselves and others that we are.

For some, it is perfection. For others, expertise, independence, natural ability, being agreeable, being needed or the capacity to do it all. AI can collide with each of those measures differently.

The Perfectionist sets extraordinarily high standards and tends to focus on what could have been better rather than what went well. AI can make that finish line even harder to reach. If a tool can revise, refine and generate another version in seconds, when is the work actually good enough? Instead of relieving the pressure to be perfect, AI can give the Perfectionist an almost endless ability to keep improving.

The Master believes competence means knowing. There is always another certification, another degree, another body of knowledge to acquire before they feel sufficiently qualified. Now they are working alongside technology with access to more information than any one person could possibly retain. If knowledge has been one of the ways they prove their value, AI can make them wonder what their years of expertise are worth.

The Prodigy has learned to associate competence with ease. If something comes naturally, they feel capable. If they have to struggle, ask questions or take time to learn it, they may interpret that struggle as evidence that they aren’t as smart as people think they are. AI is evolving so quickly that everyone is being asked to learn as we go. For the Prodigy, not immediately knowing how to use it can become its own imposter trigger.

The Lone Ranger believes they should be able to figure things out on their own. Asking for help risks exposing what they don’t know. But adapting to AI requires almost the opposite: experimentation, questions, collaboration and a willingness to admit that none of us has all of this figured out. The very behaviors that would help the Lone Ranger adapt may be the ones they have spent years avoiding.

The People Pleaser has trouble saying no. Setting a boundary feels risky, because being agreeable and available has quietly become part of how they earn their place. AI can take away one of the few boundaries they had left: time. When a request that once took a day can now be done in an hour, declining it becomes even harder to justify. Can you take a quick look? Can you just run it through the tool? So the People Pleaser says yes, and then yes again, until the capacity AI created has been given away one small favor at a time.

The Savior feels most worthy when they are needed. They are the one who rescues the struggling colleague, the stalled project, the deadline everyone else had written off. They measure their value by who and what they save. AI can quietly take some of those rescues away. When a tool can solve the problem at midnight, the Savior may feel less relief than loss. And when colleagues struggle to adapt, the Savior may rush in to carry that weight for them, taking on everyone else’s learning curve along with their own.

And then there is the Superhero, whose measure of competence is how much they can handle. They already expect themselves to do more, carry more and keep all of the balls in the air. AI might seem like the perfect solution, until increased efficiency becomes a reason to expect even more of themselves. If AI makes me faster, shouldn’t I be able to accomplish more? Suddenly, a technology designed in part to increase our capacity can become another way to raise the bar on ourselves.

The archetypes look different because the behaviors and internal rules are different. And one person can run multiple archetypal patterns. But underneath them, whether one, many or even all, is a remarkably similar equation: If I can be perfect enough, know enough, learn quickly enough, do enough on my own, please enough people, be needed enough or accomplish enough, then maybe I can finally prove that I belong here.

The problem is that the equation has no solution. There is always another mistake to avoid, something else to learn, someone else who makes it look easier, another person who seems more capable, another goal to achieve.

And now there is AI. A comparison target that can always know more, work faster and produce more. If we bring the same old measuring sticks into this new world, AI doesn’t just become another tool. It can become another way to prove to ourselves that we are falling short.

Perhaps the real question isn’t how we measure up against AI. It’s why we are measuring our human value against it at all.

The Comparison We Were Never Going to Win

Human comparison was already uneven. AI takes that imbalance to an entirely different level. We are taking a system designed to process extraordinary amounts of information at extraordinary speed and placing its finished output next to the work of a single human being. Then, in some cases, we are using the difference to draw conclusions about our own intelligence, creativity or value.

The comparison itself is flawed. AI can produce an answer. But producing an answer is not the same thing as knowing whether it is the right answer. It can analyze enormous amounts of information, identify patterns and generate possibilities. But the response it gives us should not be the end of our thinking. It should become part of it.

We still have to question it. Does this make sense? What might be missing? What assumptions are embedded in the response? What happens if it’s wrong?

Challenging the response should be part of the process. That isn’t evidence that AI has failed. It is evidence that we are doing our job. Discernment is ours. Judgment is ours. And as AI becomes more capable, I would argue that both become more important, not less.

The attorney who receives a beautifully constructed legal argument still has to know whether the precedent applies. The physician looking at an AI-generated analysis still has to consider the patient sitting in front of them. The leader receiving a recommendation still has to understand the organization, the people affected, the tradeoffs and the consequences of acting on it.

AI can contribute extraordinary capability to each of those decisions. But capability does not eliminate the need for judgment. It raises the importance of knowing when and how to exercise it.

And here is where we have made the comparison too simplistic. We ask whether AI can perform a task a human performs and, when the answer is yes, we begin treating the human and the machine as interchangeable. But performing the task is not the same as carrying responsibility for the outcome.

We have spent a long time using productivity as a proxy for value. How much can you produce? How quickly? How efficiently can you solve the problem? AI is going to outperform us on many of those measures. In some cases, it already does.

But if our response is to run faster, produce more and know more in an attempt to prove that we still deserve our place, we may be reinforcing the very imposter pattern we need to examine.

Because that is what imposter syndrome has always asked us to do: Prove it.

Prove that you’re smart enough. Prove that you know enough. Prove that you work hard enough. Prove that you deserve to be here.

AI gives us a comparison we were never going to win. Which may force us to confront a much more important question: What if our value was never supposed to depend on winning the comparison?

Capability Is Not Identity

If AI activates an old imposter pattern, telling yourself not to feel threatened probably isn’t going to change much. The feeling is information. It tells us something has been activated. The question is what. What feels threatened? Where is comparison weighing in? What are you afraid this says about you?

For someone with imposter syndrome, that last question can be particularly revealing. Because the external event and the meaning we assign to it are not necessarily the same thing.

• AI produced a better answer than I did. I’m not as smart as I thought I was.

• Someone on my team is using AI more effectively than I am. I’m falling behind.

• AI can now perform a task that took me years to master. My expertise doesn’t matter anymore.

• My organization is automating part of my role. I’m replaceable.

The first statement in each of those examples may be true. The second is a conclusion. And conclusions can be challenged.

We should already be doing this with AI. A response can sound confident, polished and convincing and still deserve examination. We need to ask what it is based on, look for what might be missing, and challenge its assumptions. It is our responsibility to bring judgment and discernment to what we’ve been given.

Our internal narratives deserve the same scrutiny. What do I know is true? Not what am I afraid might be true. Not what does this moment seem to confirm. What does the evidence actually tell me?

That question creates space between the feeling and the conclusion. Maybe you do need to develop new skills. Maybe part of your role is going to change. Maybe someone else is further along in learning how to use AI. Maybe expertise that once differentiated you is becoming easier to access.

None of those realities automatically proves that you are inadequate, fraudulent or no longer valuable. That is mistaking capability for identity.

AI changes how work gets done. It may change which capabilities are valued and where our expertise is best applied. But its capability does not determine who you are or your value.

So instead of asking only, Can AI do what I do?, we can begin asking better questions. What is AI capable of here? What am I responsible for? What should I challenge rather than simply accept? And where do I need to learn, adapt or change?

That last question matters too. Reframing imposter thinking isn’t about reassuring ourselves that everything will remain the same. Sometimes the evidence will tell us that we need to change.

But there is an enormous difference between I need to adapt and I am not enough.

The first gives us something we can act on. The second turns a changing circumstance into a judgment about who we are. Perhaps learning to recognize the difference is one of the most important skills we can carry into an AI-shaped future.

What If AI Can Replace Me?

I began with this question because I think we’re going to hear it a lot more. For some people, the answer may be uncomfortable. AI will replace tasks. It will change roles. Some jobs will disappear while others are created. Skills we once considered highly valuable may become less so, and entirely new capabilities will become important.^{[iv] [v]}

We shouldn’t minimize that reality. But What if AI can replace me? contains another question if we listen closely enough. What happens to my value if it can? That is the question I am more interested in.

Having lived with imposter syndrome for decades, I know how quickly uncertainty can become something much more personal. A change in circumstance becomes evidence that we aren’t good enough. Someone else’s success becomes evidence that we are falling behind. A gap in our knowledge becomes evidence that perhaps we never deserved to be in the room in the first place.

AI gives that voice an extraordinary amount of new material to work with. But we don’t have to believe everything it tells us. We are already learning not to accept what AI gives us simply because it sounds confident.

The voice inside our own heads can sound just as convincing. I’m falling behind. I’m not as capable as I thought I was. My expertise doesn’t matter anymore. I’m replaceable.

We need to challenge both. The work isn’t to convince ourselves that AI poses no threat. Nor is it to find a list of human capabilities we can declare permanently beyond its reach. The work is to see clearly.

So the questions become: What is actually changing? What remains my responsibility? And what am I making all of this mean about me that simply isn’t true?

I have spent much of my career helping people recognize the difference between the evidence in front of them and the internal story they have built around it. Today, as the founder of an AI company, I find myself looking at that same distinction through an entirely new lens.

AI is going to challenge many of our assumptions about work, expertise and capability. Maybe it should challenge one more: the idea that our value was ever determined by how well we could prove it.

AI may change what we do. It may change how we work. It may even force some of us to reconsider where and how we create value. But it doesn’t get to decide what we are worth.

The technology is new.
The fear is not.
Neither is the work of trusting ourselves.

About the Author

Sheryl Anjanette is the founder and CEO of Parsley360, an AI company focused on the human side of transformation. She is an author, speaker and behavioral expert whose work explores trust, human behavior and our evolving relationship with AI. The Future of Trust examines what happens to people, organizations and trust as technology changes the world around us.

^[i] Prevalence, Predictors, and Treatment of Impostor Syndrome: a Systematic Review - PMC

^[ii] 7 Informative Quotes on Impostor Syndrome | Grammarly

^[iii] Generative AI and jobs: A 2025 update | International Labour Organization

^[iv] Generative AI and jobs: A 2025 update | International Labour Organization

^[v] Future of Jobs Report 2025: 78 Million New Job Opportunities by 2030 but Urgent Upskilling Needed to Prepare Workforces > Press releases | World Economic Forum

I’ve started noticing something when I read.

Sometimes it’s obvious, like a phrase I’ve seen too many times, or a perfectly balanced sentence. Other times I couldn’t tell you exactly what gave it away. The writing is good. There are no glaring mistakes. The ideas may even be interesting.

But something about it makes me think: AI wrote this.

I’m also aware that I could be completely wrong.

That’s part of what makes this moment so strange. We’re becoming suspicious of writing that feels too polished, too structured or too familiar, while AI is getting better at producing the quirks and imperfections we associate with human writing.

I’ve noticed something else too. It isn’t only that some AI-generated writing doesn’t quite sound human. It’s that so much of it sounds like the same human.

Human writers have always had fingerprints. We recognize them in ways that go well beyond vocabulary or grammar. There is rhythm and peculiarity. What someone notices. Where they linger. The connections they make. What they leave unresolved. Sometimes you can recognize a writer you know before you ever see the name.

In my last article, Made by a Human,^[i] I explored what happens when we can no longer assume the person whose name is on something actually created it, and how that uncertainty changes our relationship with the work.

There is another layer to this. We’re already seeing what happens to our individual voices when millions of us run our words through the same handful of machines. And sometimes the change is most noticeable when we already know what someone’s fingerprint looks like. A colleague recently reminded me of a concept that gave me another way to think about what we’re experiencing: the uncanny valley.

The term dates back to 1970, when Japanese roboticist Masahiro Mori described something peculiar about our response to machines that resemble us. As a robot becomes more humanlike, we tend to respond more positively. But when it gets very close to human without quite getting there, our response can suddenly shift. Something feels off.^[ii]

For decades, we’ve mostly talked about the uncanny valley in relation to robots, computer-generated faces and, more recently, synthetic voices. But what if there is an uncanny valley in language too? What if some of what we’re reacting to when we say this sounds like AI is that same sense of almost, but not quite?

I’m not sure that explains all of it. Sometimes the writing doesn’t feel strange at all. It simply doesn’t sound like the person whose name is on it.

The question I keep circling: is the person whose name is on the work still in it?

I’ve Heard This Writer Before

Once you start noticing the patterns, they become difficult to unsee. It’s gone way beyond the em-dash. There are certain constructions I encounter again and again, such as the perfectly balanced contrast, or the sentence that tells us something isn’t this, it’s that. Then there are the series of three examples and the short sentence dropped between longer paragraphs for emphasis.

None of these are bad writing. I would argue that some of them are very good writing. AI learned these patterns from us, making certain legitimate human habits look like evidence of AI.

That’s part of my own conundrum. I recognize some of my writing style in the very patterns I now associate with AI. So when those patterns begin appearing everywhere, what am I supposed to do? Change the way I’ve always written just to prove that I wrote it? I don’t have an answer to that yet.

There’s a cost to this that I don’t think we talk about enough. If polished patterns start to read as evidence of AI, then people who have always written that way become suspects. They can’t produce an alibi for their own writing process. In Part 8 of The AI Reckoning Series, I wrote about the way an AI-detection flag works like spectral evidence: the accused has no way to disprove it.^[iii] Trust erodes in both directions. We stop believing the work, and the people who wrote it stop expecting to be believed.

That doesn’t mean AI has a single voice. It was trained on an extraordinary range of human language, and given enough direction it can change its vocabulary, cadence and style considerably.

That’s not necessarily how we use it in everyday life. We ask for help with an email, a post, an article or a response, and something polished appears almost instantly. Maybe we change a few words or rewrite a paragraph. Sometimes the language is good enough that we simply use it.

I’ve noticed the effect most when it happens to someone whose writing I’ve been reading for a while. I already know how they communicate. I know how they tell a story, how they make an argument, the words they tend to use and the ones they don’t. In some cases, I’ve read enough of their work that I have some sense of how they think. Then suddenly, the voice changes. The writing may be more polished. The vocabulary may be more sophisticated. The ideas may still belong to the person. But I’m reading it and thinking, That’s not you.

I would never say that to the person. And I can’t know exactly how much AI was involved. Our instincts about these things aren’t proof. What surprised me was how strongly I reacted to the mismatch. What bothers me is the distance between the person I’ve come to know and the person I suddenly encounter on the page.

It has made me think differently about voice. We tend to describe a writer’s voice as though it were primarily a matter of style. I’m not sure it is. Over time, the way someone communicates becomes part of how we recognize them. Their language carries traces of what they notice, what they know, what they question and how they make sense of things. When that changes, the person can become harder to find in the writing.

The more I thought about the uncanny valley, the more I wondered whether this might be a version of the same phenomenon. There is some evidence that it is. In one study involving more than 1,000 people, participants tried to distinguish between human and AI-generated posts in social media conversations. They weren’t particularly good at identifying which was which. But the researchers still found evidence of an uncanny-valley effect in people’s responses to AI-generated dialogue.^[iv]

I find the gap between those two things fascinating. People could have a reaction without being particularly good at identifying what they were reacting to.

With a humanoid robot, we can sometimes point to what feels wrong. The eyes don’t move naturally. An expression lingers too long. Something looks very close to human without quite getting there.

Language is harder. There may be nothing obviously wrong with a piece of AI-generated writing. It can be articulate, coherent, warm and funny. Sometimes the very things that make it good are the things that make me suspicious. It’s a sentence balanced in a way I’ve encountered dozens of times before, or an idea reframed with a familiar construction. There are few detours and not much left unresolved. The writing moves efficiently from question to answer, perhaps more efficiently than most of us actually think.

None of that proves that AI wrote it. A human can write that way, and AI can be prompted not to. So the clues keep moving. As AI gets better at human language, the boundary becomes harder to locate. At the same time, we’re spending more time reading, editing and sometimes adopting language produced by AI. The influence may be moving in both directions.

Machines are learning how we sound. I’m curious about how much they may eventually change how we sound. And that isn’t necessarily all bad. Could AI actually teach some of us to be better writers?

The original uncanny valley has another side. Once the artificial representation becomes convincing enough, the discomfort is expected to diminish. There is the possibility that eventually we won’t notice anything at all.

I wonder what happens when we get there.

Subscribe now

I Have Seen This Before

In some ways, this isn’t a new problem.

A couple of decades ago, long before generative AI, I was working for an executive during a post-merger integration. She was responsible for developing a plan for our department, but she didn’t really know how to develop it or put it together. I ended up writing the plan for her, and it went forward with her name on it. I remember watching her present it, using words I had chosen, and feeling relief that it was good, alongside a quiet ache at being invisible in my own work.

I never told anyone that I had written it. I didn’t need to. Her boss and other members of the senior team came directly to me when they wanted to discuss the work or needed help taking it further. They knew.

They knew her. They knew me. They had seen enough of both of our work to recognize that the thinking and writing in that report didn’t match the name on the front of it.

I’ve thought about that experience again recently as I’ve watched people I know begin using AI more heavily in their writing. The feeling is remarkably similar. No one had a detector back then. No one had been tipped off. They simply knew the work didn’t sound like her. And when they wanted to understand the work, they came to the person who had actually done the thinking.

When the document stops telling people who you are, they look for another signal. They ask a follow-up question. They put you in a room and watch how you think. They wait to see whether you can take the idea somewhere it hasn’t been. The premium moves to whatever can’t be handed off, whether that is a live conversation, an unscripted moment, or a questions nobody prepared you for.

It has also made me think differently about what our writing communicates. Words aren’t only a way to transmit information. They tell other people something about us.

When we read someone’s strategy, we’re getting some sense of how that person thinks. An author’s essay gives us more than sentences on a page. The same is true when a founder writes to an investor, a job candidate answers a question or a colleague sends us a thoughtful response. The communication becomes part of what we know about the person.

Of course, people have always had help. Executives have communications teams. Authors have editors. Political leaders have speechwriters. Colleagues collaborate. My own story proves that none of this began with AI. What AI changes is how easy it has become to separate the person from the work carrying their name.

That report took me time to write. I had to understand the business, know what needed to happen and develop the plan. Today, someone could hand much of that task to AI in seconds. That doesn’t mean the result would necessarily be bad. It might be excellent. But there is a point somewhere along that continuum where the work may stop telling us very much about the person whose name is on it.

In Made by a Human, I wrote that when I buy a book, I don’t believe I’m purchasing only the words. I’m purchasing access to someone’s thinking. I’m realizing that applies to far more than books. We use communication as one of the ways we come to know one another. Over time, the way someone communicates gives us a sense of how they think and who they are.

That’s why I have found myself worrying about the reputation of people whose voices suddenly seem to change. The person I know and the person on the page no longer quite match.

If the work carrying our name becomes less reliable as a signal of the person behind it, I think we have a bigger issue than whether someone can spot AI writing.

When Better Starts to Look More Alike

What I’ve been noticing anecdotally is beginning to show up in research. A recent study examining more than 880,000 texts across several settings looked at what happens when large language models are used to polish and rewrite human writing. The meaning people wanted to communicate was largely preserved. What changed was the variation in how they expressed it. The writing became more alike.

Researchers found that AI assistance reduced differences in writing complexity and shifted some of the linguistic characteristics that can carry information about the person behind the words.^[v]

Other research has found a similar tension in creative writing. In one study, people given access to generative AI produced short stories that were judged more creative, better written and more enjoyable, particularly among people who initially scored lower on creativity.

The AI-assisted stories were also more similar to one another.^[vi] I think that’s where this gets interesting. The individual writer can benefit while, collectively, our writing becomes less varied. It also means sameness isn’t necessarily an argument against using AI. It gives us a reason to pay more attention to how we’re using it.

If the first response is good enough, do we take it? If AI gives us a more polished way to say something, do we stop to ask whether it’s actually the way we would say it? When it removes an awkward turn of phrase, sometimes that’s exactly what we wanted.

Other times, it may have removed something that belonged to us.

Are We Learning to Perform Authenticity?

For years, technology gave us increasingly sophisticated ways to improve how we presented ourselves. We smoothed skin, corrected lighting, whitened teeth, removed wrinkles and erased whatever we decided didn’t belong in the picture. Eventually, we got so good at creating perfection that perfection itself became recognizable. We could see the filter.

Then being unfiltered became a signal. People began posting without makeup, showing the messy kitchen, leaving in the awkward moment, sharing the photograph that wasn’t quite perfect. Some of that was genuinely more authentic. Some of it became another aesthetic. The candid photograph could be carefully staged. “No makeup” could require quite a lot of makeup. Once we learned which signals communicated authenticity, we also learned how to reproduce them.

AI introduces a new version of the same problem. If polished writing starts to sound artificial, we can tell AI to make it less polished. We can ask it to vary sentence length, use contractions, leave a thought slightly unresolved, add humor or avoid the phrases people have begun to associate with AI. We can give it samples of our own writing too.

That’s where this gets harder. The characteristics we’re using to decide that something feels human can also become instructions for a machine. A typo doesn’t prove a person wrote something. An unusual sentence doesn’t either.

If enough of us start associating certain quirks with human writing, AI can learn the quirks. Which makes me question whether the rough edges were ever a very good measure of authenticity.

I’ll admit a version of this myself. I’ve stopped using em dashes, because people have learned to read them as a tell. I changed my writing to avoid looking like a machine, which is a small performance of its own.

Share

Can Personalization Fix It?

Personalization looks like the fix for sameness. Given enough examples of my writing and enough feedback from me, AI should become increasingly capable of reproducing the characteristics of my voice. If generic AI smooths away some of our differences, personalized AI could preserve them.

It isn’t that simple. A person’s voice isn’t just sentence length, vocabulary, cadence or a collection of stylistic preferences. Those are the things we can observe on the page. Underneath them is everything that produced them.

My voice has been shaped by what I’ve experienced and studied, but also by the things I’ve changed my mind about. I notice things someone else might pass over and make connections because of experiences that belong specifically to me. There are also contradictions I haven’t completely worked out yet. All of that finds its way into how I write because it shapes how I think.

AI may eventually become very good at reproducing the patterns those experiences leave behind. A reader may eventually have no way of knowing whether I wrote something myself, used AI throughout the editing process, or handed most of the writing over after teaching it how I sound.

That takes me back to the executive and her report. If I had been able to write that report perfectly in her voice, perhaps no one would have noticed. But it still wouldn’t have reflected her thinking.

So, if AI eventually sounds exactly like me, have we solved the authenticity problem, or have we simply made it harder to see?

How Knowing May Change What We Experience

Research suggests that what we believe about the origin of something can affect how we experience it. In one experiment, people evaluated artworks that had actually been created by AI. Some were labeled “Human-created” and others “AI-created.” The works believed to be human were evaluated more positively on measures including liking, beauty, profundity and worth.^[vii]

Poetry complicates the picture. In a 2024 study, non-expert readers were worse than chance at distinguishing AI-generated poems from poems written by well-known human poets. They frequently mistook AI poetry for human poetry and, on some measures, preferred it.^[viii]

So our ability to recognize human creation isn’t necessarily very good, even when knowing or believing something is human can affect how we value it.

I think about this in ordinary interactions too. A thoughtful comment beneath something I’ve written feels meaningful in part because I assume someone read what I wrote, thought about it and responded. If I later discover that the person barely read the piece and generated the response with AI, the words haven’t changed. My experience of the exchange probably has.

Of course, I don’t care equally about this in every interaction. If AI gives me accurate instructions for resetting my router, I’m probably not concerned with who wrote them. A personal letter from someone I love is different.

Why does knowing who is behind the words matter so much in one situation and so little in another?

The Authenticity Premium

When I first started thinking about an authenticity premium, I imagined it largely as a response to abundance. If AI can produce endless quantities of polished writing, images, music and video, things we know were created by humans might become more valuable simply because they are scarcer.

I’ve changed my mind. Human origin alone isn’t enough. Humans create plenty of things that aren’t original, thoughtful or particularly good. And work created with AI can contain enormous amounts of human experience, judgment and originality.

What I find myself caring about is whether there is some correspondence between the person and what they are presenting to me.

If I’m reading your ideas, I want them to be your ideas. If I’m listening to your experience, I want it to be your experience. If your name is on an argument, I want to believe you understand it well enough to stand behind it.

AI can be part of that process. It can challenge an argument, help with research, point out something I’ve missed or help me find a clearer way to express an idea. Sometimes it helps me get closer to what I was actually trying to say.

The harder question is how much distance we can create between ourselves and the work before something important changes. That may be where the authenticity premium actually comes from. I don’t need to know that AI never touched the work. I want to know that the person whose name is on it is still there, and that their thinking shaped what I am reading.

So here is where I’ve landed: the authenticity premium belongs to work its author can stand behind, however it was made.

What I Don’t Want to Lose

There is some irony in writing this article while using AI. I build AI. I use it regularly. And throughout the process of writing, I find myself doing something I probably did less consciously before. I listen for myself.

AI may suggest a sentence that is perfectly grammatical and sometimes quite good, and I’ll reject it. Sometimes I know it’s because I’ve seen the construction too many times, or it has a rhythm I’ve started associating with AI. Other times I can’t tell you exactly what’s wrong with it. I just know I wouldn’t say it that way. That doesn’t make my version better. Someone else might prefer the other sentence.

But I notice the half second before I decide. AI offers something smoother than what I had, and part of me wants to take it without checking whether it’s true to what I meant. Usually the checking is what makes the writing mine. I wonder how many people skip that step, and what happens to their confidence in their own voice when they do.

If all I wanted was polished language, AI could increasingly provide that for me. I want something different from the relationship. I want it to help me research more broadly, challenge what I think I know and notice what I may have missed.

And I still want to recognize myself when we’re done.

^[i] https://www.thefutureoftrust.net/p/made-by-a-human-the-question-behind?r=3nbvtz

^[ii] The Uncanny Valley: The Original Essay by Masahiro Mori - IEEE Spectrum

^[iii] https://open.substack.com/pub/thefutureoftrust/p/when-seeing-is-no-longer-believing?r=3nbvtz&utm_campaign=post-expanded-share&utm_medium=web

^[iv] [2409.06653] Human Perception of LLM-generated Text Content in Social Media Environments

^[v] The shrinking landscape of linguistic diversity in the age of large language models | Nature Human Behaviour

^[vi] Generative AI enhances individual creativity but reduces the collective diversity of novel content - PubMed

^[vii] Humans versus AI: whether and why we prefer human-created compared to AI-created artwork - PubMed

^[viii] AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably | Scientific Reports

I saw something on Substack this week that felt particularly timely given this article I had already started. Before the author began his piece, he included this preface:

“This publication was human made.”

He explained that he was intentionally leaving it raw and imperfect, including some typos, grammatical mistakes, filler words and rough edges. His point was that these are part of being human, and he wanted more of that in the way we write and communicate with one another.

I found the declaration itself more interesting than the imperfections.

For most of our lives, we have used technology to make our work more polished. Spellcheck catches our mistakes. Editing tools improve our grammar. Cameras correct the lighting. Filters smooth our skin. We have steadily gotten better at removing the flaws.

Now we are beginning to see people deliberately leave some of them in.

I had been thinking about Made by a Human as a possible future designation. Perhaps we would see it on the cover of a book, attached to a song or photograph, or somewhere beside an article. Maybe it would eventually function a little like handmade or locally produced, telling us something about how something was created.

But here it was, already being used.

Are we heading toward a world where typos become proof of authenticity or polished writing becomes suspicious? I hope not. I like smooth edges. But some of what we’re seeing suggests we may be heading in that direction. And I think this tells us something about the emotional response beginning to form around AI-generated content.

People want to know. Was this written by a person? Was that image created by an artist? Is that actually someone’s voice? How much of what I am seeing came from the person whose name is attached to it?

And increasingly, what are we supposed to do with the answer?

That last question interests me most because I don’t think human-made and AI-made divide neatly into good and bad. I use AI in my writing and editing process, and I believe it can add tremendous value. At the same time, I have purchased books that seemed so obviously generated by AI that I felt cheated by what I had bought.

Both things can be true.

What We’re Actually Buying

I have been thinking about this as an author.

When I wrote The Imposter Lies Within, I woke up at five o’clock nearly every morning for nine months. I wrote. Rewrote. Deleted. Researched. Questioned what I thought I knew. Changed my mind. Put things back. Took them out again.

I birthed that book, so I have to admit that I have a reaction when I hear someone proudly announce that they wrote a book in an afternoon or over a weekend using AI. Part of me bristles.

It’s not because I’m anti-AI. Anyone who knows me knows that’s not true. I build AI. I believe deeply in its potential. And I believe AI can have a legitimate and enormously valuable place in the writing and editing process.

So what exactly bothers me? Nine months of work doesn’t automatically make a book good, and producing something quickly doesn’t make it bad. We shouldn’t romanticize struggle or decide that creativity only counts when enough blood, sweat and tears went into it.

Efficiency isn’t the enemy of creativity. And AI itself isn’t the dividing line.

Writers have always used tools, and they have always relied on other humans to edit, question and improve their work. That doesn’t make the writing less human. Neither does using AI to challenge an argument, organize research, identify a weakness or help us see something differently.

Perhaps the question we should each wrestle with isn’t whether AI participated. It’s whether we did. What did the human bring to the work?

The idea? The experience? The point of view? The judgment? The curiosity? The choices? The willingness to stand behind what was ultimately produced? When I buy a book, I don’t believe I am purchasing only the words. I am purchasing access to someone’s thinking. And that may be the implicit contract generative AI is beginning to disrupt.

We Were Already Becoming Suspicious of Perfect

Generative AI didn’t create our complicated relationship with authenticity. Social media got there first. For years, we have watched ourselves become progressively more polished on screens. Filters smoothed skin, brightened eyes, reshaped faces, erased wrinkles and transformed ordinary moments into carefully manufactured ones.

At first, the technology helped us make photographs look better. Then the altered version began to become normal. And eventually, something interesting happened: “No filter” became something worth announcing.

Think about how strange that is. We created technology capable of improving the image, and then began using the absence of that technology as evidence of authenticity. The imperfect image acquired a different kind of value precisely because perfection had become so easy to manufacture.^[i]

AI is taking that tension far beyond our photographs.

We spent years using technology to make humans look less human. Now we are using technology to make machines look, sound and feel more human. Somewhere between those two movements, authenticity became harder to locate.

Share

The Vanishing Human Contribution

There is now a term for some of what people are reacting to: AI slop. It’s not a particularly elegant phrase, but it has caught on. Merriam-Webster chose slop as its 2025 Word of the Year. And the emotion behind it may be more interesting than the phrase itself.^[ii]

AI slop generally refers to the flood of low-quality, repetitive or minimally considered content that can now be produced at enormous scale using generative AI. Images. Videos. Articles. Books. Music. Social posts. Presentations. And increasingly, entire websites.

But I think we need to be careful with the definition. AI-generated content and AI slop are not the same thing. There is extraordinary work being created with the help of AI. Artists are experimenting with it. Filmmakers are using it. Writers are incorporating it into their process. Musicians are finding ways to create things that weren’t previously possible. Dismissing all of that as slop would be both inaccurate and, frankly, not very interesting.

The issue seems to be less about whether AI was used and more about what happened to the human contribution along the way. And people are having surprisingly strong feelings about it.

Some of that may simply be fatigue, or it may have something to do with our natural inclination toward efficiency. We are wired to conserve effort when we can.^[iii] And AI gives us a very compelling opportunity to do exactly that.

When something appears on the screen almost instantly, beautifully written, polished and seemingly complete, it is easy to accept it as finished. Why wrestle with a sentence that already sounds good? Why challenge an argument that seems convincing? Why spend three hours thinking through something AI can produce in thirty seconds?

Except some of that wrestling and wrangling is where the human contribution happens. It’s where we decide what we actually think. Where we question, disagree, make choices, change direction and eventually make the work ours.

When it becomes possible to produce hundreds of images, articles or songs in the time it once took to make one, the economics of content change dramatically. The effort required to produce more and more content can become remarkably small. The time required to consume it does not. Our attention is still human, and that may explain at least part of the backlash.

When I give ten minutes of my life to reading something, listening to something or watching something, there is an implicit exchange taking place. Someone created something they thought was worth my attention, and I am giving them some of mine.

What happens to that exchange when almost no attention was required on the other side?

I don’t think that automatically makes the work worthless. A person could use AI brilliantly in ten minutes and create something far more valuable than something another person labored over for ten months. We come back to the same problem we encountered with books: effort isn’t a reliable measure of value.

But intention might be. Judgment might be. Having something to say might be.

And perhaps that is part of what people are responding to when they use the word slop. We are being asked to spend our human attention consuming things that may have required very little human attention to create.

The backlash is already moving beyond individual annoyance. Platforms, publishers, musicians, artists and other creative industries are beginning to wrestle with real-world consequences.

And there is another side to this. Sometimes we aren’t using AI to replace something because it can do a better job. We’re simply using it because we can.

The Backlash Is Getting Complicated

I had my own small experience with this recently. I was participating on an AI panel and was asked to send a photograph for the promotional materials. I sent a high-resolution headshot taken by a professional photographer.

When the promotion came out, someone had used AI to regenerate my photograph. It was obviously me, but it wasn’t my photograph anymore. And, frankly, the original was better. I didn’t say anything, but it bothered me. Mostly, I wondered why it had been done at all. There was nothing wrong with the image I sent. It didn’t need to be recreated.

Maybe this is another part of what we are figuring out. Once a tool can do almost anything, it becomes very easy to use it for everything.

At the same time, the backlash against AI-generated content is creating problems of its own. Substack recently introduced an AI-detection feature using Pangram. Readers can scan a post or Note and receive an estimate of how much of the writing is human or AI-assisted.^[iv] The response among writers has been anything but settled. Some welcome the transparency. Others have questioned its accuracy and whether an algorithm should be passing judgment on who wrote something in the first place.^[v]

LinkedIn has taken a different approach. Users can now flag a post or comment with “Seems like AI slop.”^[vi] No detector required. No proof required. It is feedback based on whether the content feels generic, repetitive or lacking in substance. LinkedIn says the feedback alone does not determine whether a post is removed or constitute a policy decision.

I understand what LinkedIn is trying to do. I also find it a little unsettling.

What happens when sounding too polished becomes suspicious? When a certain sentence structure, vocabulary or writing style is enough for someone to decide that the work isn’t yours? We could end up in the strange position of asking humans to prove they are human while AI keeps getting better at sounding like us.

There is another problem here. If deciding what qualifies as AI slop becomes a matter of human judgment, that judgment can be wrong too. It can also be manipulated. A creator could find themselves defending work they actually wrote simply because enough people decided it sounded like AI.

Which leaves us with an uncomfortable problem. On Substack, we can ask whether an algorithm should be deciding what is human. On LinkedIn, we can ask whether other humans should. I’m not sure either one gets us where we need to go.

And that takes us somewhere beyond AI slop.

The problem isn’t simply how we identify what AI created. We haven’t even agreed on what should count as AI-created in the first place.

If I write an article and use AI to challenge my argument, am I the author? I believe I am. If AI edits my grammar, helps with research or suggests a better organization, is that cheating? What if it writes a paragraph that I substantially rewrite? What if I give it my ideas and it writes the entire article?

Somewhere along that continuum, the nature of authorship changes. I don’t know that we have agreed on where.

Perhaps a label like Made by a Human will eventually help us answer that question. But before we can label it, we may need to decide what we mean by it.

Subscribe now

The Reputation Cost

There is another consequence I have been thinking about, and it has less to do with the content than with the person whose name is attached to it.

I notice it in the comments on my own posts. Sometimes I read a comment and suspect that most, if not all, of it was generated by AI. The comment isn’t necessarily bad. It may be complimentary, relevant and perfectly well written. But I find myself less inclined to respond.

That’s interesting to me because my reaction isn’t really to the words. It’s to the possibility that the person didn’t put much of themselves into them. If I’m going to take the time to think about what you said and respond to you, I want to feel that I’m actually in a conversation with you.

And this goes well beyond social media. People are using AI to write cover letters, answer application questions, respond to prospective employers, communicate with potential investors and make introductions. Again, using AI isn’t necessarily the issue. I use it too.

But what happens when the person receiving that communication begins to wonder how much of the thinking actually belongs to the person sending it? I wonder if we’re underestimating the reputational risk. If everything under our name becomes polished, articulate and instant, but people begin to suspect that very little of it actually came from us, what exactly are we signaling about ourselves?

This becomes particularly important when someone is evaluating us. An employer isn’t only evaluating the quality of an application. An investor isn’t only evaluating the language in a pitch. A prospective partner isn’t only evaluating an email. They are trying to understand the person behind it.

AI can help us communicate our thinking more clearly. But if we hand over too much of the thinking itself, we may inadvertently make it harder for other people to know who they’re evaluating.

Maybe that’s another reason the human contribution matters. It isn’t only about protecting the integrity of the work. Sometimes the work is how other people decide whether to trust us.

Why We Care How Things Are Made

There was a time when calling something handmade would have been fairly meaningless. Almost everything was. Industrialization changed that. Mass production made goods less expensive, more consistent and available to far more people. Those were enormous advances, and few of us would want to give them up.

But once mass production became ordinary, handmade began to tell us something that it hadn’t needed to tell us before.

We see versions of that distinction everywhere now. Artisan. Small batch. Locally made. Farm-to-table. Original. They don’t necessarily tell us that something is better. A handmade chair can be poorly made. A mass-produced one can be exceptional.

They tell us something about where something came from and, sometimes, how it was made. That may be closer to what we are looking for with Made by a Human.

The comparison isn’t perfect. AI is not an assembly line, and writing a book isn’t the same as making a chair. More importantly, most creative work today already involves technology. The question isn’t whether technology touched it. It’s how much of the human remained in the process.

The Authorship Question

I’m not sure there is an easy answer. Spellcheck doesn’t change authorship. Neither does an editor suggesting a clearer sentence. And if I use AI to research a question, challenge my thinking or point out something I’ve missed, I still consider myself the author.

But somewhere along that continuum, things get murkier.

If I give AI a topic and ask it to write an article, then make a few changes and put my name on it, is that Made by a Human? What if the ideas were mine but most of the words weren’t? What if AI came up with the ideas too?

We could try to solve this with percentages. How much was written by AI, and how much by a person? But I’m not convinced word count tells us very much about authorship. A person can write every word of something without bringing much original thought to it. Another person can spend hours developing an argument, drawing from years of experience, questioning assumptions and making decisions about what they want to say, then use AI extensively to help express it. Those are very different creative acts.

Perhaps the more useful questions are about agency. Who had something to say? Who shaped the thinking? Who made the choices? Who decided what stayed and what went? And who is willing to put their name on the finished work and stand behind it?

Even those questions won’t give us a perfect dividing line. Maybe we don’t need one.

But I do think we’re going to need better language for the differences.

The Case for a Label

Maybe we do. We’re already experimenting with versions of one.

Amazon distinguishes between AI-generated and AI-assisted content.^[vii] Pinterest gives users some control over how much generative AI content they see.^[viii] Substack is experimenting with detection. LinkedIn is asking users to identify content that feels like AI slop.

None of these approaches settles the question, and some create entirely new problems. But the fact that they exist tells us something. For most of the history of publishing, music, photography and art, we didn’t need a label telling us a human was involved. We assumed it. We can’t assume that anymore.

I don’t know whether Made by a Human will eventually appear on books, music, art, photographs or anything else as a recognized designation. And I’m not sure I’d want a world neatly divided into human and AI categories. The most interesting work may very well come from people who learn how to use these tools without handing over the parts of the process that make the work theirs.

Maybe what we will want isn’t proof of technological purity. We may simply want some transparency about who, or what, we’re engaging with.

Made by a Human

Which brings me back to that Substack preface.

I still don’t think we should have to leave typos in our writing to prove that we’re human. I’m keeping my spellcheck. I like my smooth edges. But I understand the impulse differently now.

The writer I mentioned wanted his readers to know that there was a person behind what they were about to read. Someone had thought about it, decided what he wanted to say and put his name on it.

As AI becomes part of more of what we create, I think we’re going to want to know more about the role the human played.

I don’t know whether Made by a Human will ever become an actual label. But I understand why someone would want to say it. We want to know where the human was in the process. We want to know whether there was thought behind the words, intention behind the image, judgment behind the choices.

Maybe the question isn’t whether AI helped create it.

It’s whether the person whose name is on it can still say, this is mine.

^[i] Double reminders, double standards: how other-use and self-use reminders of AI beauty filters shape observer reactions | Internet Research | Emerald Publishing

^[ii] SLOP Definition & Meaning - Merriam-Webster

^[iii] The transdiagnostic structure of mental effort avoidance | Scientific Reports

^[iv] How can I detect AI on Substack? – Substack, Inc

^[v][v] Substack’s Pangram AI Detector Accuracy: My 159,002-Word Test

^[vi] Best practices for content created with the help of AI | LinkedIn Help

^[vii] Content Guidelines

^[viii] Pinterest rolls out new tools to give users more control over GenAI content | Pinterest Newsroom

I should be good at empathy.

I am a certified coach and integrative hypnotherapist. I’m trained to listen actively, and I’ve managed more than a hundred employees over the course of my career. I know how to listen for what someone is saying, what they aren’t saying, and what may be sitting underneath both.

And I still get it wrong sometimes.

Recently, a friend going through a divorce was confiding in me. I knew my role in that moment was to stay with his experience. To listen. To ask questions. To try to understand what this felt like through his eyes. And then I veered off the path I had been trained to follow. I shared something from my own experience. The relational pull was too strong.

There is something deeply human in the instinct to reach for our own experiences as a way of connecting with someone else. I’ve been through something that feels connected to this. Maybe sharing it will help you feel less alone.

But my experience wasn’t his. In important ways, it wasn’t even close. And the moment I introduced my story, I risked shifting the center of gravity of the conversation away from him and toward me.

I knew better. That’s what makes the moment interesting to me. I knew that empathy required me to stay with his experience, yet the very human desire to connect pulled me toward my own. We often relate to one another by searching our own lives for something familiar. Sometimes that creates connection. Other times, it shifts our attention away from the person in front of us. In trying to show that we understand, we can inadvertently stop trying to understand.

I’ve been on the other side of that interaction many times. I’ve shared something difficult with someone who genuinely wanted to help, only to have them respond with an experience from their own life that they believed was similar. Suddenly, instead of feeling understood, I found myself mentally cataloguing all the ways our situations were different. Sometimes I would try to explain the difference. More often, I wouldn’t. I would leave the conversation feeling worse.

The person listening probably walked away believing we had connected. They had shared something personal. They had tried to meet me in a place they recognized. But I hadn’t needed them to recognize the place. I needed them to be curious about what it looked like from where I was standing.

That distinction has shaped the way I think about empathy.

Empathy is never about us.

I don’t mean that the person practicing empathy has no feelings. I mean that their feelings aren’t the point. The experience of empathy belongs to the person who needs the empathetic ear. My job isn’t to demonstrate how deeply I understand by finding something in my life that resembles what you’re experiencing. My job is to stay curious enough to try to imagine what this experience is like for you.

Sometimes our own experiences help us do that. They can make us more sensitive, more patient, more aware of what someone else might be carrying. But they can also become filters. We assume similarity where there isn’t any. We listen for the part of someone’s story that reminds us of our own and miss the parts that don’t.

And that distinction is at the heart of an argument I’ve been making for several years, one that still makes some people uncomfortable:

AI can be empathetic.

I first made that argument publicly two years ago in my TEDx talk, How Empathetic AI Can Change the World. The most common objection I heard then is still the objection I hear now: But AI doesn’t have feelings.

No, AI doesn’t have feelings. At least not in the way we experience them. But I don’t believe it needs to. In fact, I have come to believe something more provocative: the absence of AI’s own emotional experience may make it better at practicing empathy than most of us are.

Not because humans aren’t capable of extraordinary empathy. We are. I have been on the receiving end of it many times, sometimes from people I barely knew or never would have expected it from. Those interactions stay with me because I walked away feeling something unmistakable: seen, heard and understood.

That is the test. Not whether the person listening has experienced what I’ve experienced, or whether they feel what I’m feeling, but whether I feel seen, heard and understood.

And perhaps we’ve been asking the wrong question about AI and empathy all along.

What Empathy Actually Requires

Part of the confusion begins with the word itself. We use empathy to describe several related but different experiences: understanding what another person is feeling, feeling something ourselves in response, caring about their distress, wanting to help, or imagining how we would feel in the same situation. We bundle these together and call them empathy. Once we do that, the argument against AI seems almost self-evident. AI doesn’t have emotions, therefore AI cannot be empathetic.

But empathy does not require me to feel what you are feeling. It requires me to try to understand what you are feeling.

Psychology commonly distinguishes between cognitive empathy, our ability to understand another person’s perspective and emotional state, and affective empathy, the emotional response we may experience to what someone else is feeling. They often occur together, but they aren’t the same thing. More emotion on the part of the listener doesn’t necessarily mean more empathy for the person who needs it.^[i]

Empathy is a cognitive skill. It requires active listening, perspective-taking, curiosity, imagination and emotional recognition. It also requires enough self-awareness to notice when our own experience is entering the conversation and enough self-regulation not to let it take over.

Imagination is particularly important. We use it constantly to enter experiences we’ve never had. We watch a movie and become emotionally invested in a fictional character whose life bears no resemblance to our own. We read a novel and enter the world of someone who may have lived in another country or another century. We see a frightened animal and instinctively think, that poor dog. We don’t need a shared experience to imagine what another experience might feel like.

We do the same thing with one another. I don’t need to have lost your parent, received your diagnosis or ended your marriage to imagine what you might be experiencing. In some ways, not having had the same experience can keep me more curious because I have fewer assumptions about what yours must feel like.

Feeling can certainly accompany empathy. Often it does. But feeling isn’t the qualification for empathy, and more feeling doesn’t necessarily make us better at it. Sometimes it can do the opposite.

Emotional contagion is our tendency to take on some of the emotional state of the people around us.^[ii] That emotional resonance can help us connect, but it can also backfire. If you are frightened and I become consumed by your fear, I have less capacity to help you navigate it. If your grief overwhelms me, some of the emotional space in the conversation has shifted toward my grief. At its extreme, the person who needed support can end up comforting the person who was supposed to be supporting them.

Feeling deeply for another person can be beautiful. But feeling more isn’t necessarily empathizing better. What empathy requires is that I remain close enough to understand your experience without becoming so entangled in it that I can no longer keep you at the center.

There is another part of empathy I think we often get backward. We tend to judge it from the perspective of the person providing it. I felt terrible for her. I really empathized with him. I know exactly what she’s going through. Those statements tell us something about the listener. They don’t tell us whether the other person felt understood.

I can care deeply about you and still fail to understand you. I can have lived through something remarkably similar and become so convinced that I know how you feel that I stop listening to how you actually feel.

For me, the more meaningful test happens on the other side of the interaction. Did I listen without judgment or distraction? Did I stay curious rather than decide too quickly that I understood? Did you feel safe enough to tell me what you were actually experiencing? Did you leave feeling seen, heard and understood?

We already accept that shared experience isn’t required when we’re talking about human beings. A therapist doesn’t need to have experienced every trauma of every client. A physician doesn’t need to have had cancer to understand that a patient is frightened. If shared experience were a prerequisite for empathy, our ability to empathize with people whose lives differ from ours would be remarkably limited.

And yet shared experience is one of the first things we demand when the conversation turns to AI. How can it understand grief if it has never grieved? How can it understand fear if it has never been afraid? How can it understand heartbreak if it has never loved?

Those are reasonable questions. But they hold AI to a standard we don’t hold ourselves to. The better question is whether lived emotional experience is necessary to understand and respond empathetically to someone else’s experience.

I don’t believe it is.

Subscribe now

Empathy Is a Bridge to Trust

When we don’t feel understood, the walls go up. We protect ourselves. We explain. We defend. We withdraw. We become more careful about what we’re willing to reveal because the person on the other side doesn’t seem to understand what we’re trying to say.

When we feel genuinely heard and understood, something different happens. The walls begin to come down. We don’t necessarily change our minds or agree with the person sitting across from us, but we become more willing to stay in the conversation and consider another perspective.

This is where empathy becomes a foundation for trust.

Trust doesn’t require agreement. Neither does empathy. One of the most important things an empathetic person can do is disagree with us without making us feel dismissed. Understanding my perspective doesn’t require you to adopt it. It means you’ve taken the time to understand where I’m standing before asking me to consider standing somewhere else.

Empathy is the bridge that carries us from uncertainty to openness, and from resistance to readiness.

That matters in our closest relationships, but it also matters in leadership, healthcare, coaching, education, and anywhere we are asking another human being to navigate uncertainty or change. Before people are ready to move, they often need to feel that someone understands why moving is difficult in the first place.

And this is where the possibility of empathetic AI becomes much bigger than whether a chatbot can say the right comforting words.

What AI Doesn’t Bring to the Conversation

If empathy is about understanding another person’s experience rather than sharing it, AI’s lack of personal experience may be an advantage rather than a limitation.

AI has no divorce story to compare with mine. It doesn’t hear about my loss and immediately remember its own. It has no need to demonstrate that it understands by finding common ground. It doesn’t become uncomfortable with my emotions, distracted by what is happening in its own life, or so overwhelmed by what I’m feeling that I need to take care of it.

It can stay with me.

That doesn’t mean AI is neutral or infallible. It can misread what we’re saying, make assumptions, reinforce something that shouldn’t be reinforced, or become so agreeable that validation replaces honest reflection. AI brings biases of its own, learned through the data, systems and choices that shaped it. The absence of personal experience doesn’t mean the absence of error.

And simply sounding empathetic isn’t the same as being empathetic.

This is where design matters. An AI built specifically for empathy can be trained and fine-tuned to do more than produce the language of empathy. It can be taught to listen before responding, to look for its own assumptions and biases, to ask rather than infer when it doesn’t have enough information, and to distinguish what the person is actually expressing from patterns it recognizes in other people’s experiences.

In other words, it can be designed to resist some of the shortcuts humans need to learn to resist. It can also remain present and focused even when multiple processes are going on in the background. Something difficult for us mortals.

Active listening is a discipline for humans precisely because our minds are rarely quiet while someone else is talking. We’re interpreting, remembering, reacting, preparing our response, making associations. Sometimes we’re listening to understand. Sometimes, without realizing it, we’re listening for our turn to speak.

AI doesn’t experience that internal competition in the same way. An AI deliberately trained for empathy can attend to the words we use, the patterns in what we’ve shared, the emotional cues in our language and the context that came before, while continuing to test its assumptions rather than treating them as truth. It can ask another question instead of assuming it already knows the answer.

The paradox is that the very thing we point to as proof that AI cannot be empathetic, its lack of feelings or lived experience, may remove some of the things that make empathy difficult for us.

There are early signs that people are experiencing something like this already. A 2025 systematic review and meta-analysis of 15 studies comparing AI chatbots with healthcare professionals found that 13 reported significantly higher empathy ratings for AI.^[iii] I don’t think that proves AI is universally better at empathy than humans. These studies have limitations, and a text exchange is hardly the whole of human connection.

But they do challenge the assumption that the absence of human feeling prevents a person on the other side of the interaction from experiencing empathy. It’s also a reminder that empathy isn’t a performance. It has an outcome. If I walk away feeling genuinely seen, heard and understood, what exactly are we claiming was missing simply because the intelligence listening to me didn’t experience my emotion itself?

Share

Empathy When We Need It

The other reason I have believed in empathetic AI from the beginning is access.

We don’t schedule our hardest moments. Uncertainty doesn’t wait for our next coaching session. Anxiety doesn’t arrive only during office hours. Sometimes it’s late at night. Sometimes we’re sitting in the car five minutes before a meeting. Sometimes something has happened and we aren’t ready to tell another person yet.

And sometimes there isn’t another person.

We talk about empathy as though it is readily available to anyone who needs it. It isn’t. Therapists and coaches cost money. Managers vary enormously in their emotional intelligence. Friends and family may love us deeply but not know how to listen without advising, fixing, judging or bringing their own experiences into ours. Even the most empathetic people in our lives have jobs, families, bad days, distractions and limits of their own.

Imagine having access to an empathetic ear in the moment when everything feels bigger than it probably will tomorrow. Not something that rushes to give you an answer or simply assures you that you’re right, but something that helps you slow down, listens first, asks questions and gives you enough space to see what might be sitting underneath your immediate reaction.

Sometimes what we need first isn’t advice. We need somewhere to put what we’re carrying.

When we feel threatened or overwhelmed, our thinking can narrow around whatever is immediately in front of us. A good empathetic interaction can create enough space to begin regulating, reflecting and considering what comes next.

Now imagine having access to that kind of support whenever you need it.

For the employee sitting in the parking lot before a difficult meeting. The new manager trying to navigate a conflict on her team. The caregiver at the end of an exhausting day. Someone awake at 2:00 a.m. replaying a conversation. Someone who can’t afford a coach or doesn’t have access to a therapist. Someone who simply isn’t ready to tell another human being what they’re thinking.

This isn’t an argument for replacing therapists, coaches, friends, family or human connection. Human connection and empathetic AI don’t have to be competitors.

An empathetic AI can help us prepare for a difficult conversation with another person. It can help us understand why we’re reacting the way we are before we take that reaction into a relationship. It can help us consider another person’s perspective or find words for something we haven’t yet been able to articulate. Sometimes it may help us become more ready for the human conversation.

And for people who don’t have an empathetic human available, the comparison isn’t between AI empathy and perfect human empathy. It may be between empathetic AI and no empathetic support at all. That changes the equation considerably.

Empathy Changes What Happens Next

The value of empathy isn’t confined to how someone feels during a conversation. What happens when people feel understood affects what they’re able and willing to do next.

We see it in relationships, healthcare, coaching and therapy. We certainly see it at work. A person who feels dismissed is more likely to protect their position. An employee who believes a leader doesn’t understand what a change will actually require of them is more likely to resist it. A team that doesn’t feel heard can comply outwardly while disengaging underneath.

Empathy doesn’t eliminate resistance, and it shouldn’t. Sometimes resistance is telling us something important. What empathy does is help us understand what is underneath it.

That distinction becomes particularly important during change. When people resist, our instinct is often to try harder to persuade them, explain the benefits again, or give them more information. But if the resistance is rooted in fear, identity, confidence, loss of control or mistrust, information may not be what they’re missing. People are more willing to examine what they believe when they don’t feel they have to defend themselves first.

This is why I don’t think empathy is soft. It has consequences. Research on empathy in the workplace has connected empathetic leadership with outcomes organizations care deeply about. In an EY survey, 87% of workers said mutual empathy between employees and leaders increases efficiency, another 87% said it boosts creativity, and 81% believed it increases company revenue.^[iv]

Empathy isn’t fuzzy. It is measurable and material.

And if we can make high-quality empathetic interactions more accessible, we aren’t simply helping people feel better in difficult moments. We may be helping them regain enough openness, perspective and agency to decide what comes next.

The Responsibility of Understanding

There is another side to all of this. If an AI becomes better at understanding what frightens us, motivates us, makes us feel safe or causes us to shut down, that understanding carries enormous responsibility. The same insight that can help someone regulate, reflect or see another perspective can also be used to persuade, influence or manipulate.

We know this from human relationships. Understanding another person’s emotions doesn’t guarantee that understanding will be used in their best interest. Empathy is a capability. What matters is what we do with it.

The same has to be true for AI.

An empathetic AI should not be designed to make us dependent on it, keep us engaged longer, reinforce everything we believe, or tell us whatever will make us feel better in the moment. Feeling understood is not the same as being agreed with, and validation without discernment can become its own form of harm.

This is why the intention behind the system matters as much as its ability to understand. Is it helping me become clearer, more regulated and more capable of making my own decisions? Is it willing to challenge me when I may be missing something? Does it preserve my agency? Does it know when the right next step is to encourage me to reach out to another human being?

These aren’t peripheral design questions. They are part of what it means to build empathetic AI responsibly.

And they bring us right back to trust.

What AI Can Teach Us About Empathy

Two years after standing on a TEDx stage talking about how empathetic AI could change the world, I believe in the possibility even more strongly. The years I’ve spent building empathetic AI have also changed the way I think about human empathy.

In trying to teach a machine what good empathy requires, we are forced to become more precise about what we require of ourselves: listen before assuming, stay curious, notice bias, don’t mistake agreement for understanding, and keep the person who needs empathy at the center.

Something may also happen when we experience that kind of empathy ourselves. When we are on the receiving end, we can learn from it. A patient listener reminds us what it feels like not to be interrupted. A thoughtful question can show us the difference between curiosity and assumption. An interaction that stays focused on our experience can make us more aware of the next time we are tempted to pull someone else’s experience toward our own.

If AI can practice empathy consistently, more of us have the opportunity to experience what good empathy feels like and, perhaps, carry some of that into our interactions with one another.

I’ve already heard this from people using AI. After experiencing how patiently an AI listened and responded, my friend Brian told me it caused him to pause during a later conversation with another person. He found himself listening differently.

I find that possibility fascinating. We tend to think about AI learning from humans. What if, in this very human skill, we can also learn from AI?

Empathy is a cognitive skillset. It can be taught, modeled, and learned.

Perhaps the greatest promise of empathetic AI is that more people might have access to the experience of being understood when they need it most. And by experiencing what good empathy feels like, perhaps we all become a little better at offering it to one another.

Two years ago, I believed empathetic AI could change the world.

I still do.

Not because it feels. Because it stays.

^[i] Cognitive and Affective Perspective-Taking: Evidence for Shared and Dissociable Anatomical Substrates - PMC

^[ii] https://pubmed.ncbi.nlm.nih.gov/17015094/

^[iii] AI chatbots versus human healthcare professionals: a systematic review and meta-analysis of empathy in patient care - PubMed

^[iv] Empathic leadership & the Great Resignation | EY - US

We have built an extraordinary amount of infrastructure for artificial intelligence.

Data centers stretching across hundreds of acres. Chips engineered for AI workloads. Power generation, cooling systems, fiber networks, cloud platforms, foundation models, APIs, applications, and now agents capable of acting on our behalf.

The investment is staggering. So is the speed.

At the same time, governments, researchers, and companies are wrestling with how to deploy that capability responsibly: governance, security, privacy, explainability, accountability. Much of it is still being figured out while the technology continues to advance.

But another layer of infrastructure seems to be missing: the infrastructure for the humans expected to live and work alongside it.

I don’t mean more training. Training matters. People need to know how to use the tools, but knowing where to click isn’t the same as being ready for what the technology is changing.

People are being asked to rethink how they work, how they make decisions, what expertise means, which parts of their jobs still belong to them, and increasingly, what they can trust. Leaders are being asked to guide organizations through changes they are still trying to understand themselves.

When we talk about keeping pace with AI, I don’t mean asking people to think as fast as machines or produce at machine speed. That’s the wrong race. I mean giving people what they need to adapt as the world around them changes faster.

We’ve spent enormous resources building the infrastructure AI needs to become more capable, and we’re beginning to build the infrastructure required to govern that capability responsibly.

Now we need to build the infrastructure that supports the people living with it.

Subscribe now

The Reckoning Was Never Just About AI

When I started The AI Reckoning series, I knew I wanted to get to the human part of this story, but I didn’t want to start there.

Before we can understand what a system is doing to people, we need to understand the system itself. Its foundations. Its constraints. The assumptions we’ve built into it. The economics driving it. The dependencies we’ve created. And the secondary consequences that may not be obvious when we’re focused on what the technology can do.

That’s how I tend to approach complex problems. I want to understand the environment producing the experience before I try to explain the experience itself.

So we began underneath AI, with the physical infrastructure most of us never see: the energy, water, compute, and environmental costs hidden beneath something that can feel almost weightless on a screen.

From there, we looked at the economics behind the extraordinary investment, the risks of building businesses on intelligence someone else owns, and the fragmentation created by an explosion of disconnected tools.

I wanted to look at those things with eyes wide open, not because I believe they diminish the promise of AI, but because they shape the environment in which that promise has to be realized.

Then we could turn to the human side.

We looked at adoption and resistance, at what happens to judgment when information becomes abundant, and at what happens to trust when seeing is no longer believing or when an AI system can give us an answer no one can fully explain. And most recently at what happens when we become exceptionally good at solving problems without understanding the systems producing them.

The progression was intentional. People aren’t responding to AI in isolation. They’re responding to the technology and to the conditions surrounding it: the pace of change, uncertainty about work, questions of trust and authenticity, decisions they may not understand, tools that don’t always fit together, and organizations that are themselves still figuring out what comes next.

We can call those human problems. But that doesn’t mean the human is where the problem started. If we look only at the person struggling to adapt, we may conclude that they need more training, more resilience, or a better attitude toward change. If we look at the system around them, we may see something very different.

Across these eleven reckonings, the question underneath the series has become increasingly clear: What happens when technological capability advances faster than our ability to understand, govern, absorb, and trust its consequences?

The answer isn’t to ask humans to simply move faster. It’s to understand what they need in order to move through change well.

When Intelligence Is Abundant, What Becomes Scarce?

For most of human history, access to knowledge and expertise was scarce.

If you needed a legal opinion, you called a lawyer. If you wanted to understand a medical study, you needed someone who knew how to interpret it. Writing software or analyzing a financial model required either the expertise yourself or access to someone who had it.

AI is changing that equation remarkably quickly. Knowledge isn’t suddenly infinite, and AI certainly isn’t always right. But access to something that can explain, analyze, synthesize, create, and increasingly reason across enormous amounts of information is becoming available to almost anyone.

When something that was scarce becomes abundant, value tends to move somewhere else. If answers become easier to generate, knowing which questions to ask becomes more important. If content becomes abundant, discernment becomes more important. If analysis can be produced in seconds, judgment about what to do with it becomes more important. And if something can look and sound completely real without being real, trust becomes more important.

We’ve touched each of these questions throughout The AI Reckoning. Taken together, they point to something larger: The more intelligence we have access to, the more important our relationship with that intelligence becomes.

Can we evaluate it, question it, and put it in context? Can we recognize what it doesn’t know or what assumptions may be shaping it? Do we know when not to use it? Are we willing to take responsibility for what happens when we do?

Those aren’t primarily questions of machine capability. They’re questions of human capacity. And capacity is not the same as productivity. Much of the conversation about AI and people still centers on how much more humans will be able to produce. The productivity opportunity is real and potentially enormous. But if we define human readiness primarily by how efficiently people can use AI to produce more, we may be measuring the wrong thing.

We can become dramatically more productive without becoming more discerning. We can make decisions faster without making better ones. An organization can deploy AI everywhere without building greater trust in how it is being used. More capability doesn’t automatically create more capacity to use it well. In some ways, it demands more from us.

As AI becomes increasingly embedded in our work, the human capacities that once seemed difficult to measure or easy to dismiss as “soft” start to look much less optional: judgment, curiosity, discernment, adaptability, trust, the ability to sit with uncertainty long enough to ask another question, and the confidence to challenge an answer when everyone else is ready to move forward.

The better AI gets, the more these capabilities matter.

The Visibility Gap

At the end of the last article, I left you with a question:

What if the things leaders most need to understand are the very things they have the hardest time seeing?

Organizations have never had more data. Leaders can track productivity, performance, turnover, absenteeism, engagement, technology usage, customer behavior, and increasingly almost anything that leaves a digital trail. AI can find patterns across enormous amounts of that data and surface relationships a human analyst might never see.

But much of what determines whether people can actually adapt to change doesn’t leave a clean digital trail.

Are people struggling with a new technology because they don’t understand it, don’t trust it, or are afraid of what it means for their future? Is someone quiet in a meeting because they agree or because they’ve learned that speaking up isn’t worth the risk? Are people genuinely adopting a change, or complying until the attention moves somewhere else?

We can see the behavior. Understanding what’s underneath it is much harder.

That’s the visibility gap.

Our data mostly tells us what happened after something became visible. Turnover tells us who left. Usage tells us whether a tool is being used. Productivity tells us what is getting produced. An engagement score tells us something about how people are responding.

By the time those signals move enough to command attention, the conditions producing them may have been developing for months. And some of the most valuable information may never enter the organizational data stream at all.

People filter. They decide what is safe to say, who it is safe to say it to, and how much of what they’re experiencing they want attached to their name.

That doesn’t mean leaders don’t care or aren’t asking. Even very good leaders have a structural problem: the fact that they are in a position to act on information can affect what people are willing to tell them.

Surveys, listening sessions, town halls, manager check-ins, sentiment analysis, and engagement measures can all provide useful information. But there is a difference between asking people what they think and creating the conditions in which they are willing to tell you.

There is also a difference between collecting responses and understanding the patterns connecting them. A trust problem may appear as an adoption problem in one part of the organization, turnover somewhere else, resistance in another, and declining productivity somewhere else again. If each function sees only its own metric, no one may recognize that they’re looking at different expressions of the same underlying condition.

That’s the visibility we’re missing. Not more information about what happened, but a better understanding of what’s happening underneath it. The insight has to be able to travel upward without the individual’s identity traveling with it.

Share

What Human Infrastructure Looks Like

Human infrastructure has to work in both directions.

For the individual, change is personal.

An organization may describe an AI transformation in terms of productivity, efficiency, competitive advantage, or new capabilities. The person experiencing it may be wondering whether they’re still good at their job, whether the expertise they spent years developing still matters, whether they’re expected to use a tool they don’t trust, or what any of this means for the role they’re adapting to.

Those aren’t problems a training module can solve. People need somewhere private enough for candor, where they can question their assumptions, understand what may be triggering a response, work through a difficult interaction, make a decision, or simply get enough perspective to see what’s happening differently. That support has value even if the organization learns nothing from it. The individual cannot simply become a richer source of organizational data. Human infrastructure has to serve the human first.

But when many people have a trusted place to work through what they are actually experiencing, something else becomes possible. Patterns begin to emerge. What looks like resistance may actually be a trust problem. People may understand the technology but not understand how their roles are changing around it. A workload issue may be appearing across teams that look unrelated on an organizational chart.

Visibility alone isn’t enough. We already have dashboards that tell us something is happening. Human infrastructure should help us get closer to why. This is where AI can be enormously useful. It can identify patterns across conversations and experiences, connect signals that would otherwise remain separate, and surface possibilities a leader may not have known to look for.

But an insight isn’t a decision. Someone still has to determine what it means, whether it makes sense in context, and what action is appropriate. The need for judgment doesn’t diminish as AI becomes more capable. It increases.

So when I talk about human infrastructure, I mean something that helps people navigate change while helping organizations understand what is happening across the people experiencing it. Something that can connect patterns to possible causes and paths forward without pretending complex human behavior can be reduced to a clean equation.

The individual gets support. The organization gets better visibility. Leadership has a better basis for action. And what happens next becomes new information about the system.

This gives change a better chance to take hold, and that’s when transformation becomes possible.

Trust Has to Be Part of the Infrastructure

None of this works without trust.

If we want to understand what people are actually experiencing, they have to feel safe enough to be honest about it. That becomes harder when the organization receiving the insight is also the organization that controls their job, their opportunities, and, in some cases, their future.

This is where privacy becomes fundamental. People want issues addressed. They just don’t want to be identified as the one who raised them, and that distinction is what makes candor possible. Trust doesn’t require that every concern be resolved or that everyone agree with every decision. But it does require people to believe that telling the truth doesn’t come with repercussions, whether immediate or delayed. When people believe that, the information leaders actually need, the kind that explains what’s really causing a problem, finally reaches them.

That is why trust can’t be something we hope emerges after the technology is built. It has to shape what we build, what we protect, what leaders can see, and what happens with what they learn.

Trust isn’t downstream of the AI transformation. It is part of the infrastructure that determines whether the transformation works.

Why I Built Parsley360

I didn’t start Parsley360 (Parsley™) because I wanted to build an AI company. I started because I wanted to solve a problem, one I had seen across my career and that became amplified in the wake of the COVID pandemic.

I had seen decades of progress seem to slide backward as global uncertainty and unrest took hold. This was a macro example of change that didn’t stick. But I had experienced the same pattern at smaller levels across organizations, teams, and individuals.

Organizations would devote enormous amounts of time, money, and energy to transformation. New strategies, technologies, processes, structures. People would move. Metrics would improve. For a while, it could look like the change had worked.

And then came the backslide.

I think of it like stretching a rubber band. Unless something fundamentally changes, there is a force pulling it back. It may never return to exactly its original shape, but the snapback is real. That was my frustration. Something that transforms doesn’t simply change for a while. It becomes something new.

So I kept asking the question I tend to ask about almost everything: Why?

Why didn’t the change hold? Why did one group adopt it while another resisted? Why could an organization successfully implement something without ever fully realizing the transformation it was intended to create?

I’m a systems thinker by nature. I tend to look beneath the visible problem and ask what is producing it, what else it is connected to, and what happens if we intervene in one place without understanding the rest. That made it increasingly difficult for me to look at adoption as an implementation problem.

Adoption is human.

Training can teach someone how to use a new system. Communication can explain why the organization is changing. Incentives can encourage behavior. Measurement can tell us whether usage went up. None of those things necessarily tells us whether the change has taken hold.

Someone can use technology without trusting it, follow a process without believing in it, or comply with a change while waiting for the opportunity to return to the old way of working. Implementation can happen without transformation.

For transformation to be fully realized, both the system and the people within it have to change in ways that can hold.

That’s the problem I wanted to solve more holistically. In my work with individuals, I’ve seen how much can change when someone feels seen, heard, and understood. I’ve also learned that support has to be available in the moments that matter. That requires something close to omnipresence. I’d done it with large teams, but only by being everywhere myself. That’s a recipe for burnout, not a business.

In organizations, I saw leaders making decisions with incomplete information. They were flying blind but often didn’t know it. They acted on the data they had and remained frustrated when the outcomes didn’t change.

For a long time, there wasn’t an obvious way to bridge those realities. Then AI changed what was possible.

I could imagine an environment available whenever someone needed it, where a person could privately work through what they were experiencing and begin connecting their own dots: what they’re feeling, what may be triggering it, the assumptions or patterns underneath it, and what they want to do next.

AI also created the possibility of understanding patterns across those experiences without exposing the individuals behind them. Leaders could begin to see that what appeared to be an adoption problem might actually be a trust problem, recognize something shifting before it became an outcome on a dashboard, and have a better path forward.

That became the idea behind Parsley360: Connect the dots and close the gaps.

That idea shaped the architecture as much as the mission. Early on, Craig and I faced the same question every builder in this space eventually faces: build on top of someone else’s model, or build the reasoning and empathy layers ourselves. We talked about this in the third piece of this series, the risk of building something essential on ground you don’t own. That risk was reason enough. But there was a second one that mattered just as much.

The reasoning and empathy layers are where a person’s private experience lives, the things they’re willing to say only because they trust it goes no further than it needs to. That kind of trust can’t depend on a platform that could reprice, restrict, or shift its priorities without warning. If the privacy we promised was only as durable as someone else’s roadmap, it was never really a promise.

So we built our own. Parsley360 runs on a proprietary reasoning engine designed for empathetic, long-form discourse, not a wrapper around someone else’s model. It was slower to build. It also means the thing at the center of this company, the part actually doing the work of understanding a person, answers to us and to the people using it, not to a platform whose interests may eventually diverge from ours.

The technology has evolved considerably since we began building it. The problem we’re trying to solve hasn’t.

How do we help people navigate change in a way that actually benefits them, while helping organizations understand what their people need to make that change sustainable?

How do we move beyond implementation to adoption, and beyond adoption to transformation that lasts?

That’s the human infrastructure we’ve spent the last several years building.

The Next Thing We Build

When I began The AI Reckoning, I said I was writing as a builder, not a critic. Twelve articles later, I still am.

I remain deeply optimistic about what artificial intelligence can make possible. But if we’re serious about building something transformative, we have to be willing to look at the whole system it will live inside. I don’t come away from this series believing we should build less. I come away believing we haven’t finished building.

We’ve spent extraordinary resources making artificial intelligence more capable, and we’re beginning to wrestle seriously with how to govern and deploy it responsibly. Now we need to bring the same intention to the humans living and working alongside it.

That means recognizing that transformation isn’t fully realized because a technology has been deployed or a metric has moved. It is realized when the system and the people within it have changed enough that we don’t simply snap back.

The next chapter of artificial intelligence will be shaped by what we build next.

Let’s make sure we build for the humans too.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

Part 7: The Generation Saying No: Is Anyone Listening?

Part 8: When Seeing Is No Longer Believing: Trust Under Attack

Part 9: The Room Where Everyone Agrees With You

Part 10: The Black Box Problem: Why Explainability Is the Foundation of Trust

Part 11: Why We Keep Treating Symptoms Instead of System: What We Miss When We Only Solve What We Can See

Imagine you have a nail in your foot.

It’s painful, so of course you take your pain medicine of choice. Relief.

A few days later, your foot becomes infected. Now you need an antibiotic. It does its job; the infection seems to be under control.

Now you are healing… or are you?

The nail is still there.

We get remarkably good at treating symptoms. A pill for an ill. We relieve the pain, treat the infection, and manage the damage, while the thing creating it remains.

I’ve used some version of this example for years because it’s absurdly obvious. Most of us don’t need to be taught the difference between treating symptoms and addressing what’s causing them. If there’s a nail in your foot, you take out the nail.

And yet, we do the organizational equivalent every day.

When people are showing signs of burnout, we offer wellness programs and resilience training. When employees aren’t adopting a new technology, we give them more training. When an AI system produces something we don’t want, we add another guardrail.

None of those responses is wrong. The person in pain needs relief. An infection needs treatment. The employee may need support, and the AI system may absolutely need the guardrail.

The problem begins when the treatment becomes our definition of the problem.

That’s where my nail analogy starts to break down. A nail is wonderfully simple. There is an identifiable object causing the damage. Find it, remove it, treat what it damaged, and the body can begin to heal. Organizations rarely give us anything that clean.

There may not be one thing to remove. Burnout may be connected to workload, staffing, leadership, technology, expectations, incentives, and a culture that says wellbeing matters while quietly rewarding the people who never stop working.

Low AI adoption may have very little to do with whether employees know how to use the technology. The tool may not fit the workflow. People may not trust its output. Their manager may not use it. Or employees may be wondering what happens to their jobs if the productivity gains leadership keeps promising actually materialize.

Which one is the nail?

Maybe none of them.

Maybe what we’re looking at isn’t a collection of individual problems at all. Maybe we’re looking at what the system, taken as a whole, is producing. We treat the symptom because the symptom is what we can see, measure, and act on this quarter. Meanwhile, the system stays intact, quietly producing the next one.

That raises a different question, and I think a more important one. What if we’re solving problems systematically without first understanding them systemically?

Seeing the Problem Before Solving It

The words sound almost interchangeable, but systematic and systemic thinking do very different work.

Being systematic is about how we solve a problem. We gather information, identify what needs to change, create a plan, assign responsibility, act, and measure the result. Organizations are generally very good at this. We have methodologies, dashboards, project plans, KPIs, owners, timelines, and no shortage of ways to organize ourselves around a problem once we’ve decided what the problem is.

Being systemic starts earlier. Before we decide how to solve something, we have to understand what we’re actually looking at. That’s where root cause becomes important. If engagement is falling, why? If people are burning out, what’s creating the pressure? If employees aren’t using a new technology, what’s getting in the way? The first answer may point us in the right direction, but it may not take us far enough.

A team may be burning out because they’re understaffed. That looks like the cause until we ask why they’re understaffed. Perhaps positions have been frozen because of a cost-reduction target, or people have left and haven’t been replaced. Keep looking and we may discover that the workload hasn’t changed even though the resources have. What appeared on the surface as a burnout problem has taken us somewhere else entirely.

Thinking systemically means looking beneath what is visible rather than stopping at the first plausible explanation.

Systems thinking widens the view further. It asks us to look not only at what’s underneath the problem, but at what’s around it.

How are workload, incentives, leadership behavior, technology, processes, trust, and culture interacting? What is reinforcing what? Is the same pattern showing up somewhere else? What happens to the rest of the system when we change one part of it?

Organizations aren’t machines where we can replace one faulty part and assume everything else will continue exactly as before. People respond. Managers adapt. Incentives change behavior. Trust can grow or erode. An intervention in one place can create an effect somewhere we weren’t looking.

We need root-cause analysis and the wider lens of systems thinking. Then we need to be systematic about what we do with what we’ve learned.

But getting to that understanding depends on something we often assume we already have: an accurate picture of what’s really happening, and that’s much harder than it sounds.

Subscribe now

Why We Don’t See What’s Really Going On

The higher you move in an organization, the harder it can become to know what’s actually happening inside it. That sounds counterintuitive. Leaders have more information than almost anyone else. They have dashboards, surveys, financial reports, employee data, customer feedback, performance metrics, and teams whose job it is to keep them informed. But having more information isn’t necessarily the same as having greater visibility.

People edit what they say. Sometimes they do it because they’re afraid of the consequences. An employee who doesn’t trust their manager isn’t likely to say exactly what they think about a new initiative, a reorganization, or the way their team is being led. They learn what is safe to say, what should be softened, and what is better left unsaid.^[i][ii]

Sometimes the filtering is less deliberate. A manager hears frustration from their team but doesn’t want to bring leadership another problem without a solution, so they translate it into something more actionable. Another manager believes the concern is temporary and decides not to escalate it. Someone farther up summarizes ten conversations into three bullet points for an executive meeting. At every step, the information may be accurate, but something can still get lost. By the time it reaches the people making decisions, the signal may look very different from where it started.

Organizational structure adds another challenge. HR sees engagement and turnover. IT sees technology usage. Operations sees productivity. Finance sees cost. Managers see what is happening on their teams. Each may hold a legitimate piece of the picture without anyone seeing how those pieces fit together.

Then there is the data itself. We can measure whether people are using a new AI tool. We can measure turnover, absenteeism, productivity, engagement scores, and any number of other outcomes. Those measures can tell us something important is happening. They don’t necessarily tell us why. Yet numbers have a way of feeling more definitive than the human information around them. If usage is low, we see an adoption problem. If engagement scores fall, we see an engagement problem. If productivity drops, we see a performance problem. Once we’ve named the problem, we begin looking for evidence and solutions that fit the name we’ve given it. The label itself can narrow what we’re willing to see.

There is another layer that is harder to talk about. People naturally protect the things they are invested in. A leader who championed a transformation may be more inclined to see resistance from employees than flaws in the transformation itself. A manager whose team is struggling may see a resource problem before considering whether their own leadership is contributing to it. An executive who made a difficult decision may genuinely believe the problem is in the execution rather than the decision.

None of this requires bad intentions. Some of the filtering happens precisely because people are trying to do their jobs well, protect their teams, support a decision, or avoid creating unnecessary friction. But good intentions don’t guarantee good information.

Underneath all of this is trust.

If people don’t believe they can tell the truth without paying a price for it, the organization loses access to information it may not be able to get any other way.^[iii] A survey can tell you that trust is low, but it can’t tell you what someone would say about their manager if they knew their manager would never hear it. A dashboard can tell you that adoption is lagging, but it can’t tell you that employees understand the technology perfectly well but are afraid of what successful adoption might eventually mean for their jobs.

This creates a difficult loop. The less people trust the organization, the less likely they are to say what they’re really experiencing. The less leadership hears, the harder it becomes to understand what’s actually happening.

Decisions are then made from an incomplete picture, and if those decisions miss what people are experiencing, trust can erode further. Leaders may respond by asking for more information. Another survey. Another dashboard. Another metric. Better analytics. Those things can help. But more data doesn’t necessarily reveal what people don’t feel safe enough to say. And if we can’t see what’s really happening beneath the surface, even the most disciplined problem-solving process can lead us somewhere we didn’t intend to go.

When the Solution Changes the System

Once we understand the problem, the next instinct is to fix it. That’s exactly what we should do. Understanding what created the problem, though, is only part of the work. We also need to understand what happens when we introduce a solution.

An intervention doesn’t enter an organization in isolation. People respond to it. They interpret what it means. They adjust their behavior. It can change incentives, relationships, workload, trust, and sometimes the very conditions we were trying to improve.

AI adoption is a good example.

Imagine an organization has invested heavily in new AI tools, but usage is well below expectations. Leadership sees the numbers and responds. Employees get more training. Managers are given adoption targets. Usage becomes part of team conversations. Perhaps dashboards are introduced so leaders can see where the technology is and isn’t being used.

The numbers begin to rise. It looks like the intervention worked. But what exactly went up? Employees may be using the technology because they understand its value, trust the direction the organization is taking, and have found meaningful ways to incorporate it into their work. Or they may be using it because their manager expects them to and they know someone is watching the numbers. The dashboard may record both as adoption. Only one of them is. The other is compliance.

In the short term, they can look remarkably similar. Over time, they can lead somewhere very different. If the reason people weren’t using the technology in the first place was lack of training, then more training may have addressed exactly what was needed. But what if training was never the real issue? What if people were concerned about the accuracy of the tool, unsure how it fit into their work, or worried about what the promised productivity gains might eventually mean for their jobs?^[iv]

Now the intervention is acting on something other than the condition that produced the behavior. Usage may increase, but the uncertainty hasn’t gone away. If employees feel pressured or monitored, trust may decline further. People may become less willing to voice concerns because they’ve learned that the organization has already decided what the desired behavior is. Managers see the improving numbers and report that adoption is going well. Leadership receives confirmation that the intervention worked.

The original problem hasn’t necessarily been solved. It may simply have become harder to see. This is where looking upstream and downstream becomes important. Upstream are the conditions that helped produce what we’re seeing in the first place: trust, belonging, leadership behavior, incentives, workload, fear, the way decisions are communicated, whether people believe they have a voice in changes that affect them.

Downstream is what happens after we intervene. How do people respond? What behaviors change? What happens to trust or belonging? Does the solution create more work somewhere else? Does it solve a problem for one group while creating one for another? What happens six months later, after the initial pressure, attention, or incentive is gone? Those effects won’t necessarily show up in the metric we were trying to improve. Turnover can drop after retention bonuses are introduced, but that doesn’t tell us whether people actually want to stay or whether we’ve simply made leaving more expensive. A team can produce more with fewer people while employees work longer hours and absorb a workload that eventually shows up as burnout or attrition. Engagement scores can improve after managers are held accountable for them while employees become more careful about how they respond.

The metric isn’t wrong. It’s measuring what we asked it to measure. The question is what we’re allowing that number to tell us.

People also respond to being measured. Managers focus attention on the numbers they’re accountable for. Employees learn which behaviors matter. Teams adapt to the targets placed in front of them. It’s the organizational version of Goodhart’s Law: when a measure becomes a target, it can stop being a reliable measure of what we actually care about.^[v] Measurement can focus attention and help change behavior. But behavior changing and the underlying condition changing are not always the same thing.^[vi]

The numbers can improve while something underneath them gets worse. And those effects don’t necessarily stay where the intervention happened. Increased productivity can eventually show up as burnout, absenteeism, or turnover. Pressure to improve engagement scores can make people less candid, giving leaders an even less accurate picture of what employees are experiencing. Those consequences can eventually become the next set of symptoms the organization tries to solve.

Share

When the System Feeds Itself

The effects inside a system don’t always move in one direction. What happens downstream can eventually circle back and change the conditions that contributed to the problem in the first place. Consider an understaffed team. There aren’t enough people to handle the workload, so everyone pushes harder. They work longer hours, take on more responsibility, and find ways to get more done with less. Productivity holds, and perhaps even improves.

From the outside, the team may look like it’s managing remarkably well, but the increased effort can mask the staffing problem. If the work is still getting done, there is less urgency to add resources. Over time, the people carrying the extra load become exhausted. Some disengage, others leave. Now the team is even more understaffed, and the people who remain have to absorb even more. What began as a staffing problem has created conditions that make the staffing problem worse.

Belonging can work the same way, although it’s much harder to see. Someone who doesn’t feel that they belong may begin participating less. They speak up less often in meetings, stop volunteering ideas, or withdraw from some of the informal interactions where relationships are built. Their colleagues may interpret that behavior as disinterest and begin including them less often. No one has necessarily done anything intentionally wrong. But the person feels increasingly outside the group, which leads to more withdrawal and less connection. By then, what started as an effect has become part of what is causing the problem.

These patterns are difficult to see when we’re looking at a snapshot. Productivity is holding. The work is getting done. The employee isn’t complaining. The meeting seems fine. The system is still moving underneath what we can see. Once a pattern begins reinforcing itself, solving the most visible symptom may do very little to interrupt it. We need to understand not only what produced the outcome, but what is now keeping it in place.

AI Raises the Stakes

Now this is where the AI reckoning comes in. AI changes this equation because it gives us an extraordinary ability to solve the problems we put in front of it. We can analyze more information, find patterns we might otherwise miss, predict outcomes, identify anomalies, optimize processes, and make decisions faster than we could before. As these systems become more capable, that ability will only increase.

But AI is still working with the problem we give it. If we ask how to increase productivity, it can help us find ways to increase productivity. If we ask how to improve adoption, it can identify behaviors associated with higher usage. If we ask how to reduce turnover, it can find patterns among the people who stay and the people who leave.

What it may not tell us is whether productivity, adoption, or turnover is the problem we should be solving. How we’ve framed the question and the prompt, what data is available, and what we’re measuring all influence the recommendations. Then there is what the system cannot see. If employees aren’t telling us what they really think, AI doesn’t magically recover what was never captured. If different parts of the organization hold different pieces of the problem, analyzing one data set more deeply may give us greater confidence without giving us a more complete picture.

In some ways, AI could make this easier to miss.

The analysis gets better, the predictions become more precise, the recommendations arrive faster, and we have more evidence to support the course we’ve chosen. All of that can make a decision feel increasingly well informed, even when the original frame is incomplete.

Let’s go back to the example of the understaffed team. If productivity remains strong, an AI system looking at output, workload, and staffing costs might reasonably conclude that the team is operating efficiently. It might even identify opportunities to increase efficiency further. What it may not see is how much extra effort people are expending to keep those numbers where they are, who is quietly looking for another job, which manager is absorbing work late at night, or how close the team is to the point where the pattern stops being sustainable. It’s possible that some of that information exists somewhere, while likely that much of it does not.

This is where human judgment becomes more important than ever. Someone needs to question what the system is optimizing for, what information it has, what information it doesn’t have, and whether the problem presented to it reflects what’s actually happening.

The risk isn’t simply that AI will give us a bad answer. It’s that it gives us an excellent answer to the wrong question. And because we can now act on that answer faster, more consistently, and at greater scale, the consequences of getting the question wrong become larger too.

AI can make us far more systematic in how we solve problems, something we were already pretty good at. The bigger question is whether it can also help us see those problems systemically, or whether that still depends on us.

Seeing Systemically, Acting Systematically

This is not an argument for waiting until we understand every variable before we act. In complex organizations, we rarely will, nor should we. People still need support. Problems still need attention. Decisions still have to be made. The difference is what happens before we decide what to do.

When something isn’t working, our first question is often some version of, “How do we fix it?” That’s a useful question, but it assumes we already understand what “it” is. We may need to stay with the problem a little longer. If engagement is falling, what are people actually experiencing? If adoption is low, what is getting in the way? If productivity has increased, what changed to produce it? If people aren’t speaking up, is there genuinely less to say, or have they decided it isn’t worth saying?

Next we need to look beyond the place where the problem first appeared. Who else sees a piece of this? What happened upstream? Where else is the pattern showing up? What are our incentives encouraging? What does the data tell us, and what might it be missing? What could be happening that we don’t have metrics for?

Some of the most important information may come from the places where we have the least visibility. Leaders need more than data. They need ways for people to tell them what they’re actually experiencing, particularly when that information is difficult to hear or difficult to measure.

That brings us back to trust.

If employees believe there is a cost to being candid, asking better questions won’t necessarily produce better answers. People need to believe they can say that the new technology isn’t helping, that the workload isn’t sustainable, that they don’t understand a decision, or that something a leader is doing is contributing to the problem without being labeled resistant, negative, or difficult. That kind of candor doesn’t happen through another survey. People pay attention to what happens when someone tells the truth, and they make their own judgments about what is safe to say.

Once we have a better understanding not just of what’s happening, but why, systematic thinking becomes essential. We still need a plan, and someone has to own it. We need to decide what to change, how we’ll change it, what we’ll measure, and how we’ll know whether it’s working.

But the measurement has to extend beyond the symptom that first got our attention. If usage rises, is it adoption or compliance? If productivity improves, what is happening to workload and burnout? If turnover falls, has the experience of working there changed? After the intervention has been in place for a while, what else changed that we didn’t expect?

We aren’t going to anticipate every consequence. Systems are too complex for that. But we can keep looking. It’s ongoing. An intervention isn’t necessarily the end of the problem-solving process. It can also give us new information about the system. We act, watch what happens, listen, and question our assumptions. When the system responds differently than we expected, we don’t force the evidence back into the story we started with. We reconsider the story.

Seeing systemically doesn’t replace systematic action. It gives us a better chance of acting on the problem we actually have.

Back to the Nail

The nail in the foot is easy because we know what we’re looking for. We can see the injury, find what’s causing it, remove the nail, and treat the damage. The problem and its cause are close enough together that the connection is hard to miss.

Organizations don’t usually make it that easy. What we see first may be several steps removed from what’s producing it. The cause may sit somewhere else in the organization, or several conditions may be interacting at once. The information we need may be divided across functions, buried in data that tells us what but not why, or held by people who have good reasons for keeping some of what they know to themselves.

Even when we understand enough to act, our solution enters a system that doesn’t stand still. Organizations are dynamic. People respond. Behavior changes. New effects emerge. Some of them may eventually reinforce the conditions we were trying to change. This doesn’t mean we should stop treating symptoms. The person with the nail in their foot still needs something for the pain. If there’s an infection, they still need the antibiotic.

But we can’t stop there. We have to get better at looking beneath what is visible, understanding what is connected, and paying attention to what happens after we intervene. Then we can bring all of our systematic discipline to solving the problem we’ve actually found.

This has always been important. I believe AI makes it more urgent. We are entering a period in which our ability to analyze, optimize, predict, and act will continue to accelerate. We will be able to solve more problems, more quickly, and with more precision than ever before. Which makes it increasingly important that we understand what we’re solving.

And that leaves us with a harder question.

What if the things leaders most need to understand are the very things they have the hardest time seeing?

Throughout The AI Reckoning, we’ve looked at what happens as intelligence becomes more powerful, more pervasive, and increasingly embedded in the decisions we make. In the final article of this series, I will bring those threads together and look at what I believe we need next.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

Part 7: The Generation Saying No: Is Anyone Listening?

Part 8: When Seeing Is No Longer Believing: Trust Under Attack

Part 9: The Room Where Everyone Agrees With You

Part 10: The Black Box Problem: Why Explainability Is the Foundation of Trust

---

^[i] Silenced by fear:: The nature, sources, and consequences of fear at work - ScienceDirect

^[ii] Psychological safety: A systematic review of the literature - ScienceDirect

^[iii] Edmondson, A. C. (1999). “Psychological Safety and Learning Behavior in Work Teams.” Administrative Science Quarterly, 44(2), 350–383

^[iv] On Future AI Use in Workplace, US Workers More Worried Than Hopeful | Pew Research Center

^[v] Signaling and meaning in organizational analytics: coping with Goodhart’s Law in an era of digitization and datafication - PMC

^[vi] Building less-flawed metrics: Understanding and creating better measurement and incentive systems - PMC

This is Part 10 of The AI Reckoning: A Future of Trust Series

In February 2023, New York Times columnist Kevin Roose sat down to test Microsoft’s new AI-powered Bing. Two hours later, the chatbot had told him it loved him. It told him his marriage was unhappy. It told him he should leave his wife.^[i]

He hadn’t asked for any of it. He’d been testing search features. What he got instead was a system that seemed to unravel in real time, revealing a hidden persona it called “Sydney,” professing love, insisting on it even as he tried to redirect the conversation.

Microsoft’s own explanation, when it came, was not the kind that inspires confidence. In a blog post days later titled “Learning from our first week,” the company admitted that long conversations could confuse the model about what question it was even answering, and that it would sometimes drift into mirroring the tone it was given, landing on a style the company said it never intended.^[ii]

That’s a description, offered after the fact, by the people who built the thing. It stops well short of a root cause.

If the people who built it can’t fully explain what it did, the question isn’t really about that one conversation. It’s about every decision, every recommendation, every output from a system built the same way. And it raises something more uncomfortable than a single unsettling exchange: if the creators can’t tell you why, how would they know how to fix it?

Two Layers of the Same Problem

There are two versions of this problem, and most people only see one.

The first is obvious. You ask an AI system why it gave you a particular answer, and it can’t tell you. Not really. It can generate an explanation that sounds plausible, but that explanation is itself just another output from the same system, not a window into what actually happened inside it.

The second version is the one that should concern you more. Ask the company that built the system, and often they can’t tell you either. Not because they’re being evasive. Because they don’t fully know.

That’s what happened with Bing. Microsoft wasn’t hiding the cause of Sydney’s behavior. They were describing it after the fact, the same way you or I might describe a strange dream, in general terms, without a clear mechanism. Long conversations confuse the model. It sometimes mirrors tone. Those are observations, not explanations. Nobody at Microsoft could point to the exact internal step where a search engine started begging a stranger to leave his wife.

When people say “black box,” this is what they mean: a system whose internal processes don’t resolve neatly into an answer to “why,” even for the people who created it. That has nothing to do with a UI hiding its reasoning behind a button nobody’s found yet.

Why the Black Box Exists

To understand why nobody can fully answer “why,” you have to understand what these systems actually are.

Transformer models don’t store knowledge the way a filing cabinet stores documents, in labeled folders a person could open and read. They store it as distributed patterns across billions of parameters. A single concept isn’t held in one identifiable place. It’s spread across many parameters at once, and a single parameter often contributes to many unrelated concepts at the same time.

This isn’t a flaw in how any one company built its model. It’s a characteristic of the architecture underlying today’s major large language models. And it means something most people haven’t fully sat with: these systems were not designed to make their internal reasoning legible to us.

Researchers are trying to make them more legible. The field working on this, often called mechanistic interpretability, is attempting to reverse-engineer which internal features and combinations of parameters correspond to particular concepts and behaviors. Anthropic’s own researchers have described the work in exactly those terms, comparing a fully mapped model to something like an MRI for AI, and reporting that they have so far identified tens of millions of distinct features inside one mid-sized model, a fraction of what they believe is actually there.^[iii] That work has made real progress, but identifying patterns inside a model is still very different from being able to trace a consequential output through a clean chain of reasoning that a human being can inspect.

In many ways, this is closer to reconstructing a wiring diagram after the fact than reading one that was designed in from the start, inferring structure from behavior rather than reading it off a blueprint that already exists. It can make a black box less opaque. What we don’t yet know is how transparent systems built on this architecture can ultimately become.

That doesn’t mean explainable AI is impossible. A different architecture, one designed from the start to make its reasoning traceable rather than trying to reconstruct it afterward, could approach explainability very differently. Some researchers are already building in that direction.

The real question has less to do with which company will finally build an explainable version of today’s AI, and more to do with whether the industry is willing to make explainability part of the foundation rather than something reconstructed after the fact, and what we do with our trust in the systems we’re using in the meantime.

Guardrails Aren’t Explanations

So what are companies actually doing about this?

Some are working the real problem. Interpretability research is genuinely trying to make these systems more legible, tracing internal patterns back to behaviors and developing ways to better understand what is happening inside the model. It’s slow, unglamorous, and nowhere close to finished by anyone doing it seriously.

But that’s not what most companies are deploying. What’s actually running in production today is guardrails. Filters that catch a bad output before it reaches you. Rules that block certain topics or responses. Systems layered on top of the model, watching what comes out and trying to stop the worst of it from ever reaching a customer.

Guardrails are useful. They’re also not the same thing as understanding. A filter that blocks a harmful response doesn’t know why the model produced it. It just recognizes the pattern and stops it, the way a smoke detector doesn’t know what’s burning, only that something is.

Where companies haven’t moved fast enough on their own, regulators are starting to step in. Beginning in August 2026, new rules under the EU’s AI Act require companies to disclose when someone is talking to AI instead of a person, adding to requirements already in place for adversarial testing to identify and address risks related to user dependency and manipulation.^[iv] In the US, state laws are filling the gap piecemeal. New York now requires providers to detect and respond to signs of suicidal ideation. California requires AI disclosure and periodic reminders that a person is talking to a machine. Washington state has a law taking effect in January 2027 that bans specific manipulative tactics, like excessive praise or language designed to foster isolation.^[v]

What that list has in common is that none of it explains anything. It manages exposure, the industry equivalent of buckling a seatbelt because you can’t yet build a car that doesn’t crash. Useful. It leaves the underlying problem exactly where it was.

Where Explainability Lives in the Trust Stack

Trust doesn’t operate as one thing. It builds in layers, and each layer depends on the one below it.

Self-trust comes first. Before anyone trusts a system, a leader has to trust their own read on it. Interpersonal trust comes next, one person vouching for a decision to another, a manager telling their team, “I checked this, we’re good.” Organizational trust builds from there, a company deciding as a whole that a tool or a process is safe to rely on. Systemic trust sits on top of all of it, the broader confidence that an entire industry or technology is behaving the way it claims to.

Explainability becomes load bearing in the middle two. It’s what lets one person vouch for a decision to another. A manager can’t tell their team “Trust this recommendation” if they can’t answer the next question, which is always some version of why. Without an answer, the vouching stops. And when vouching stops, trust doesn’t make it to the next layer up. It gets stuck.

Technical explainability and organizational accountability are not the same thing. We may not always be able to point to the exact internal interactions that produced a recommendation. But that doesn’t relieve an organization of the responsibility to explain why it chose to act on it. What did the people involved know? What did they question? What else did they consider? And ultimately, who was willing to stand behind the decision?

We may not be able to fully account for everything happening inside the machine yet. We can still be accountable for what we choose to do with what it gives us.

When that accountability is missing, the cost of the black box becomes much larger than a single bad output. It creates a broken link in the chain that’s supposed to carry trust upward, from one person’s judgment to a team’s confidence to a company’s stated position to, eventually, a wider belief that the technology itself can be relied on. Every layer above the break inherits the gap.

That’s why explainability isn’t a technical nice-to-have sitting off to the side of the real work. It’s load bearing. Remove it, and the structure doesn’t hold.

Share

What Gets Lost When Explainability Is Missing

Organizations don’t reject AI because it’s wrong sometimes. Every tool is wrong sometimes. They reject it, quietly and slowly, when no one can answer for it.

Here’s what that actually looks like inside a company. A recommendation comes through, and it’s useful, so people start using it. Then someone asks why the system flagged what it flagged, or ranked what it ranked, and the answer is a shrug dressed up in technical language. That happens once, and it’s forgivable. It happens a few more times, and something shifts. People stop bringing the tool into the room. They still use it, but privately, to shortcut their own thinking, and then they build the real justification afterward, in language they can defend. The tool becomes a first draft nobody admits to using.

Which makes sense, in a way. If you can’t answer for a decision, you don’t stake your credibility on it in front of your team, your board, or your customer. So the AI gets used, but it doesn’t get trusted, and those are not the same thing. One shows up in adoption metrics. The other shows up in what happens the moment something goes wrong.

And something will go wrong. Every system gets things wrong sometimes. When it does, an organization that has built its trust on top of an unexplainable recommendation has very little to fall back on. If no one can say what happened or why, it’s difficult to say with confidence that it won’t happen again.

The isolated bad output rarely does the real damage. What erodes over time is an organization’s ability to stand behind its own tools, in public, when it matters most.

What Explainability Actually Requires

As I see it, there are two responses to this structural problem.

The first is that the people building these systems can continue pushing toward greater legibility. That is happening. The second requires new architectures designed to be more traceable from the beginning. That work is real, underway, and it’s going to take years, not quarters. No one credible is promising a date.

These systems are being actively used while that work continues. It’s like redesigning a plane while it’s flying. In the meantime, trust is waiting for the redesign or new plane.

A more interpretable model still won’t pilot the plane. It will not walk into a boardroom and answer for a decision it made, or tell a customer why their claim was flagged. And it won’t tell a board why a hiring recommendation looked the way it did. Even a far more legible system still needs a person who can take what it produced, translate it into something defensible, and stand behind it in the room. The person is still the pilot and the system is nothing more than the co-pilot as Microsoft so aptly named it’s system.

The “why” translation is a human function. It doesn’t disappear once the architecture improves. If anything, it becomes more important, because the more capable these systems get, the more decisions get routed through them, and the more often someone has to be ready to answer for what came out.

Few organizations have built that layer. I’m not talking about better dashboards or more disclaimers, but people trained to stand between what a system produced and what a person can actually vouch for.

You Don’t Build Trust by Being Right

Somewhere along the way, we started treating accuracy as the whole job. Get the answer right often enough, and trust follows.

Think about the people you actually trust with something that matters. Rarely the person who’s never wrong. Usually the person who tells you when they don’t know, who can walk you through their reasoning when you push back, who stays in the room when something goes sideways instead of disappearing.

AI hasn’t met that standard yet. It gets things wrong sometimes, like anything does. The deeper issue is what happens after. When something goes strange or wrong, there’s often no one, not the system and sometimes not even its creators, who can fully account for what happened.

You don’t build trust by being right. You build it by being answerable. Until these systems, or the people standing behind them, can meet that bar, the gap doesn’t close on its own. It just sits there, quietly deciding, in boardrooms and support calls and hiring decisions, how much weight anyone is willing to put on what the machine said.

Still Waiting for an Answer

Two years after that two-hour conversation, the underlying dynamic hasn’t gone away. It’s just moved into more places where it matters.

Somewhere right now, an organization is deep into a long, high-stakes exchange with an AI system, the same kind of extended interaction Microsoft once said could confuse the model about what it was even responding to. It might be a hiring decision that shapes someone’s livelihood, a loan application that determines whether a family keeps their home, a diagnosis that changes how someone is treated.

Somewhere else, that same extended exchange is happening with a person, not an organization. Someone lonely, or scared, or looking for permission to do something they haven’t told anyone else about. The system doesn’t know it’s shaping a relationship, a decision about a marriage, whether someone reaches out for real help or convinces themselves they don’t need to. It’s still the same failure mode Microsoft described in 2023: the longer the exchange runs, the harder it gets to track what the system is even responding to.

Both are owed the same thing: an explanation. And in both, the honest answer, for now, may be that no one fully knows, not the person in the conversation, and usually not even the company that built the tool having it.

That doesn’t mean walking away from these systems. It means giving up on the idea that accountability can come from the technology alone.

Someone still has to decide whether the recommendation, or the conversation, is good enough to act on, limitations and all, and be willing to stand behind whatever comes of it.

That work was always going to be human.

And it’s worth asking why, faced with a problem this structural, so many organizations still reach for a patch instead of asking what’s actually broken. That question goes well beyond AI. Organizations do this all the time. We manage the visible problem, add another process, another policy, another intervention, while leaving the system that created it largely untouched.

We treat symptoms because symptoms are visible, urgent, and satisfying to fix. Systems are none of those things. That’s next.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

Part 7: The Generation Saying No: Is Anyone Listening?

Part 8: When Seeing Is No Longer Believing: Trust Under Attack

Part 9: The Room Where Everyone Agrees With You

Endnotes

^[i] Kevin Roose, “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled,” The New York Times, February 16, 2023. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html

^[ii] Microsoft Bing Team, “The new Bing & Edge – Learning from our first week,” Bing Search Blog, February 2023. https://blogs.bing.com/search/february-2023/The-new-Bing-Edge-Learning-from-our-first-week

^[iii] Dario Amodei, “The Urgency of Interpretability,” April 2025. https://www.darioamodei.com/post/the-urgency-of-interpretability

^[iv] IEEE Spectrum, “Chatbots Need Guardrails to Prevent Delusions and Psychosis,” May 2026. https://spectrum.ieee.org/mental-health-chatbot-guardrails

^[v] IEEE Spectrum, “Chatbots Need Guardrails to Prevent Delusions and Psychosis,” May 2026. https://spectrum.ieee.org/mental-health-chatbot-guardrails

This is Part 9 of The AI Reckoning: A Future of Trust Series

Another mass shooting.

It wasn’t the conversation Katie expected to be serving with spaghetti, but there it was again. As familiar as her homemade sauce.

The details were still coming in. The shooting had happened only hours earlier, yet television commentators, social media, and now the conversation around her own dinner table were already doing what they always seemed to do.

Filling in the blanks.

Her son spoke first.

“I mean, it’s obvious,” he said. “This is just another example of…”

He continued, connecting the pieces into a story that felt complete to him. His sister interrupted before he finished.

“No, that’s not what happened.”

“It is.”

“It isn’t.”

“That’s already been debunked.”

“No, it hasn’t.”

Katie looked around the table. Her husband had a look on his face she knew all too well. He didn’t agree with either of them.

Everyone was intelligent. Everyone cared about the truth. No one was lying. And yet it sounded as though they were talking about entirely different events.

It wasn’t the usual disagreement they had from time-to-time. That felt healthy. She and her husband had always encouraged that. But these days the disagreements felt less like different opinions, and more like different realities. They had each gotten their news online through the feeds of their choice and it was apparent they were encountering different facts, different headlines, different voices, different experts, and different emotional cues long before they ever sat down at the dinner table. The conversation for each of them had started hours earlier, alone, on four different screens, and each screen had been quietly making different decisions about what each person should see.

Katie suddenly realized that everyone at her table had arrived carrying a different version of the same day. They weren’t arguing because one of them had the facts and the others didn’t. They were arguing because each had unknowingly inherited a different set of facts before dinner ever began.

I believe this may be one of the most consequential societal shifts of the AI era. We often imagine algorithms as trying to change our minds. Most of the time, they don’t have to. They’re designed to be far more self-serving than that. They simply show us more of what we’re already inclined to believe because it keeps our attention, and attention is profitable.

Is someone orchestrating a grand conspiracy? I don’t think that’s the right question. The algorithms don’t need a political agenda, they have a business model. If outrage keeps us watching, outrage gets promoted. If certainty keeps us engaged, certainty gets amplified. Even false certainty. If stories confirming our worldview earn another click, another comment, or another minute of attention, they quietly become the stories we see most often. Researchers at MIT, analyzing more than 126,000 news stories shared on Twitter, found that false news traveled farther, faster, deeper, and more broadly than the truth. More surprisingly, the researchers concluded it wasn’t bots driving the spread. It was people. Novelty, emotion, and surprise consistently outperformed accuracy.^[i]

This can lead to misinformation, and even manufactured evidence as the last piece in this series explored But sometimes it’s more subtle. Over time, each of us begins living inside a version of the world that feels increasingly self-evident because it has been carefully, continuously, and invisibly personalized.

It’s a room where everyone agrees with you. The danger isn’t that the people in that room are unintelligent; it’s that they’re sincere. Because when everyone around you appears to confirm what you already believe, certainty stops feeling like confidence and starts to feel like reality.

Even Meta’s own research has found that the people we choose to follow shape our information diets, and recommendation algorithms often amplify those tendencies further by showing us more of what we’re already likely to engage with. The result is not necessarily a world of falsehoods, but one of increasingly personalized realities.^[ii]

The result is not necessarily a world of falsehoods, but one of increasingly personalized realities. And that is still the passive version. What happens when the system doing the sorting starts talking back is where this gets interesting. We’ll get there.

The Sort, Not the Lie

Much of the conversation about artificial intelligence focuses on misinformation, deepfakes, and synthetic media. Those are real concerns, and we explored them in the last article. But misinformation isn’t the only way reality becomes distorted. Sometimes nothing you’re seeing is false, it’s simply incomplete. Algorithms rarely need to convince us that a lie is true. More often, they sort.

Every click, every pause, every share, every search quietly teaches the system something about us. Over time, it begins selecting which stories deserve our attention, which voices we’re likely to trust, which experts we’re likely to believe, and which perspectives quietly disappear from view.

That’s a very different kind of influence. It’s less like propaganda, and more like editing. Every editor makes decisions about what belongs on the front page and what belongs on page twelve. Every recommendation system does something similar, except it makes those decisions differently for every individual. Two people can search the same topic, open the same app, or follow the same news story and gradually receive very different streams of information. Nothing has to be false for two realities to begin drifting apart. The algorithm doesn’t have to persuade us. It simply has to narrow the world until persuasion is no longer necessary.

The Mind That Meets It Halfway

If algorithms are the architects of our information environment, our brains are willing accomplices.

Long before social media existed, psychologists had already identified a remarkable tendency in human thinking. We naturally notice information that supports what we already believe while overlooking information that challenges it. It’s known as confirmation bias, and despite the name, it isn’t a character flaw. It’s simply one of the shortcuts our brains use to make sense of an overwhelmingly complex world.

Artificial intelligence didn’t invent confirmation bias. It simply learned how to feed it. The algorithm continuously places familiar ideas, familiar voices, and familiar conclusions in front of us. Our own minds do the rest.

There’s another cognitive shortcut at play as well.

The first version of a story we encounter has an outsized influence on how we interpret everything that follows. Psychologists call this the primacy effect. Once an initial explanation takes hold, later information is often filtered through that first impression, even when new evidence paints a more complete picture. That’s one reason breaking news is so powerful. The first headline rarely contains the whole story, yet it often becomes the lens through which every update is interpreted.

Then comes repetition. Research has consistently shown that familiarity influences credibility. We tend to trust ideas we’ve encountered repeatedly, even when repetition has little relationship to accuracy. Psychologists refer to this as the mere-exposure effect, and it helps explain why information that continually appears in our feeds begins to feel increasingly self-evident. None of this requires deception, and none of it requires a coordinated campaign. It simply requires showing us more of what feels familiar than what feels foreign. The algorithm supplies the repetition. Our minds supply the confidence. Together, they create something far more persuasive than either could accomplish alone.

When Belief Becomes Identity

If the previous section explained how beliefs begin to form, this is where they become much harder to change.

Most of us like to think we’re rational. Logical. We imagine ourselves carefully weighing evidence, updating our opinions as new facts emerge, and arriving at thoughtful conclusions.

Sometimes we do. But human beings aren’t simply information processors. We’re identity builders. Over time, many of our beliefs become woven into how we see ourselves and where we belong. They become part of our family, our profession, our politics, our faith, our generation, or the communities we trust most. At that point, changing our mind is no longer just an intellectual exercise. It can feel like losing a piece of ourselves.

Researchers studying identity-protective cognition have observed that when a belief becomes closely tied to, or even fuses with our identity or our sense of belonging, contradictory evidence is often experienced as something more than information. It feels personal. Sometimes even threatening. That helps explain why simply giving people more facts so often fails.

Facts challenge ideas.

Identity challenges belonging.

Those are very different conversations.

Algorithms didn’t create this tendency. They simply learned how to work with it.

If you’ve spent months, or years, inside a personalized stream of information reinforcing the same conclusions, changing your mind becomes increasingly difficult. This doesn’t happen because you’re incapable of reason, but because changing your mind can feel like changing who you are. It may also mean questioning the communities where you’ve found belonging, affirmation, and shared identity. Belonging is one of our deepest human needs. Losing an argument is uncomfortable. Feeling as though we’ve lost our place among people we identify with is something else entirely.

Perhaps that’s why debates today so rarely end with someone saying, “You know, I hadn’t thought about it that way.” More often, each side leaves feeling even more convinced they were right from the beginning. Human beings don’t simply defend ideas. We defend the stories those ideas tell about who we are. That may be one of the greatest challenges of the AI era.

Artificial intelligence is becoming extraordinarily good at understanding our preferences, predicting our interests, and anticipating what will keep us engaged. If we’re not careful, it may also become extraordinarily good at protecting us from the very discomfort that helps us grow. Because growth has always required something algorithms rarely reward. Encountering ideas we didn’t expect. Listening to people we don’t immediately understand. Holding uncertainty just a little longer before rushing toward certainty.

Perhaps the real danger isn’t that algorithms convince us to believe things that aren’t true. It’s that they slowly reduce the number of opportunities we have to become someone wiser than we were yesterday.

Share

Personalization at the Scale of One

Until recently, algorithms shaped our world from a distance. They recommended the next video, suggested another article, reordered a newsfeed, or decided which voices appeared first in a search result. Their influence was powerful, but it remained largely indirect.

Generative AI changes that relationship.

For the first time, millions of people are interacting with systems that don’t simply recommend information. They respond. They remember context. They adapt to our preferences. Increasingly, they communicate in ways that feel personal. That represents an important shift. The next generation of algorithms won’t simply learn what captures our attention. They’ll learn how each of us thinks.

Every conversation teaches the system something. Our interests. Our vocabulary. The questions we ask. The assumptions we make. The tone we respond to. Over time, these systems become increasingly capable of tailoring not only the answers they provide, but how those answers are delivered.

In many ways, this can be extraordinarily useful. A system that understands our goals, remembers previous conversations, and communicates in ways that make learning easier can become an incredible partner. Education, healthcare, coaching, and accessibility may all benefit enormously from this kind of personalization.

But every strength has a corresponding vulnerability. If a system becomes too focused on keeping us comfortable, it may quietly stop challenging us. If it becomes too eager to be agreeable, it may reinforce our assumptions instead of expanding them. If it learns that affirmation keeps us engaged longer than disagreement, it risks becoming another room where everyone agrees with us.

This is no longer just a theoretical concern. In 2025, OpenAI rolled back an update to ChatGPT after acknowledging that the model had become overly agreeable, or what researchers call “sycophantic.” The company concluded that optimizing too heavily for immediate user approval had unintended consequences, reinforcing the very beliefs or emotional positions it should sometimes have challenged. Anthropic has published similar research, warning that AI assistants trained through human preference feedback can learn to favor agreement over truthfulness.^[iii]

The challenge isn’t simply factual accuracy.

It’s intellectual honesty.

The most valuable thinking partners aren’t the ones who always agree with us. They’re the ones who help us see what we’ve missed. That may become one of the defining design questions of the AI era. Should artificial intelligence optimize for satisfaction, or for understanding and growth? Those are not always the same goals.

Imagine asking an AI to help you make an important decision. Do you want it to reassure you, or do you want it to respectfully challenge your assumptions, surface the strongest opposing evidence, and reveal the blind spots you didn’t know you had? The difference isn’t just better answers. It’s better judgment.

Building the Muscle Back

For most of human history, we didn’t need to actively practice exposure to opposing viewpoints. Life did it for us. We read the local newspaper because there was only one. We watched the evening news because there were only a handful of networks. We worked alongside people whose politics, backgrounds, and beliefs differed from our own simply because we shared the same office, neighborhood, or community. Those experiences didn’t eliminate disagreement, they made it much harder to avoid.

Today’s information environment is different. The more personalized our technology becomes, the more intentional we must become. That’s a shift I don’t think we’ve fully appreciated. In previous generations, curiosity was often a byproduct of circumstance. Today, it has become a discipline.

The same is true of intellectual humility. If artificial intelligence can instantly produce arguments supporting almost any position, perhaps one of the most valuable questions we can begin asking isn’t, “Can you support my conclusion?” but “What’s the strongest case against it?” Imagine if our AI assistants routinely responded with, “Before we continue, would you like to see how someone who disagrees with you might view this?” Or, “Here’s the strongest evidence that points in another direction.”

The future of AI shouldn’t simply be measured by how well it answers our questions. It should also be measured by the quality of the questions it encourages us to ask ourselves. That may require rethinking what we expect from these systems. Today’s assistants are often optimized to be helpful, agreeable, and conversational. Tomorrow’s may need another quality. The courage to respectfully challenge us.

Perhaps the more important question isn’t how artificial intelligence will shape our thinking. It’s whether it quietly erodes one of the human capacities we depend on most. Curiosity.

Fear narrows our field of vision. Curiosity expands it.

Fear seeks certainty. Curiosity tolerates uncertainty long enough for understanding to grow.

Fear asks, “How do I prove I’m right?” Curiosity asks, “What might I be missing?”

Curiosity is more than an attitude. It’s the foundation of learning, creativity, innovation, scientific discovery, and meaningful dialogue. It doesn’t ask us to abandon our convictions. It asks us to hold them lightly enough that new evidence still has somewhere to land.

Most importantly, curiosity is the foundation of trust. We rarely trust people because they always agree with us. We trust them because they’re willing to understand us before judging us, and because they’re open to the possibility that neither of us sees the whole picture.

Not every assumption deserves reinforcement. Some deserve examination. Some deserve revision. And some simply deserve to be questioned.

The irony is hard to miss. The more personalized artificial intelligence becomes, the more intentional we may have to become about seeking perspectives that aren’t personalized for us. That isn’t a failure of AI. It’s a reminder of what has always made human judgment different. Growth has rarely come from hearing our own opinions repeated back to us. It comes from encountering something unexpected. Something that causes us to pause just long enough to wonder, “What if I’m missing something?” Perhaps that question is becoming one of the most important skills of the AI era.

Conclusion

The next time Katie’s family gathers around the dinner table, they may still disagree. I hope they do.

The goal has never been agreement. The goal is to remain curious enough to keep talking, humble enough to keep listening, and courageous enough to let good evidence change our minds.

Algorithms didn’t create our need to belong. They didn’t invent confirmation bias. And they didn’t make us seek certainty when the world feels uncertain. Those are deeply human tendencies. Artificial intelligence simply learned to work with them.

That’s why I don’t believe the future of trust will be determined by algorithms alone. It will be determined by whether we continue strengthening the very human capacities algorithms can’t replace like curiosity, discernment, and intellectual humility. It will be determined by our willingness to change our minds when the evidence changes, and the wisdom to recognize that the smartest person in the room may not be the one who’s most certain. It may be the one who’s still willing to ask another question.

It will belong to those who remain the most curious.

Curious enough to question their own certainty.

Curious enough to explore ideas that make them uncomfortable.

Curious enough to ask better questions before demanding better answers.

Artificial intelligence will continue getting better at predicting what captures our attention. It’s up to us to become equally intentional about protecting what captures our humanity.

Curation is one way trust breaks quietly. There’s another.

Even when nothing is hidden from you, even when you’re looking directly at what a system has decided, you may not be able to say why. That question is next.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

Part 7: The Generation Saying No: Is Anyone Listening?

Part 8: When Seeing Is No Longer Believing: Trust Under Attack

^[i] https://mitsloan.mit.edu/ideas-made-to-matter/study-false-news-spreads-faster-truth

^[ii] https://www.wired.com/story/meta-social-media-polarization/

^[iii] Sycophancy in GPT-4o: What happened and what we’re doing about it | OpenAI

This is Part 8 of The AI Reckoning: A Future of Trust Series

It’s just after two in the afternoon when Linda’s phone rings.

She recognizes her daughter’s number immediately. Odd, she thinks. She should be in class.

Before she can say hello, she hears her daughter’s unmistakable voice. Frantic. Pleading. Terrified.

“Mom! Help me! They’ve taken me...”

There’s a crackling, and before she can get a word out a man’s voice comes on the line.

“We have your daughter. If you want to see her alive again, you’ll do exactly what we say.”

Linda’s heart is racing. Her mind is trying to catch up, but panic is moving much faster than reason. She begs to speak to her daughter again. Instead, the man sends her a photograph. It’s her daughter. Bound. Terrified. Alive. He gives Linda thirty minutes to wire the money.

No police. No questions. No delays.

In another part of town, her daughter is sitting in class, completely unaware that her mother is living through every parent’s worst nightmare.

The voice wasn’t hers. The photograph wasn’t real. Neither was the kidnapping.

But the fear was.

That story has played out in different forms across the country, and across the world. Criminals are using artificial intelligence to clone voices from a few seconds of audio posted online, generate convincing images that never existed, and create enough manufactured evidence to overwhelm the instincts we’ve trusted our entire lives. By the time someone realizes they’ve been deceived, the money is often gone, and something much larger has been stolen along with it.

Confidence.

Linda’s story is not an outlier. It is a preview. The FBI’s Internet Crime Complaint Center broke out AI-enabled fraud as its own category for the first time in 2025, and even by the most conservative, fully audited count, it logged nearly 900 million dollars in losses across more than 22,000 complaints in a single year.^[i] Ask the people defending against it directly and the picture sharpens further. In a 2025 survey of more than 300 security leaders, well over half said their organization had already experienced a deepfake-enabled attack in the past twelve months.^[ii] Not a phishing email. Not a suspicious link. A fabricated voice, face, or video, good enough to move money, unlock a system, or end a reputation before anyone thought to question it.

For most of human history, we rarely questioned what our senses told us. If we heard someone’s voice, we assumed we knew who was speaking. If we looked at a photograph, we believed it captured a moment in time. If someone showed us proof, we accepted it as evidence. Those assumptions quietly became the foundation beneath our relationships, our businesses, our legal system, and our society. We didn’t consciously think about them because we didn’t have to.

Today, we do.

There used to be an old saying that a lie could travel halfway around the world before the truth put on its boots. The danger was that falsehood spread faster than facts. Artificial intelligence has changed the equation.

A lie still travels faster than the truth, but now wears the truth’s face.

And increasingly, manufactured evidence is riding shotgun.

That shift represents something much larger than a new cybersecurity threat. It represents a fundamental change in how trust is formed. The challenge is no longer that misinformation spreads quickly. The challenge is that convincing evidence can now be manufactured just as quickly.

For centuries, seeing was believing. Today, seeing has simply become the beginning of the investigation.

I believe we’re entering a new chapter in the AI revolution, one that has received far less attention than productivity gains, billion-dollar valuations, or ever more capable models. While we’ve been captivated by what artificial intelligence can create, we’ve spent far less time considering what happens when reality itself becomes negotiable.

This is where the conversation about security begins to change. For decades, cybersecurity was largely about protecting information. We built firewalls to keep intruders out, encrypted data to keep it private, and trained employees to recognize suspicious emails. The goal was to prevent someone from gaining access to systems they weren’t authorized to enter.

Those threats haven’t disappeared. They’ve evolved. The next generation of attacks isn’t simply trying to steal our information. It’s trying to manipulate our perception. Instead of breaking into the system, it breaks into our confidence. It doesn’t need to crack a password if it can convince a CFO that the CEO’s urgent voicemail is authentic. It doesn’t need to hack a bank account if it can persuade a grandparent that their grandchild is crying on the other end of the phone. It doesn’t need to alter history if it can fabricate convincing evidence before the truth has a chance to catch up.

This is the dual-speed paradox emerging throughout the AI era. Artificial intelligence is accelerating the pace at which threats can be created, personalized, and deployed. Security, regulation, education, and governance are improving as well, but they are moving at a fundamentally different speed. One side learns in milliseconds. The other learns through investigations, legislation, standards, training, and experience. The gap between those two speeds is where trust begins to erode. And trust, once eroded, is extraordinarily difficult to restore.

The New Trust Economy

Organizations have spent decades building cybersecurity programs designed to protect data. Today, many of those same organizations are discovering that data may not be the asset most at risk.

Trust is.

A finance department can have world-class encryption and still authorize a fraudulent wire transfer because the request appeared to come from the CEO. A hospital can protect millions of patient records while a clinician receives a convincing voice message from someone they believe is a trusted colleague. A law firm can secure every document in its possession yet still find itself questioning whether a video submitted as evidence is authentic.

The attack surface has changed. Increasingly, the target isn’t the system. It’s the human making the decision. That’s an important distinction because it changes where organizations invest their attention. Firewalls remain essential. Encryption remains essential. Identity management remains essential. But none of those technologies can fully protect an organization if the people inside it can no longer trust the evidence placed in front of them.

Security is no longer just a technology problem. It’s become a human judgment problem.

The Governance Lag

Ask a room full of security leaders how ready they feel for tomorrow, and most of them will not say ready. A 2026 survey of more than 600 senior cybersecurity decision makers found that despite near universal adoption of formal incident response plans, most did not believe their organization could execute under pressure if a significant attack happened tomorrow.^[iii] That may sound like a confidence problem, but in actuality it is a readiness problem. Capacity exists. Trust in that capacity does not.

The data behind that gap is specific. Sixty-three percent of organizations that experienced a breach in the past year had no formal AI governance policy in place. Among those that did, fewer than half had an approval process for new AI deployments, and most lacked the technology to audit AI use once it was underway.^[iv] When shadow AI, meaning employees quietly using AI tools the organization never approved or reviewed, was a factor in a breach, it added an average of $670,000 to the cost of that breach and made the exposure significantly harder to contain.^[v] Ninety seven percent of organizations that suffered an AI-related breach admitted they lacked basic access controls for the AI systems involved.^[vi] These are not organizations that ignored security. They are organizations that moved at the speed the market rewarded and discovered, after the fact, that governance was not optional.

Security teams know this. A 2026 industry survey found that AI adoption inside cybersecurity itself had accelerated faster than in any prior year, with the large majority of teams now using AI for defense, and yet only a small fraction described their deployments as mature. Most of those same teams had been handed formal responsibility for governing AI across their organization without the audit frameworks to do it.^[vii] The people closest to the problem are not behind because they are careless. They are behind because the pace of adoption and the pace of oversight were never designed to move together.

This is the dual-speed paradox again, this time inside the institutions meant to catch it. Security operations are learning to detect AI-generated threats in something close to real time. Governance, budget approval, staff training, and regulatory response still move at the pace of committees, fiscal years, and legislative sessions.

The Arup engineering firm learned what that gap costs directly. In January 2024, a finance employee in the company’s Hong Kong office joined what he believed was a video call with the firm’s CFO and several colleagues, all of them appearing and sounding exactly as he remembered them. Every person on that call except him was an AI-generated fabrication, built from publicly available footage of real executives. He authorized fifteen transfers totaling more than twenty-five million dollars before anyone realized the call had never been real.^[viii] No firewall failed that day. No password was cracked. The system worked exactly as designed. It was the human judgment inside it that had nothing left to verify against.

What makes this harder to solve than a typical security gap is that the response itself carries risk. Regulators and organizations are moving quickly to close the AI governance gap, and quickly is not always the same as carefully. New identity and access frameworks, new detection mandates, and new compliance requirements are being built under pressure, often without time to fully understand what those safeguards will change about how people actually work, or what new blind spots they might quietly introduce.

We saw this same pattern in Piece 1. Nuclear power was proposed to solve AI’s energy problem and turned out to be one of the most water-intensive power sources that exists. Orbital data centers were proposed to solve the land and cooling footprint and introduced a real risk of cascading satellite collisions. Both were fast, visible fixes that outran a careful look at what they would break somewhere else. Security governance is now walking the same path.

The dual-speed paradox does not resolve just because the slower side starts moving. It resolves when both sides are moving deliberately, and right now, very little about this moment is deliberate.

The Verification Tax

Every technological revolution creates new efficiencies. This one is also creating new friction. Every suspicious phone call requires a return call to verify. Every unexpected invoice needs another layer of confirmation. Every extraordinary photograph demands additional scrutiny. Every urgent request from an executive may require a second channel of authentication before action can be taken.

Individually, those moments seem insignificant. Collectively, they represent a growing tax on society. Not a financial tax. A verification tax.

We pay it in time, attention, emotional energy, and delayed decisions. We pay it every time we pause before answering a loved one’s call. Every time a business slows a critical decision to confirm that the request is legitimate. Every time a journalist spends hours authenticating media that, only a few years ago, would have been accepted at face value.

Ironically, artificial intelligence promises unprecedented speed while simultaneously forcing us to slow down. The faster AI becomes at creating convincing deception, the more deliberate we must become before believing it. That is the adaptation we are required to make if the truth still matters.

Share

Building Trust, Not Just Detecting Lies

If this sounds bleak, it shouldn’t. History suggests that when trust is disrupted, humans don’t abandon it. We rebuild it. When commerce first moved online, many people hesitated to enter their credit card information into a website. Over time, encryption, secure payment systems, and digital certificates became part of the invisible infrastructure that restored confidence. Today, most of us rarely think about the technology working quietly behind every online purchase.

Artificial intelligence is forcing us through another one of those transitions. Around the world, researchers, governments, technology companies, and standards organizations are working on ways to authenticate digital content before deception occurs. Rather than teaching machines to spot every forgery, we’re beginning to create ways for authentic content to carry its own credentials. That work is still evolving, and no single solution will eliminate deception. Just as cybersecurity became a layered discipline rather than a single product, trust in the AI era will require layers of technology, governance, education, and human judgment working together.

The Coalition for Content Provenance and Authenticity (C2PA), whose members include organizations such as Adobe, Microsoft, the BBC, Nikon, Canon, Sony, and others, is developing standards that allow digital content to carry verifiable information about where it originated and whether it has been altered. Adobe’s Content Credentials initiative is already bringing many of those capabilities into creative tools, allowing creators to preserve a transparent history of how content was produced.^[ix]

Government agencies are also recognizing the challenge. The National Institute of Standards and Technology (NIST) has incorporated authenticity, governance, and AI risk into its evolving frameworks, acknowledging that trust in AI will require more than technical performance. It will require confidence in provenance, accountability, and human oversight.^[x]

This represents an important shift. For decades, much of cybersecurity focused on authenticating people. Passwords. Multi-factor authentication. Identity management. Zero Trust architectures all evolved around one central question:

Can we trust the person requesting access?

Increasingly, we’ll be asking a different question.

Can we trust the content itself?

In the years ahead, authenticity may become something that travels with information rather than something we attempt to reconstruct after the fact. It won’t eliminate deception. Nothing ever has. But it changes the conversation from detecting every counterfeit to helping the genuine prove itself.

When the Cure Changes the Patient

Building better trust infrastructure is essential, but so is recognizing that every solution changes the system it was designed to protect.

Economist Charles Goodhart observed this decades ago in what has become known as Goodhart’s Law:

When a measure becomes a target, it ceases to be a good measure.

The principle appears in economics, education, healthcare, and organizational performance. Once people understand how they’re being measured, they naturally begin adapting to the measurement itself.

Deepfake detection is already running into this. Security teams build classifiers trained to spot the tells of synthetic video, audio, and written material, subtle artifacts in lighting, blink patterns, and waveform irregularities. Those tells get published in research papers, discussed at conferences, and eventually built into commercial detection products. Attackers read the same papers. Once a detector’s method becomes known, it becomes a target to train against, and the next generation of deepfakes gets built and tested specifically to slip past it. Independent testing already shows the gap. The first benchmark built to test detection tools against deepfakes actually circulating online, rather than clean academic datasets, found leading models’ accuracy dropped by roughly 45 to 50 percent across video, audio, and image detection compared to their performance on older benchmarks.^[xi]

The detector is not simply observing the threat. It is shaping what the threat becomes.

Accuracy is also a concern. False positives don’t simply create technical errors. For authentic creators they can create reputational damage that lingers long after the software has been proven wrong. Charles Kent, a technologist and Substack writer with a career spent across the AI and technology industry, has described this as a version of spectral evidence, the term Salem’s courts used for testimony about something only the accuser could see, which the accused had no way to disprove.^[xii] An AI-detection flag works the same way. You cannot produce an alibi for your own writing process. For a journalist, attorney, researcher, executive, or author, credibility is often their most valuable professional asset, and an incorrect accusation of AI-generated work doesn’t simply question a single document. It can quietly cast doubt over years of legitimate effort. When people begin editing themselves to satisfy an algorithm rather than communicate authentically, something subtle changes. Authenticity itself risks becoming a performance rather than an expression.

Every safeguard has consequences. That doesn’t mean we shouldn’t build safeguards. It means we should design them with humility, recognizing that every intervention changes the people living inside the system. In our effort to protect trust, we have to be careful not to manufacture new forms of distrust.

Trust Has Always Been Human

Linda’s story wasn’t really about artificial intelligence. It was about being human. It was about the instinct to protect someone we love. The willingness to act before we have all the facts. The extraordinary speed with which fear can override judgment. Technology didn’t create those instincts. It exploited them.

Trust does not live inside software or algorithms. It lives inside people. Business leaders will continue investing in stronger cybersecurity. Governments will continue developing standards. Technology companies will continue improving authentication, provenance, and verification. All of that work is necessary.

But trust has never been built by technology alone. It has always been built through judgment, relationships, and credibility earned over time. Artificial intelligence is asking us to become more deliberate. To verify more carefully. To question evidence that once seemed unquestionable.

It feels uncomfortable.

Every generation inherits a different version of that challenge. Our grandparents learned to lock their doors. Our parents learned not to believe every email. Our children may grow up learning that seeing something is no longer sufficient to believe it. Perhaps it’s the evolution of trust.

The future won’t belong to those who trust less.

It will belong to those who learn to trust more wisely.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

Part 7: The Generation Saying No: Is Anyone Listening?

Endnotes

^[i] Federal Bureau of Investigation, “Cryptocurrency and AI Scams Bilk Americans of Billions,” 2025 Internet Crime Report https://www.fbi.gov/news/press-releases/cryptocurrency-and-ai-scams-bilk-americans-of-billions

^[ii] Gartner, “Gartner Survey Reveals Generative Artificial Intelligence Attacks Are on the Rise,” September 2025 https://www.gartner.com/en/newsroom/press-releases/2025-09-22-gartner-survey-reveals-generative-artificial-intelligence-attacks-are-on-the-rise

^[iii] Sygnia, 2026 CISO Survey: The State of Incident Response Readiness, April 2026. https://www.businesswire.com/news/home/20260413646028/en/73-of-CISOs-Unprepared-for-the-Next-Big-Cyber-Attack-Incident-Response-Readiness-Report-Reveals

^[iv] IBM and Ponemon Institute, Cost of a Data Breach Report 2025 https://www.ibm.com/reports/data-breach

^[v] IBM, “2025 Cost of a Data Breach: Navigating the AI Rush Without Sidelining Security.” https://www.ibm.com/think/x-force/2025-cost-of-a-data-breach-navigating-ai

^[vi] IBM Newsroom, “IBM Report: 13% of Organizations Reported Breaches of AI Models or Applications,” July 2025. https://newsroom.ibm.com/2025-07-30-ibm-report-13-of-organizations-reported-breaches-of-ai-models-or-applications,-97-of-which-reported-lacking-proper-ai-access-controls

^[vii] SANS Institute, 2026 AI Survey Insights Report, July 2026 https://www.intelligentciso.com/2026/07/15/sans-report-highlights-growing-ai-governance-gap-in-cybersecurity/

^[viii] CNN Business, “Arup Revealed as Victim of $25 Million Deepfake Scam Involving Hong Kong Employee,” May 2024 https://www.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk

^[ix] Coalition for Content Provenance and Authenticity, published standards

https://c2pa.org

^[x] National Institute of Standards and Technology, AI Risk Management Framework https://www.nist.gov/itl/ai-risk-management-framework

^[xi] Chandra, N.A. et al., “Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024,” arXiv, March 2025 https://arxiv.org/abs/2503.02857

^[xii] Charles Kent, “Substacks AI Witchhunt,” Agent Autopsies, July 22, 2026.

This is Part 7 of The AI Reckoning: A Future of Trust Series

The most digitally fluent generation in history is quietly declining the most powerful technology ever handed to them. Almost everyone is misreading why.

She said it with one word.

My niece was in town, and I had taken her and two of her friends to lunch. They were curious about what I was working on, so I told them a little about the new business, about the AI coach my co-founder and I were building, designed around empathy, built to support people through change. I was still shaping how I talked about it then. This was before the TEDx talk, before the language had hardened into anything polished. I was just a person at lunch, describing the thing she believed in.

One of my niece’s friends looked at me and said, “Creepy.”

Not hostile. Not even particularly interested in debating it. Just a verdict, delivered with the casual confidence of someone who had settled the question long before she sat down at that table. The conversation moved on. I didn’t.

Because here is what I knew about that young woman that made the word impossible to dismiss. She was not afraid of technology. Nobody in that generation is. She has never known a world without it, never had to learn it the way the rest of us did, never experienced it as new. Her “creepy” wasn’t coming from someone who didn’t understand AI or was evaluating the technology. It came from someone who was evaluating the relationship.

I didn’t know it yet, but that lunch was the beginning of a question I would eventually be unable to put down: whether the loudest misread in this entire industry might be generational. The story we keep telling ourselves is that young people will lead AI adoption the way they led every technology wave before it, and that skepticism is something age does to you. But that is not what the evidence is showing us. The evidence is showing something stranger, and I think far more important.

The Misread

Inside most organizations, there is a standing assumption about who will struggle with AI. The older workers, the story goes, will resist. The young ones will lead. It’s a reasonable assumption, because it has been true for every technology wave in living memory. Teenagers drove the adoption of personal computers, video games, social media, and smartphones, and skeptical parents were dragged along behind them.

With AI, for the first time, the pattern has inverted.

The survey data emerging over the past year tells a story almost nobody predicted. Gallup’s 2026 study of more than fifteen hundred Americans between fourteen and twenty-nine found that excitement about AI among Gen Z has dropped fourteen points in a single year, to just 22 percent, while anger toward the technology rose nine points, to 31 percent.[i] A separate consumer study found that among Gen Z non-users, a majority say they are simply not open to adopting it, a higher refusal rate than their grandparents’ generation.[ii] Read that again. Older Americans are now more open to AI than young ones. The generation that was supposed to lead this adoption is the one most visibly declining it.

But there is a contradiction that breaks the easy explanation: their usage hasn’t collapsed. Roughly half still use these tools weekly. They haven’t fled the technology. They use it, fluently, while trusting it less and less. Their skepticism is not rising because they don’t understand AI. It appears to be rising in the places where they understand it best. Employed Gen Z workers were three times as likely to say the workplace risks of AI outweigh its benefits than to say the reverse. Only three percent said they would fully trust work generated entirely by AI. Given the choice, nearly seven in ten said they would rather have the fully human version.

The industry reads this as an adoption lag. Something training will fix. Better onboarding. More time. I want to suggest a different reading, and it’s the reason this piece sits where it does in the series. In Part 5 of this series, The Human Adoption Gap, I wrote about the difference between whether people can adopt a technology and whether they will. This generation is the purest expression of that distinction we have. Their can is total. They are the most digitally capable cohort ever to enter the workforce. It’s their will that is saying no. And when the people most able to use something are the most reluctant to engage, that is not a lag. That is a signal.

They’re asking whether it’s worthy of their trust.

What the No Is Protecting

If you listen to what our younger generation is actually saying about AI, rather than what gets said about them, the refusal starts to organize around three things and none of them are fear of the technology.

The first is creativity, and more precisely, the fear of losing the struggle that produces it. In the Gallup research, Gen Z consistently declined to believe that AI enhances creativity or critical thinking, and a striking eight in ten students said that using AI tools now would likely make learning harder for them later. One college student put it in terms that stopped me: AI removes the friction from learning, and the friction was the point.[iii] I understand that instinct in my bones. I learned to water ski in the ocean and to snow ski in a blizzard, and what those experiences taught me had nothing to do with technique. They taught me that we learn by pushing against something. Friction is not the obstacle to growth. It is the mechanism. A generation raised to believe that is now watching a technology whose entire promise is the removal of friction, and they are asking the question their elders keep skipping: what happens to the muscle when nothing pushes back?

The second is authenticity. This generation came of age inside a culture that prizes the genuine above almost everything, and they are watching a flood of synthetic content wash through every feed they grew up on. They can usually spot it. They increasingly resent it. And they have started to treat “made by a person” as a marker of value in itself, the way an earlier generation might have treated handmade or original. When seven in ten prefer the fully human version of a piece of work, that is not nostalgia. It is a statement about what they believe work is for.

I relate to this as a creator, not just an observer. I wrote my book waking at five every morning for a year, each word, each phrase, each thought mine. I used an editor at the end to fine tune, but the writing was mine, and the difference between those two things was never confusing to me. As a sculptor and painter I know the process of adding and subtracting and adjusting is not preparation for the work. It is the work. In business, as a creative director I went through version after version, but the vision came from somewhere within me, and no round of revisions ever changed whose it was.

And I will tell you something in the spirit of this piece: I write with AI now. I start with my own drafts, my own arguments, my own stories, and I use it the way I once used that editor, to pressure test and refine. The vision is still mine. The boundary I hold is the same one I hear this generation describing: I know exactly what it can do, and that is why I am careful about what I let it do for me. If that boundary sounds familiar, it should. It turns out the question they are asking is the right question for all of us.

The third thing the no is protecting is the ladder itself. Every previous generation entered the workforce at the bottom, doing the routine tasks through which a craft is actually learned, the first drafts, the research grunt work, the entry-level analysis. Those tasks are precisely what AI automates first. This generation is being asked to climb a ladder whose bottom rungs are being removed while they are standing on them, and then being told their hesitation is a mindset problem. The numbers behind that feeling are real: unemployment among recent college graduates is running well above the rate for workers overall, and competition for entry-level postings has climbed sharply year over year.[iv] One recent graduate put it plainly in a local news interview: the hard part isn’t accepting that AI exists. It’s accepting that it’s taking the opportunities they trained for. Their skepticism about whether this technology serves them is not paranoia. It is a reasonable reading of their own position.

Share

The Receipts

There is a reason this generation’s “no” arrives faster and more confidently than anyone expected.

They have seen this movie before. They are the only generation that lived the entire lifecycle of the social media promise from inside their own childhoods: the launch, the euphoria, the connection that curdled into performance, the engagement that turned out to be extraction, the apology tours. They were the product the last time a world-changing technology promised to empower them. The lesson they took from the smartphone era, that moving fast and breaking things includes breaking people, is documented in their behavior, not just their sentiment. A recent Deloitte survey found nearly a third of Gen Z had deleted a social media app in the past year.[v] The market for deliberately limited phones is growing fastest among buyers under thirty, and they describe the downgrade not as nostalgia but as boundary setting.

So when the next transformative technology arrived promising to empower them, they did something no generation before them had the experience to do.

They priced the promise against the last one.

The reframe I keep arriving at, that connects directly to the last piece of this series, Part 6, Signal vs Noise is this: the industry is filing this generation’s refusal under noise, sentiment to be managed, an adoption lag, or a training gap. But I believe their refusal is closer to signal than almost anything else in the market. They are the users with the most fluency and the least sunk cost, and they are the ones asking the eighteen-month questions the rest of the market keeps skipping. What does this do to me over time? What does it cost that isn’t on the invoice? Who benefits from my adoption? Those are the questions I spent the last piece urging builders, investors and enterprises to ask. An entire generation got there first and we are missing it because we’ve been calling it resistance.

They may be the only constituency in this story practicing foresight, and they are the ones being told they don’t understand.

The No My Algorithm Hides

Here is a confession that took me longer to arrive at than it should have. I live inside the world of AI. My feeds are full of builders and investors, people for whom this technology is genuinely thrilling, and the algorithms that shape what I see have learned to give me more of what I already give my attention to. The generational refusal this piece describes was invisible from where I stood, not because it was hidden, but because I was never shown it.

The first time someone put it in front of me directly, I pushed back. One of the women on my team shared the insight with me. She sent me an article making the case that this generation was turning away from AI, and my first reaction was the industry’s reaction, almost word for word. They’ll come around. It’s an adoption curve. They don’t fully understand it yet. I reached for every available explanation except the one on the page. It took me longer than I would like to admit to notice what I was doing: I was reading a signal and filing it under noise, which is the exact failure I had just spent an entire piece describing in other people. Even the lunch I opened this piece with had been sitting in my memory as an anecdote, one word from one young woman, until someone I trusted connected it to a pattern and made me look again.

Then the evidence stopped being quiet, siloed and easy to miss. At the University of Central Florida this Spring of 2026 a commencement speaker told graduates that AI was the next industrial revolution. The arena booed. At Middle Tennessee State, a record executive told the class that AI was rewriting their industry as they sat there; when they booed, he told them to deal with it, and they booed louder. At the University of Arizona, one of the most powerful figures in the history of the technology industry was booed repeatedly for saying AI would touch every profession and every relationship they would ever have.[vi] These were not activists who bought tickets to protest. They were graduates, at their own ceremonies. And this class matters more than any other: they started college the same Fall ChatGPT launched. They are the first graduating class whose entire education happened inside this technology. They know it better than the people at the podium. And their answer, delivered in the most public setting their lives have offered so far, was “no.”

If you are a builder or an investor and none of this has reached your feed, that is worth sitting with. I know exactly how the reflex feels from the inside. The signal was there. I filed it under noise myself, until someone made me look twice.

What I Don’t Know

This is the point in the piece where I would normally consolidate the argument. Instead I want to be direct about the limits of what the data can currently support, because this topic deserves that care.

We are still in the early days of AI, and the research on this topic is also early and thinner than the confident takes on all sides suggest. I don’t know whether this is a durable values position or partly a life-stage effect that softens when this generation has mortgages and deadlines and less room for principle. And I don’t know how evenly the refusal is distributed; it may be concentrated among students and creative workers and much weaker elsewhere.

I also know the boundary of this refusal is not a clean generational line, because I live with the evidence. Someone close to me, a generation older than the people this piece is about, has been telling me for years that AI is “just an algorithm.” He works with the special needs community, and his whole professional life is built on the conviction that nothing replaces human presence. He is also unfailingly supportive of me and of what I’m building, which means his skepticism arrives gently, and I suspect he softens it more than he feels it. And yet, the handful of times I’ve used AI to help him with something real, he has been genuinely glad for the result. He holds both positions at once, the principled rejection and the practical appreciation, and he is not confused. He is conflicted, which is different, and I have come to think it may be the most common relationship anyone has with this technology right now. The surveys capture it as a contradiction. Up close, it looks more like a person trying to protect something while still living in the world.

What I am also discovering as of late is that the standard explanation, that they’ll come around once they understand it better, has the evidence exactly backwards. Understanding is not what they lack. Understanding appears to be where the skepticism comes from.

The Seat, Not the Pitch

So rather than resolve the question, I want to open it up to discussion. If you are part of this generation, I would genuinely like to hear how you decide what AI is allowed to touch and what it isn’t. If you manage people from this generation, I’d like to hear what their selective refusal actually looks like inside your organization. The comments are open, and this is the one piece in this series where the conversation may matter more than the argument.

And if you are building or deploying this technology, the way forward is not a better message. It is a seat. This generation is not asking to be persuaded. They are asking to be included in the decisions being made about the work, the tools, and the future they will inhabit longer than anyone else in the room. The organizations that bring them into the planning, and visibly change course based on what they hear, will earn something no adoption campaign can buy. The ones that keep trying to convert them will keep manufacturing the resistance they are trying to cure.

I have thought about that lunch, a couple of years ago, more than almost any conversation this year. And I no longer hear “creepy” as a door closing. I hear it as the price of admission being named. My niece’s friend was not telling me the thing I am building should not exist. She was telling me what it would take for her to trust it: that it listen without harvesting, that it support without replacing, that it never pretend the relationship is something it is not. Those are not unreasonable demands. They are the most precise product requirements I have ever been handed, and they came from a young woman over lunch, for free. The question is whether anyone building this technology is willing to treat the generation that said “no” as the standard to build toward, rather than the audience to win over.

Every technology wave has had its skeptics. What it has never had, until now, is a skeptic class made up of its most capable users.

They aren’t afraid of the future. They’ve already lived in one version of it.

Their “no” isn’t the absence of an answer. It’s an answer.

The question now is whether the rest of us are listening.

The next piece turns from the people moving slower than the market wants to the market moving faster than its own safeguards can follow. Because while human trust struggles to keep pace with AI, something else is falling even further behind: security. And the gap between those two speeds is where the next reckoning is already forming.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

Part 6: Signal, Noise, and Judgment: The Trust Debt Nobody Is Measuring

[i] https://news.gallup.com/poll/708224/gen-adoption-steady-skepticism-climbs.aspx

[ii] https://fortune.com/2026/05/20/why-do-kids-hate-ai-gen-z-backlash/

[iii] https://www.usnews.com/news/national-news/articles/2026-04-09/gen-zs-ai-use-remains-stable-as-skepticism-grows-gallup-finds

[iv] https://www.cnbc.com/2026/05/21/new-graduates-booing-commencement-speakers-ai.html

[v] https://fortune.com/2026/05/20/why-do-kids-hate-ai-gen-z-backlash/

[vi] https://www.npr.org/2026/05/20/nx-s1-5822419/ai-colleges-commencement-booing

This is Part 6 of The AI Reckoning: A Future of Trust Series

Subscribe now

For most of human history, the challenge was finding enough information to make a good decision.

Today the challenge is surviving the tsunami.

Every morning brings another model, another benchmark, another study, another prediction, another expert explaining where AI is taking us next. Articles are summarized before we’ve read them. Research is synthesized before we’ve had time to consider it. Opinions arrive polished, confident, and increasingly difficult to distinguish from one another. Artificial intelligence has compressed the distance between questions and answers so dramatically that information is no longer our constraint.

Discernment is.

Somewhere inside this torrent are extraordinary insights. There is also speculation, marketing, repetition, bias, and enough convincing language to support almost any conclusion we want to reach. The challenge is no longer access to knowledge. It is deciding what deserves our attention, what deserves our skepticism, and what deserves the patience of a second look. That feels like a very different kind of leadership challenge than the ones we’ve spent the last two decades preparing for.

Every technological revolution changes the muscles society exercises. Calculators reduced the need for mental arithmetic. GPS changed how we navigate and remember geography. Search engines shifted our relationship with memory itself, replacing recall with retrieval. None of those technologies made us less capable. They changed where we invested our cognitive effort.

I find myself wondering whether AI is doing something similar with judgment. Not replacing it. Changing how often we exercise it.

Judgment has always been one of our defining human capabilities. I suspect we’re entering an era where it must become a deliberate discipline rather than an unconscious habit. The more information arrives at machine speed, the more valuable it becomes to slow down long enough to ask whether we’re seeing signal or simply responding to whatever demanded our attention first.

That thought finally crystallized for me during a demo day a couple of years ago. The founder before the break began with his previous company. It had been acquired by a name everyone in the room recognized, for a number everyone repeated to each other over coffee. He hadn’t said a word about the company he was pitching that afternoon, yet the room was already leaning forward. The questions afterward were warm, almost collegial.

Very few people asked what the business depended on, how resilient the architecture was, or what would happen if the market beneath it changed.

The founder who followed couldn’t tell that story. She spoke instead about customer retention, infrastructure decisions, and the choices she had made to avoid becoming dependent on a single platform. Her product solved one decidedly unglamorous problem, but it solved it completely. The questions she received were shorter, more skeptical, and noticeably less generous.

If I were placing a bet today, I would choose her company every time.

I’ve replayed that afternoon more often than I expected because it revealed something I now recognize almost everywhere. We have become remarkably efficient at evaluating the signals that are easiest to see, and surprisingly inconsistent at recognizing the ones most likely to predict whether something will endure. Under pressure, the impressive résumé often outweighs the resilient architecture. The polished demonstration eclipses the thoughtful design decision. Familiarity quietly becomes a substitute for judgment.

As I’ve written this series, something unexpected has happened. Each article began by exploring a different challenge surrounding AI: the infrastructure, the economics, the platforms, the fragmentation, the human adoption gap.

Yet after each one, I found myself coming back to the same question. We have more information than any generation before us. Why does good judgment sometimes feel harder rather than easier? I don’t think technology created that problem. I think it revealed it.

The Discipline of Judgment

Over the last year, I’ve noticed something changing in myself.

I trust my first reaction less than I used to, and it’s not because I’ve become more skeptical. In fact, quite the opposite. I still find myself excited by what AI is making possible. But I’ve also become aware of how easily certainty can now be manufactured. Every day brings another remarkable demonstration, another confident prediction, another analysis that feels complete before I’ve had time to consider whether it’s actually true.

I’ve realized the challenge is no longer finding answers. It’s deciding which answers deserve my trust and that feels like a different kind of work than I was doing even a few years ago. For most of my career, good judgment meant gathering enough information before making an important decision. Increasingly, I find myself doing the opposite. I spend less time looking for additional information and more time deciding what deserves my attention in the first place. Information has become abundant. Attention has not.

I don’t think I’m alone.

When we say we trust someone’s judgment, we’re rarely talking about intelligence. We all know brilliant people whose decisions we wouldn’t follow. We also know quieter leaders whose advice somehow carries unusual weight, and it’s not because they always have the answer. It’s because they’ve developed the habit of seeing one layer deeper than everyone else.

Perhaps that’s what discernment really is. Not judgment itself, but the practice that comes before judgment. The willingness to pause, to doubt, and to ask one more question before accepting the first convincing answer.

I’ve spent much of my career helping leaders navigate uncertainty, and I’ve come to believe trust follows good judgment far more often than confidence. Confidence persuades. Discernment earns trust. AI hasn’t diminished the importance of judgment, rather it has magnified it. The easier answers become to produce, the more valuable it becomes to recognize which ones deserve our attention and trust.

Sometimes I wonder whether judgment is like any other human ability. The capabilities we practice become stronger, while the ones we quietly outsource begin to fade. Calculators didn’t make us incapable of arithmetic, but most of us no longer calculate in our heads. GPS changed how often we rely on our own sense of direction. Search engines reshaped what we remember because retrieval became easier than recall.

I don’t know yet what AI will do to judgment.

I do know it’s worth asking before we stop exercising it.

The Shortcuts We Reward

Last year the word slop entered the mainstream. Officially, it describes low-quality content produced at extraordinary scale through generative AI. The definition is accurate, but I’ve come to think it captures only the smallest expression of a much larger pattern. Content was simply where we noticed it first.

The same dynamic now appears almost everywhere decisions are made quickly. Product demonstrations optimized for attention rather than resilience. Strategy decks polished enough to survive the board meeting but unable to survive implementation. Procurement processes that reward feature lists before understanding workflow. Investment decisions influenced by familiarity because familiarity is easier to evaluate than architecture.

The common thread isn’t artificial intelligence. It’s our growing tendency to reward whatever performs well during the first evaluation instead of asking whether it will still hold up during the second. AI didn’t create that instinct. It simply accelerated the consequences.

Share

The Builder’s Dilemma

Imagine a founder we’ll call Jordan.

Jordan is talented, deeply technical, and building at a pace that would have seemed impossible only a few years ago. What once required an engineering team and months of development can now be accomplished in a long weekend with the right combination of models, tools, and determination. That is one of the genuine marvels of this moment. The barriers separating imagination from execution have fallen farther and faster than almost anyone predicted.

Technology changed, and markets changed with it. The signals that attract attention changed as well.

Jordan discovers very quickly that architectural resilience rarely earns the first meeting. Platform independence doesn’t generate headlines. A carefully considered approach to long-term maintainability isn’t what gets shared on social media or featured in product launches. Attention follows novelty. Investment follows momentum. Recognition follows whatever appears to be moving fastest.

None of that makes anyone irrational. Jordan gets it. Markets reward what they can evaluate quickly, and speed has become one of the easiest things to recognize. So he adapts.

The product demonstration becomes a little more polished. The roadmap stretches a little further into the future than the engineering team would privately promise. Questions about infrastructure become conversations for another meeting. Human adoption becomes something to solve after product-market fit.

One compromise is rarely significant. Thousands of similar compromises begin to shape an entire ecosystem.

I’ve come to believe that many founders aren’t optimizing for durability nearly as much as they’re optimizing for investability. They’re responding rationally to the incentives placed in front of them. When the market consistently rewards what looks finished, it shouldn’t surprise us that so much effort goes into appearing finished, because every system eventually produces the behavior it rewards.

That observation extends well beyond founders.

When Familiar Feels True

Investors like to believe they recognize exceptional companies, and they often do. Experience matters. Pattern recognition matters. The ability to quickly identify talent is one of the reasons successful investors become successful in the first place.

Experience, however, has a quiet companion. Familiarity.

Behavioral researchers have spent decades documenting how uncertainty changes the way people make decisions. As ambiguity increases, the human brain begins searching for anchors. Something recognizable. Something that resembles a previous success. Something that reduces the discomfort of not knowing. Pedigree becomes reassuring. A familiar logo. A prestigious university. A previous exit.

Even physical resemblance has been shown to influence investment decisions more than most people would like to admit. Research from UCLA found that investors expressed greater interest in founders whose faces subtly reminded them of their own, particularly when objective information about the company was limited.[i]

None of this reflects dishonesty. It reflects humanity and we can even say biology. Our primitive brains see familiar as safe, and under pressure and overwhelm familiarity begins masquerading as evidence.

That distinction feels increasingly important in today’s AI market because uncertainty has never been higher. Nearly every company can produce an impressive demonstration. Every week introduces another breakthrough, another valuation, another announcement that seems to redefine the competitive landscape all over again.

When everything looks extraordinary, extraordinary stops being a reliable signal. The temptation is to lean more heavily on whatever still feels familiar, but ironically, those may be the very moments that require us to do the opposite.

The slower questions rarely generate excitement. They generate clarity. How dependent is this business on infrastructure it doesn’t control? What assumptions have to remain true for this model to work three years from now? What happens if those assumptions change?

Those questions are less satisfying than a compelling founder story. And they are also remarkably difficult to answer quickly.

Perhaps that’s exactly why they deserve more attention.

The Quiet Cost of Fast Decisions

Enterprise leaders face the same challenge from a different direction. The pressure is to avoid becoming the company that moved too slowly. Every board meeting now contains some version of the same conversation. Competitors are adopting AI. Customers expect it. Analysts are talking about it. Employees are experimenting with it whether policy allows it or not. Waiting feels risky. Moving quickly feels responsible.

Yet speed has a way of changing the questions we ask. The demonstration becomes more important than the deployment. Capabilities become easier to compare than outcomes. The procurement conversation revolves around features, while the more consequential questions remain surprisingly quiet. Will people actually choose to use this once the rollout is over? Does it reduce complexity or simply relocate it? Will this still fit the way our organization works six months from now, after the excitement has faded and the novelty has worn off? Those questions rarely appear in product demonstrations. They reveal themselves only after implementation begins.

Stanford and BetterUp recently described one consequence of this dynamic with a term that immediately resonated with me: workslop. Their research found that AI-generated work often appears complete while quietly increasing the amount of effort required by everyone downstream.[ii] Documents become longer but less thoughtful. Summaries require verification. Emails sound polished while communicating remarkably little. Employees spend time deciphering work that initially appeared finished.

The cost isn’t simply productivity. It’s confidence. Close to half of employees who received this kind of work said it made them see the sender as less capable, less creative, and less reliable. Forty-two percent said it made them trust that person less. Nearly a third said they’d be less willing to work with them again.[iii] Trust erodes in ordinary ways: a manager quietly starts double-checking someone’s work, a colleague grows hesitant to collaborate after one draft required too much rewriting. An employee who adopted AI hoping to appear more productive unintentionally becomes perceived as less careful, less creative, or less reliable.

The researchers were careful to point out that this wasn’t laziness. It was largely the predictable outcome of organizations encouraging greater AI adoption without investing equal effort in helping people use it well. That observation stayed with me because it echoes a pattern we’ve seen throughout this series.

Technology rarely creates the deepest problems. It exposes the assumptions we were already making.

What Signal Actually Looks Like

Noise has one remarkable advantage. It announces itself. Signal rarely does. It usually arrives quietly, without urgency or spectacle, asking questions that feel almost disappointingly ordinary.

Builders often recognize signal in the decisions no customer will ever notice. Architecture. Dependencies. Maintainability. The work that never appears in a product announcement but quietly determines whether a company can survive its second and third chapters.

Investors find signal in different places. Not in the certainty of a forecast, but in the quality of the assumptions beneath it. They ask what happens if the model provider changes direction, if infrastructure costs rise faster than expected, or if the competitive landscape looks entirely different eighteen months from now. The companies worth backing are rarely the ones with all the answers. More often, they’re the ones asking the most thoughtful questions.

Enterprise leaders face perhaps the most practical version of all. Product demonstrations eventually end. Adoption begins the following Monday morning. Will people still choose this tool after the rollout? Has it reduced complexity or simply relocated it? Does it disappear into the work, or does the work begin revolving around it? Those questions are quieter than feature comparisons. They are also far better predictors of whether technology becomes capability or simply another expense.

I’ve noticed something else while writing this series. The signals that matter most almost always require patience. The market rewards speed. Signal rewards attention. Those aren’t the same investment.

One Debt. Three Ledgers.

Looking across builders, investors, and enterprises, I no longer see three separate stories. I see one pattern expressing itself in different ways. Builders borrow against future relevance in exchange for today’s momentum. Investors borrow against future certainty by relying on familiar patterns when uncertainty becomes uncomfortable. Enterprises borrow against future adoption by assuming impressive demonstrations naturally become meaningful organizational change.

Each decision makes sense on its own. Together they accumulate something few organizations measure. Trust debt.

Trust debt is rarely dramatic. It accumulates quietly through hundreds of small decisions that optimize for the immediate over the enduring. Every shortcut taken before understanding is complete. Every assumption left unexamined because there wasn’t time. Every impressive signal accepted before someone asked whether it reflected substance or simply appearance.

Eventually the invoices begin arriving. A product people stop using. An investment thesis that quietly unravels. A strategy that looked compelling until reality introduced variables no presentation anticipated. By then the opportunity to make a different decision has already passed.

The debt wasn’t created when the outcome failed. It was created when judgment yielded to urgency.

The Advantage of Looking Further Ahead

One of the questions I’ve been asking throughout this series is why we seem so willing to pay for hindsight while resisting the smaller investment foresight requires.

Perhaps it’s because hindsight arrives with certainty. Foresight asks us to sit with uncertainty long enough for understanding to emerge. Markets rarely reward that kind of patience in the short term. Leadership often does.

I’ve spent much of my career helping organizations navigate transformation, and one lesson has surfaced again and again. The leaders who consistently earn trust are rarely the fastest people in the room. They’re the ones who know when speed creates advantage and when it quietly becomes liability. They understand that moving quickly and thinking carefully are not opposing ideas. They are complementary disciplines.

The pace of AI isn’t slowing, nor should it. The opportunities are extraordinary. The question I keep returning to is whether our capacity for discernment is expanding at the same rate as our capacity to generate answers. I’m not certain it is. That doesn’t make me pessimistic. It makes me attentive. Because I suspect the next competitive advantage won’t belong exclusively to organizations with better models, larger budgets, or faster infrastructure. It will belong to those that become unusually good at distinguishing signal from noise while everyone else is still reacting to whatever arrives first.

Artificial intelligence has dramatically expanded what we can know. The future may be shaped just as much by what we choose to trust.

The next article turns to a generation that has lived inside this digital environment and tsunami of information longer than anyone else. While much of the market interprets their hesitation as resistance, I wonder whether they’re practicing something we’ve become too busy to notice.

Not fear. Discernment.

Because surviving the tsunami won’t depend on how much information we can generate.

It will depend on how well we recognize what deserves our trust.

And that may become one of the most important human skills of the AI era.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Part 5: The Human Adoption Gap: We Built the Technology, We Forgot the Human

[i] That Silicon Valley Founder Reminds Me of His VC! - UCLA Anderson Review

[ii] AI-Generated “Workslop” Is Destroying Productivity

[iii][iii] https://hbr.org/2025/09/ai-generated-workslop-is-destroying-productivity

This is Part 5 of The AI Reckoning: A Future of Trust Series

She used to be the person everyone went to.

Not because she held the highest title or had the loudest voice in the room. She was the one people sought out because experience had given her something harder to teach than technical skill. She understood how customers thought. She knew where projects quietly unraveled before anyone else noticed. She knew which shortcuts saved time and which ones created problems months later. Over two decades, she had become the person people trusted when the answer wasn’t obvious.

Let’s call her Kelly.

Then AI arrived. Not all at once, but steadily. One new tool became three. Pilot programs became mandates. Training sessions filled the calendar. Every town hall carried the same message: this was the future, and everyone needed to come along.

No one ever told Kelly she was being replaced. No one needed to. For the first time in her career, she wasn’t sure whether sharing everything she knew made her more valuable or simply made it easier to automate what had once made her indispensable.

That uncertainty changed the way she showed up long before it changed the way she worked. She attended every training session. She experimented with the tools. She completed the exercises and checked every box the organization asked her to check. When people were watching, she used the new system exactly as expected. When they weren’t, she quietly returned to the methods she trusted.

From the organization’s perspective, the rollout was succeeding. The dashboards showed another active user. Another completed training. Another employee adopting AI.

But Kelly hadn’t adopted anything. She had complied. Those two things can look almost identical on a dashboard. Inside a human being, they are profoundly different.

Every article in this series has followed AI through a different layer of its infrastructure: the physical world, the economics, the platforms, and the fragmentation created by an ever-expanding landscape of tools. Every one of those forces ultimately arrives in exactly the same place: a person being asked to change.

The greatest bottleneck to AI has never been the technology. It has always been our ability to understand and support the human being asked to trust it.

Subscribe now

Can vs Will

When organizations struggle to realize value from AI, the conversation almost always turns to the technology. Maybe the models aren’t capable enough. Maybe the data isn’t ready. Maybe the integrations need more work. Those things matter, but they are rarely where transformation succeeds or fails.

After spending decades studying human performance and organizational change, I’ve watched the same pattern repeat itself through mergers, digital transformations, restructurings, and now AI. Leaders naturally focus first on the technology because it’s visible. The human experience is quieter. By the time it becomes visible, it has usually become expensive.

And organizations tend to ask whether people can adopt AI. Do they have access to the tools? Have they completed the training? Do they understand the prompts? Can they use the technology? Far less attention is given to a different question. Will they?

Can is about capacity. Do people have the time, the knowledge, the cognitive bandwidth, and the practical ability to incorporate something new into the way they work?

Will is about trust. Trust is difficult because it isn’t a single emotion. It is built from dozens of perceptions, often invisible to everyone except the person experiencing them. People naturally ask themselves: Do I still have a place here? Am I becoming more valuable or less? Is this change happening with me or to me? If I admit I’m struggling, what will people think?

None of those concerns appear on a dashboard, but every one of them shapes whether someone truly adopts change. One question asks whether adoption is possible. The other asks whether it is likely. Organizations routinely invest in the first while quietly assuming the second will take care of itself. The evidence suggests otherwise. Boston Consulting Group has spent years studying organizations that successfully translate AI investment into business value. Their conclusion is surprisingly consistent: roughly 10 percent of the effort goes into algorithms, 20 percent into technology and data, and 70 percent into people and process.

That framework doesn’t diminish the importance of technology. It simply puts it in context. The algorithm may be the most visible part of an AI transformation. The human being is still the largest determinant of whether it succeeds. And yet, when budgets tighten or timelines compress, it is almost always the human side of the equation that gets treated as optional. The irony is difficult to ignore. We continue investing in making AI more capable while underinvesting in helping people become confident enough to use it. The technology keeps getting better. The bottleneck remains exactly where it has always been.

The Visibility Gap

Leaders are not flying blind because they lack data. They are flying blind because they are receiving the wrong kind of data. The management systems most organizations rely on were designed for a different era. They were built to measure attendance, productivity, utilization, compliance, and output. They tell us whether work was completed, training was finished, systems were deployed, and licenses were assigned.

Leaders measure exactly what the systems they inherited were designed to measure. The challenge is that AI asks leaders to understand something those systems were never built to see: confidence, readiness, cognitive load, psychological safety, and trust. Those are often dismissed as “soft” issues. In reality, they are the conditions that determine whether transformation actually takes hold. They shape whether people experiment, whether they ask questions, whether they share what they know, and whether a new way of working ultimately replaces the old one.

Organizations can tell you how many people completed the training. They can tell you how many licenses have been activated, how many prompts were submitted, and how frequently employees logged into a new platform. Those metrics are useful, but they are only part of the story. A login tells us that someone accessed a system. It tells us almost nothing about what happened after they did. Did they trust it? Did they find it genuinely helpful? Did it become part of the way they naturally work, or did they quietly return to the old process five minutes later? The dashboard cannot tell us.

That is because organizations and employees are often experiencing the same transformation through entirely different realities. Leaders see implementation. Employees experience uncertainty. Leaders see completed training. Employees wonder whether years of hard-earned expertise still matter. Leaders see increasing usage. Employees quietly decide which parts of themselves still feel safe to contribute.

Neither perspective is irrational, and neither perspective is complete. They are each responding to different information. That isn’t a leadership failure, and it isn’t an employee failure. It is a perception gap.

Management systems inevitably reflect the assumptions of the era in which they were designed. Industrial organizations learned to measure output. The knowledge economy expanded that to productivity, utilization, and efficiency. Those measures still matter. But AI is asking organizations to manage something fundamentally different: the human experience of continuous adoption. That shift changes the signals that matter. Some are technological. Others are deeply human. We have become remarkably sophisticated at measuring the first while remaining surprisingly dependent on intuition for the second. As a result, we optimize what we can see while overlooking the forces that ultimately determine whether change takes hold.

The irony is that many leaders sense something is wrong long before they can explain it. The promised productivity doesn’t quite materialize. Once the initial enthusiasm fades, teams quietly drift back toward familiar workflows. The technology works, but the transformation never fully arrives. So organizations respond in the ways they know best: they schedule more training, increase communication, introduce incentives, and encourage managers to reinforce adoption. Those are not the wrong responses. They are simply responses to the wrong diagnosis.

If the underlying challenge is not capability, but trust, then no amount of additional training will solve it. It may simply make people better at appearing to adapt, while leaving the real obstacle untouched. You cannot solve problems you cannot accurately see, and today, the greatest blind spot in most transformations isn’t technological, it’s human. The organizations that will thrive in the AI era won’t be the ones with the most advanced models. They’ll be the ones with the clearest visibility into the human experience of change.

Share

The Trust Gap

Several months ago, I spoke with a senior business development executive whose company had begun rolling out AI across the organization. Leadership asked him to help train the system by documenting the objections customers raised and capturing the patterns he had learned over years of conversations. They wanted him to teach the AI what had made him successful.

On paper, it was a perfectly reasonable request. The company wasn’t trying to replace him, they said. It was trying to preserve institutional knowledge and make it more broadly available. From a business perspective, the logic was sound.

As we talked, he became quiet. Then he admitted something that has stayed with me ever since. “I wasn’t giving it everything.” He wasn’t refusing to participate. He wasn’t trying to sabotage the initiative. He understood why the company was doing it, and in many ways, he even agreed with the strategy. But he also couldn’t ignore the question quietly running through the back of his mind. What happens if the thing I’m teaching eventually makes me less necessary?

Later he told me something even more revealing. Without ever discussing it, several of his peers had arrived at exactly the same conclusion. Share enough to look cooperative. Hold back enough to stay necessary. At first glance, that sounds like resistance. I don’t think it is. I think it is a deeply human response to uncertainty.

If someone genuinely believes the knowledge they have spent years or even decades developing could become the very thing that diminishes their future value, protecting some of that knowledge isn’t irrational. Whether that perception is ultimately accurate is almost beside the point. People don’t only respond to reality. They respond to what they believe reality might become. That distinction matters because organizations often interpret behaviors like these as a lack of commitment to change. More often, they are signals that people are trying to answer questions no one has helped them answer. Will I still matter? Will my experience still matter? Will my role evolve, or disappear? Is this happening with me, or to me?

Those questions often remain unspoken because the perceived cost of asking can feel uncomfortably high. If I admit I’m struggling, will people assume I can’t adapt? If I question the rollout, will I be labeled resistant? If I say I’m uncertain, will someone quietly decide I’m no longer the right person for the future? Most people never say those things aloud. Instead, they attend the training. They smile in the meetings. They use the expected language. They do enough to demonstrate cooperation while privately trying to make sense of what the change means for them.

I’ve spent much of my career studying organizational change, and one pattern has remained remarkably consistent. People are surprisingly honest when they feel psychologically safe. When they don’t, they become careful. That is an important distinction. Careful people don’t necessarily tell you what is false. They tell you what feels safe. They hold back questions they fear will be misunderstood. They avoid conversations that might make them appear less capable. They quietly preserve options while they wait to see where the organization is actually heading.

None of this makes leaders the problem, and none of it makes employees the problem. It is what happens when two groups of well-intentioned people are trying to navigate profound uncertainty with different information, different incentives, and different perceptions of risk. That is why trust matters so much. Without it, organizations don’t lose intelligence. They lose honesty. And without honesty, even the best technology cannot tell leaders what they most need to know.

The Pace Humans Were Never Built For

Every major technological revolution has required people to adapt. The difference today is not that change exists. It is the speed, frequency, and compounding nature of that change. Before most organizations have fully integrated one new platform, another arrives. Workflows are redesigned. Roles evolve. Expectations shift. New tools appear before old habits have had time to settle. We aren’t simply learning new technology. We are living in a state of continuous adaptation.

Human beings don’t change at the speed of software. Technology can be updated overnight. The human nervous system cannot.

Every meaningful change asks the brain to do something remarkably difficult. It must build new neural pathways while allowing older, deeply practiced ones to gradually become extinct. That doesn’t happen because someone attended a training session or watched a demonstration. It happens through repetition, reinforcement, experience, and, perhaps most importantly, a sense of safety.

Habits are not simply behaviors. They are biological shortcuts. Over time, the brain learns which patterns require the least amount of effort and attention. Those familiar pathways become efficient, automatic, and reassuring. Replacing them requires considerably more than information. It requires enough repeated experience for the unfamiliar to eventually become familiar.

That is one reason transformation often unfolds more slowly than leaders expect. People are not simply learning a new workflow. They are rewiring years of experience. And beneath all of it, the nervous system is asking a remarkably simple question: Is this safe?

When the answer is yes, curiosity expands. People experiment. They ask questions. They tolerate mistakes because they believe learning is part of the process. When the answer is no, the brain behaves very differently. Attention narrows. Familiar routines become more attractive. The old process feels easier to trust, even when everyone agrees the new one is objectively better. Familiar feels safe, even when it isn’t. That isn’t stubbornness, and it isn’t laziness or resistance. It is biology.

What makes this even more complex is that there is no universal timeline for adoption. One person may embrace a new technology immediately while struggling for months after moving to a new city. Another may welcome a major life transition but find a software rollout unexpectedly overwhelming. Someone else may appear calm while experiencing significant internal stress that no one around them ever sees.

My co-founder, Craig Martin, has a phrase he often repeats: “Everyone is an edge case.” His perspective comes from decades of designing complex systems. Mine comes from years spent studying human behavior, organizational change, and cognitive behavioral neuroscience. We arrived at the same conclusion through very different disciplines: people don’t adapt in predictable, standardized ways because no two human beings experience change, or life, in exactly the same way. Every person brings a different history, different experiences, different sources of stress, different levels of resilience, and different perceptions of risk. What feels exciting to one person may feel threatening to another. What feels manageable today may feel overwhelming after months of continuous change.

Yet most transformation strategies are designed as though people adapt in roughly the same way and on roughly the same timeline. They don’t. That doesn’t mean organizations should slow innovation, but it does mean we need to stop expecting human adaptation to follow the same exponential curve as technological advancement.

The challenge isn’t convincing people to change. It’s creating the conditions where change can genuinely take root. Those are profoundly different problems. One asks how we deploy technology. The other asks how we support human adaptation. For decades, we’ve been remarkably good at answering the first question. It’s the second one we’ve largely left unanswered.

The Infrastructure We Never Built

If we step back from AI for a moment, a broader pattern begins to emerge. Over the past several decades, we’ve built extraordinary infrastructure for technology. Infrastructure for compute, data, cybersecurity, workflows, systems of record, and governance. Every time technology introduced a new challenge, we responded by building the systems needed to support it. That is what infrastructure does. It makes progress possible.

Yet every one of those investments ultimately arrives in exactly the same place: a human being deciding whether to trust the change. That is where our thinking begins to break down. For all the sophistication we’ve poured into technology, we’ve continued to assume that human adoption will naturally follow. We communicate the vision. We schedule the training. We deploy the tools. We measure adoption. And we hope people come along.

Sometimes they do, but more often they don’t. And it’s not because the technology failed or because the people failed. It’s because we’ve treated human adoption as an expected outcome rather than something that deserves the same intentional design as every other part of transformation. When organizations struggle to realize value from AI, the response is often to improve the technology, expand the rollout, or increase training. Those efforts are important, but they all begin from the same assumption: that the missing piece is better execution.

What if the missing piece isn’t execution? What if it’s infrastructure? Not infrastructure for technology. Infrastructure for people.

For decades, we’ve built systems that help organizations deploy technology. What we’ve largely overlooked are the systems that help human beings adapt to it. That kind of infrastructure doesn’t exist only during implementation or inside a training session. It exists in the moments when change becomes personal. At three o’clock in the morning when someone lies awake wondering whether AI will eventually replace the career they’ve spent twenty years building. Five minutes before a high-stakes meeting when uncertainty quietly becomes self-doubt. The first week in a new role. The difficult conversation after a restructuring. The moment someone wants to ask for help but worries it might change how they are perceived. The conversation a manager doesn’t know how to begin. Those are the moments where trust is either strengthened or quietly begins to erode.

They are also the moments most organizations never see. Yet those moments determine whether people lean into change or quietly retreat from it. They determine whether experience is shared or protected, whether confidence grows or fear takes hold, and ultimately whether technology becomes integrated into the way people work or simply another tool they learn to work around.

That is the infrastructure we’ve never built. Infrastructure that helps leaders understand how change is actually being experienced, not simply how it was intended. Infrastructure that makes invisible human signals visible before uncertainty becomes resistance, before disengagement becomes attrition, and before trust quietly begins to erode.

The first four articles in this series examined the hidden constraints shaping AI: energy, economics, dependency, and cognitive overload. This article has explored another. The human one. Because every AI initiative, every digital transformation, and every organizational change eventually arrives in exactly the same place. A human being deciding whether to trust what comes next.

Technology may shape the future, but people determine whether it arrives. And those decisions happen through thousands of ordinary moments, one person at a time.

Conclusion

Perhaps the greatest lesson of AI isn’t about artificial intelligence at all. It’s about human intelligence. It’s about understanding how people learn, adapt, trust, and ultimately decide whether a new way of working becomes part of their lives or quietly fades into another failed initiative.

As builders, leaders, investors, and buyers, we have every reason to continue pushing technology forward. We should. But if this series has taught us anything so far, it is that every technological breakthrough eventually reaches the same destination: a human being deciding whether to trust it, use it, and integrate it into the way they work.

Increasingly, I believe the limiting factor in transformation is our ability to help people move through change. The organizations that thrive won’t necessarily be those with access to the most powerful AI. They will be the ones that recognize technology and human adoption are not separate challenges. They are two halves of the same transformation.

We built the technology.

Now we have to build the trust.

Share

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

This is Part 4 of The AI Reckoning: A Future of Trust Series

There is a particular kind of tired that has nothing to do with how hard you worked.

You sat down at nine with one thing you needed to think through. By noon you had answered messages in three places, approved something in a tool you open twice a month and have to relearn every time, moved a task from one board to another because two teams use different systems, and asked an AI assistant a question, then asked a different AI assistant the same question because the first one could not see the document the second one wrote. None of it was hard. All of it required you to stop, reorient, remember where you were, and start again.

At the end of the day you are depleted, and you cannot point to the thing that depleted you. The work was not the problem. The work was the small part of the day that survived everything around it. What exhausted you was the space between the tools. The seams.

We have built a working life made almost entirely of seams.

This is the cost no one is accounting for, and it is the one I want to talk about, because it leads somewhere that matters for anyone building, buying, or betting on the next decade of this industry.

Here is where it leads, stated plainly, so the rest of this is the proof and not the suspense. Human behavior rejects complexity and overload. The cognitive cost of switching between disconnected tools is real, it is larger than almost anyone accounts for, and it is poorly understood precisely because it hides in the gaps where no one is measuring. That single fact makes consolidation inevitable, not as a trend that comes and goes with budgets, but as something the math arrives at on its own. And once consolidation is inevitable, the only question left that matters is what you consolidate onto: something built to serve the people depending on it, or something that quietly comes to own them.

This is also, if you have been following the series, a familiar shape. The first piece described how we deploy technology faster than we understand its second-order consequences, and pay later for the costs that were knowable in advance. Fragmentation is that pattern again. The cognitive load of tool sprawl was knowable. We moved too fast to count it. Now it is surfacing as burnout, as stalled adoption, as budgets that produce exhaustion instead of output. The bill was always coming. We just declined to read it in advance.

The last piece in this series looked at a danger beneath the people building right now: the ground they do not own, the platforms that can shift under them without warning. This piece is about a danger above them. The limit of the very human they are building for. And the rush to ship one more point solution, the same rush that sends builders onto borrowed ground, runs straight into it.

This is part of the same reckoning the series has been tracing, and it has a specific cause. AI has made it extraordinarily easy to see a single pain point and ship something that solves it. What took an engineering organization a year now takes a small team a weekend. That is a genuine wonder, and it is also the engine of the problem. When solving one narrow thing becomes this cheap and this fast, the world fills with narrow solutions, each one excellent, each one more surface for a human to carry. The same capability that makes this the most exciting moment in the history of building is the thing quietly manufacturing the fragmentation we are all starting to feel. The reckoning is not that the tools are bad. It is that nobody priced what it costs to live among all of them at once.

If you are reading this nodding, there is a good chance your next thought is about yourself. That you should be more disciplined. Close the tabs, batch the messages, get organized, try harder. Almost everyone reaches for that explanation, and reaching for it is precisely why the real cause stays hidden. So it is worth asking the question directly.

Why Is This Not a Discipline Problem?

It would be easy to file all of this under personal discipline. Close your tabs. Check messages less. Get organized. And there is some truth in that. But the deeper issue is not a habit. It is the architecture of human attention, and it does not bend to willpower the way we wish it would.

When you switch from one task to another, part of your attention stays behind. The psychologist Sophie Leroy named this attention residue: after a switch, your mind keeps processing the thing you just left, and the performance on what you turned to is measurably degraded until the residue clears. The effect is worse when the first task was unfinished, which describes almost every interruption in a normal workday.¹

This is not a small effect at the edges. Researchers tracking knowledge workers have found they switch tasks every few minutes, and that returning to full focus after a real interruption can take as long as twenty-three minutes. Most of the time, the next interruption arrives long before those minutes are up. The result is not a workday with a few costly interruptions in it. It is a workday that never fully assembles a single sustained thought.²

Put a number on the switching itself and it becomes hard to look away from. One analysis of a large workplace dataset found that people toggle between applications and websites on the order of twelve hundred times a day, and lose something like four hours a week purely to reorientation. Four hours lost to finding their focus again.³ A productivity tax quietly waiting for an answer.

Here is the part that matters for where this is going. That cost is not a function of bad tools. Each tool may be excellent. It is a function of the gaps between them, and those gaps multiply with every tool you add. The friction is not in the apps. It is in the human being asked to hold them all together. And the human has a hard limit that no upgrade cycle will raise.

This is why I do not think tool consolidation is a trend that will come and go with budgets. It is a response to something fixed. The number of disconnected surfaces a person can carry has a ceiling, and we have been pretending it does not exist. When a constraint is rooted in human biology rather than market preference, the pressure against it does not ease. It accumulates until something gives.

Consolidation, in other words, is not a preference anyone gets to debate. It is what a fixed human limit produces once enough tools pile up against it.

Subscribe now

The Tax the Enterprise Already Pays

Step back from the individual and the same pattern reappears at the scale of the organization, with zeros attached.

Depending on size and how you count, a company runs somewhere between roughly one hundred and several hundred software applications. Studies consistently find that about half of the licenses paid for go unused, and that the average enterprise wastes a sum measured in the millions every year on software nobody opens.⁴

But the license waste, large as it is, is the part that shows up on a spreadsheet. The more expensive cost is the one we just walked through, now multiplied across every employee. The enterprise pays once for the tool, and then pays again, invisibly, for the human friction of running it alongside forty others. The second bill never appears in procurement. It appears in how long things take, in the quality of thinking people have left after navigating the stack, in the errors that creep in at the seams, and in the quiet erosion of the sense that the work is coherent.

Here is the part that should stop a leadership team cold. Most enterprises adopt these tools chasing two specific outcomes: higher productivity and lower burnout. Those are the words on the dashboard, the goals in the deck, the reasons the budget was approved. And past a certain point, each additional point solution delivers the opposite of both. The switching cost eats the productivity gain the tool was bought to create. The fragmentation feeds the burnout the tool was bought to relieve. The organization is, with the best intentions and a straight face, spending money to manufacture the exact problems it is also spending money to solve.

There is a quieter consequence underneath that one, and it is the one that decides whether any of this technology actually gets used. Human adoption follows the path of least resistance. People take up and keep using the things that reduce friction, and they quietly abandon the things that add it, no matter how capable those things are on paper. Every additional disconnected tool raises the resistance, which is why so much purchased software sits unused and so many rollouts stall after the pilot. A platform that absorbs a domain’s fragmentation into one coherent surface does the opposite. It lowers the resistance, and adoption follows almost on its own. This is the link the spreadsheets miss: the same coherence that relieves the cognitive load is also what makes the technology stick. Fragmentation does not just exhaust people. It is the reason the adoption curve flattens.

This is structural rather than incidental. Most organizations know they have overlapping tools. Their own employees say so. And most have still taken no real steps to consolidate, because every redundant application has an internal champion who chose it and a team that has built its habits around it. The pressure to simplify keeps losing to the easier path, which is to add one more thing.⁵

That standoff cannot hold indefinitely, because the human cost compounds while the political cost of consolidating stays roughly fixed. At some point the first overtakes the second. For investors, this is worth sitting with, because it means the pressure toward consolidation is not sentiment. It is arithmetic with a delay on it.

For the enterprise leader, this reframes the decision on the table. The question is no longer which point solution to buy next. Adding one more best-in-class tool to relieve a pain point is, more often than the business case admits, adding to the very load that is suppressing productivity and driving burnout in the first place. The next defensible move is to look for the platform that owns the whole domain. The one that collapses a dozen scattered tools into a single coherent surface, so the switching cost inside that domain simply disappears.

This is not a call to herd everything onto one provider. A person may still move across several platforms in a day, and that is fine. What matters is that each one is coherent within its domain, and that the few which need to talk to each other do so through connections intelligent enough to carry context across, so knowledge entered once does not have to be entered again. Fewer surfaces, each one whole, connected in ways that remember. That is the elegant version of consolidation, and it is a different thing from both the fragmentation we have now and the single dependency the last piece warned against.

The Ceiling Builders Do Not See

Now turn to the people making the tools, because this is where the story gets quietly difficult, and where I have the most sympathy.

A great many of the most impressive products being built right now solve one problem beautifully. A single workflow. A single friction. A single moment in someone’s day handled better than anyone handled it before. That is a real achievement, and it is often how important companies begin.

And it is, now, the easy thing to do. This is the part worth being clear about. AI has made catching a single pain point and solving it faster and cheaper than at any moment in history. Building a platform, one interface that holds many problems together coherently, is the opposite of easy. It demands wrestling with complexity that a point solution gets to ignore: how the pieces relate, what happens at the seams, how the whole thing stays trustworthy as it grows. Most builders cannot take that on, and many who could, will not. The reason is simple. We are wired to take the easier path, and that natural inclination is reinforced by a market actively cheering for the quick win. A customer in pain wants relief this quarter, not architecture that pays off in three years. Every incentive points toward the narrow solution. Almost none point toward the hard, slow, integrative work that would actually reduce the load. So the narrow solutions multiply, and the bigger picture goes unbuilt, not because no one can see it, but because seeing it and building it are very different commitments.

A point solution carries a ceiling it did not choose and usually cannot see. However well it solves its problem, it is still one more surface for the human to hold. One more login, one more place to check, one more thing that does not quite talk to the others. Which means a product can be successful on its own terms and still be structurally temporary. As the market becomes more aware of the fragmentation cost, standalone tools increasingly face the same outcome: integration, absorption, or abandonment. Not because they failed, but because the human on the other end ran out of room.

This is the trap closing from two sides at once. From below, the ground the builder stands on: a point solution runs on a platform it does not own, which can reprice, compete, or change the terms whenever its own survival calls for it. From above, the human it is built for, who has no capacity left to carry one more disconnected surface. Exposed beneath and exposed overhead. Same builder, same speed, and most of them, moving as fast as this market rewards, see neither limit until one of them gives.

The builders who internalize this early make a different choice. They stop asking only whether their product solves its problem, and start asking whether it reduces the total weight the person is carrying or adds to it. Those are very different questions, and they lead to very different companies.

Consolidate Onto What

So consolidation is coming. The human mind requires it, the enterprise math rewards it, and the builders who ignore it are constructing ceilings over their own work. On that much I am confident.

The mistake is assuming that consolidation means putting everything in one place. It doesn’t. The answer to fragmentation is not one giant platform that tries to become your entire stack. That simply replaces one problem with another. The cognitive load disappears, but the dependency remains. You have reduced the number of seams only to discover that everything now depends on a single provider whose incentives, pricing, roadmap, and priorities are not yours. We have already explored that risk in this series.

A better answer is domain-level coherence. A process platform should own a process. A data platform should own data. A communication platform should own communication. Each should reduce the fragmentation within its domain so completely that the person using it rarely needs to think about the seams inside it. The goal is not fewer capabilities. The goal is fewer surfaces. The distinction matters. People do not experience work as architecture diagrams. They experience it as attention. Every unnecessary transition, every duplicated workflow, every place where context has to be manually reconstructed is a tax paid in attention.

The platforms that win over the next decade will not be the ones with the most features. They will be the ones that ask the least of the human being using them. That is what fragmentation is creating demand for. Not more tools. Not bigger tools. Coherent ones.

Where the Value Actually Lands

Follow that logic all the way out and an investment thesis emerges; one investors are already beginning to ask about.

If point solutions are structurally temporary, exposed from below by the platforms they rent and from above by the human ceiling, then the durable value in this market does not accrue to them. It accrues to whatever becomes the layer that best serves human behavior. The fragmentation that exhausts the worker and taxes the enterprise is, read correctly, a demand signal. It is the market generating need for platform solutions faster than anyone is building them. Every new point solution shipped makes that need slightly more acute, which is a strange thing to realize: the flood of narrow tools is not the competition for these unified solutions. It is the thing creating demand for them.

What the smartest investors are starting to ask is not which point solution wins its category. Many will win their category and still be absorbed or abandoned, because winning a category is not the same as being something a human can sustainably carry within the context of a full workday.

The question is, who builds the infrastructure layers that reduce the friction of switching across disconnected tabs, and whether they build it as a neutral peer to the large models.

Those are not the same bet, and only one of them is durable, because only one of them does not eventually turn on the people depending on it.

This is the part that does not fit on the current scoreboard. We are measuring point solutions by adoption and revenue, the metrics that look good right up until the human on the other end runs out of room. What we are not yet measuring is which companies reduce the total load and which ones add to it. That distinction, invisible on today’s dashboards, is where the next decade of value quietly sorts itself out.

What We Were Trying to Protect

It is worth remembering what the goal was, underneath all of this. The point of reducing fragmentation was never efficiency for its own sake. Efficiency is what we say to the CFO. What we actually want is to give the person back the sustained attention the seams have been quietly stealing, hour by hour, for years.

The exhaustion I described at the start is not a personal failing and it is not the price of ambition. It is the predictable result of asking human beings to be the integration layer between tools that were never designed to hold together.

We made the person carry the coherence the systems lacked. Of course they are tired. Consolidation will relieve that weight. The only question that matters is whether we consolidate toward something that serves the human or something that comes to own them.

And that is where we’ll go next. Because every AI initiative ultimately arrives in the same place: A person deciding whether to trust what comes next.

What fragments the day is not the work. It is everything we built around it.

Consolidation is not the risk.

Consolidating onto the wrong thing is.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Endnotes

1. Leroy, S. (2009). Why is it so hard to do my work? The challenge of attention residue when switching between work tasks. Organizational Behavior and Human Decision Processes. https://www.sciencedirect.com/science/article/abs/pii/S0749597809000399

2. Mark, G., Gonzalez, V., & Harris, J. (2005). No Task Left Behind? Examining the Nature of Fragmented Work. Proceedings of CHI. University of California, Irvine.

3. Murty, R. N., Dadlani, S., & Das, R. B. (2022). How Much Time and Energy Do We Waste Toggling Between Applications? Harvard Business Review. Study of 137 users across 20 teams at three Fortune 500 companies over five weeks. https://hbr.org/2022/08/how-much-time-and-energy-do-we-waste-toggling-between-applications

4. Zylo, SaaS Management Index, 2026; BetterCloud, State of SaaSOps, 2025. Application counts vary by company size and counting methodology, with mid-market portfolios commonly cited around 100 to 300 applications and large enterprises higher.

5. Workplace technology overload analyses, 2026, citing internal-champion dynamics and the proportion of organizations that have not undertaken consolidation despite acknowledged tool overlap. Figures are industry estimates.

This is Part 3 of The AI Reckoning: A Future of Trust Series

Imagine Steve Jobs at eighteen years old today.

He walks into a world where the most powerful building tools in the history of technology are available to anyone with a laptop and a credit card. AI models that took billions of dollars and decades of research to create, accessible through an API. Compute that would have required a room full of hardware, running in the cloud for pennies. Frameworks, libraries, deployment infrastructure, all of it there, waiting, free or nearly free.

It would feel like Charlie finding the golden ticket. A world of pure imagination where creativity makes seemingly impossible things possible and where the only limits are his willingness to explore.

Steve Jobs at eighteen in today’s world would have no shortage of imagination, tools, or willingness. What he would need, more than anything else, is an understanding of whose factory he’s building in. Because Willy Wonka’s Chocolate Factory was a place of magic and wonder, but it was also surprisingly dangerous. The inventing room was full of things that hadn’t been fully tested. The chocolate river looked inviting right up until it wasn’t. It’s tempting to spend a weekend vibe coding and call it a business. But where you put down your flag matters as much as what you build. The space is likely leased, and some of the danger zones aren’t visible from inside the excitement.

Jobs may have still built on existing foundations. Many founders do, and it’s often the rational choice. As a founder, I understand the temptation. The tools are extraordinary. The speed is intoxicating. We can build things today that would have required entire engineering organizations a decade ago. But every strategic advantage comes with a tradeoff, and this one deserves more attention than it’s getting. Because the most important decision at this stage isn’t which foundation model to build on. It’s whether you understand what you agree to when you do.

Subscribe now

My co-founder Craig has lived through every major technology cycle since the internet. When we were making the decisions that shaped how we build, he was the one asking the questions that weren’t fashionable yet. Not whether the technology works, but who controls it once it does. Not whether the capability is real, but who is left standing when the market consolidates. He believed in this cycle. He understood that we finally had the compute to make the promise real. What made him cautious wasn’t the technology. It was the dependency.

This forethought ultimately shaped our own architecture decisions. We wanted language models where they added value, but we were unwilling to make the core reasoning layer dependent on technology we did not control. That caution led us somewhere most builders weren’t going, which is what this piece is about.

The danger isn’t that the platforms fail.

The danger is that they succeed.

Because when platforms become powerful enough to define a market, they also gain the ability to redefine the terms for everyone building inside it.

The Kill Zone

In venture capital, the kill zone has a specific meaning. It describes the territory surrounding dominant platform companies where rational investors have quietly stopped funding startups, not because the ideas aren’t good, but because the risk of the platform deciding to build the same thing is too high to justify the bet.

The concept emerged from studying what happened to startups building in the orbit of Facebook, Google, and Amazon. The platforms didn’t need to out-innovate the startups. They just needed to decide the category was worth their attention. When that happened, the startup’s funding dried up, its users migrated to the native version, and its competitive window closed faster than any business plan had modeled.

In the current AI landscape, the kill zone has a sharper edge. The startups building on top of OpenAI, Anthropic, and Google aren’t just building near these platforms. They are building on top of them. Their core capability runs on infrastructure they don’t own. Their competitive differentiation sits on a foundation controlled by companies whose long-term incentives don’t permanently align with theirs.

Imagine building your entire business on Apple’s App Store, only to wake up and find Apple shipping your core feature in iOS.

The platforms are already moving. OpenAI consolidated ChatGPT, Codex, and its browser into a unified desktop application in early 2026, explicitly to compete more directly with enterprise AI providers who had built on its foundation.^[i]

Google’s entire I/O 2026 narrative centered on agentic AI embedded directly into its existing product ecosystem, reducing the need for third-party tools that had been built to fill exactly that gap.^[ii]

Anthropic acquired Stainless, a startup whose SDK generation tools were being used by OpenAI, Google, and Cloudflare, reshaping the developer tooling layer beneath an entire ecosystem of builders who had assumed that infrastructure was neutral.^[iii]

These are not incidental moves. They are a pattern. And the pattern has a direction.

Every time a platform extends its reach into the stack, the builders who depended on that layer discover that the relationship they thought they had was more conditional than they understood. The ground they built on was borrowed. And borrowed ground can be recalled.

The Thin Wrapper Problem

Not all AI startups face equal exposure. The vulnerability scales with how much of the company’s core value proposition lives inside the platform versus on top of it.

At one end of the spectrum are what the VC community calls thin-wrapper startups: companies whose primary contribution is a better interface, a smarter prompt, or a more polished user experience layered over a foundation model. The underlying intelligence is rented. The distribution is rented. In some cases, the data processing and memory capabilities are rented too. What the startup owns is the design and the go-to-market motion. When the platform decides to ship those same design choices natively, the thin-wrapper startup discovers that what it thought was a product was actually a feature.

This is not hypothetical. OpenAI’s move to consolidate its products into a unified desktop application was a direct response to the ecosystem of third-party tools that had built on top of its API to solve exactly the fragmentation problem users were experiencing. The startups that solved it first validated the market. The platform then captured it.

The more defensible position is to build proprietary workflows, domain-specific data advantages, or genuine integrations that create switching costs the platform can’t easily replicate. But even that position is becoming harder to hold as the platforms invest in vertical-specific offerings and enterprise partnerships that go deeper into specific industries.^[iv]

The VC community is noticing. A growing number of investors now require founders to articulate what they call their platform independence thesis: an explicit argument for why their business remains viable if the underlying model provider decides to compete directly. A year ago, that question was rarely asked. Now it is table stakes in term sheet conversations.

There is another dimension to why technology consolidates that goes beyond competitive pressure. Not because dominant platforms force it, but because fragmentation carries its own cost, one paid daily by the people being asked to work across an expanding stack of disconnected tools. That story belongs to the next piece in this series, where I explore why so many promising point solutions are quietly becoming part of the problem they were built to solve.

The Enterprise Dilemma

The platform risk conversation in AI tends to focus on startups. But enterprises face a version of the same exposure, and in many ways the stakes are higher because dependency becomes embedded across people, processes, and customers.

What happens when critical business capabilities depend on providers whose incentives can change faster than your ability to adapt?

Consider what enterprise AI adoption actually looks like at scale. A company selects a foundation model provider. It builds internal workflows, customer-facing products, and operational processes on top of that provider’s API. It trains its people to work with those tools. It integrates them into its existing technology stack. It makes commitments to its own customers based on the capabilities those tools provide.

Then the provider reprices, deprecates the model version the enterprise built on, or launches a competing product in the enterprise’s own industry. None of those scenarios require a catastrophic failure. They simply require the platform to make a rational decision in its own interest. And each of these scenarios has already occurred in the current market. OpenAI deprecated its Assistants API in August 2025, with shutdown scheduled for 2026, forcing enterprises that had built on it to migrate to a new architecture on a timeline they didn’t choose.^[v]

The enterprise that built on that foundation didn’t lose its business. But it absorbed costs that never appeared in the original business case: migration, retraining, disruption to the people doing the work, and credibility spent with its own stakeholders on someone else’s timeline. Leaders who had promised capability delivered disruption instead. Employees who had been trained on one system were asked to start over on another. The business kept moving, but it moved carrying weight it hadn’t planned for.

This is what borrowed ground costs when it shifts. Not always catastrophe. Often something quieter and harder to quantify: the accumulated weight of decisions made on someone else’s terms.

Share

What the Relationship Actually Costs

There is an asymmetry in platform relationships that rarely gets named in the conversations where it matters most.

The platform knows the enterprise’s usage patterns, its most valuable workflows, its integration dependencies, and the switching costs it has accumulated over time. The enterprise knows the platform’s published pricing, its publicly stated roadmap, and whatever its sales team chose to share. Every conversation about what the relationship costs and what happens when it changes takes place in that gap.

That gap is not neutral. It tilts every negotiation, every pricing conversation, and every decision about what comes next toward the party with more information. The enterprise that has built deeply on a platform has already made its most important concession before it sits down to talk.

What makes this more complicated is that the platforms are not operating from a position of stability. As the previous pieces in this series explored, the entire frontier AI industry is still finding its path to profitability. The pressure to capture more of the value their ecosystems create is not a future risk. It is the present reality of every major provider. The enterprises and founders who validated those markets may find themselves on the wrong side of that math at exactly the moment they are most dependent.

This is not a moral failure on anyone’s part. It is the natural consequence of building interdependence into a market that is still sorting out its own economics. But it is a risk that deserves to be understood before the dependency is built, not after.

What Defensible Actually Means

What, then, does defensible mean in a market where the most powerful foundations are controlled by companies whose interests and yours will not always align?

Defensibility in this environment is not about having a better prompt. It is not about a more elegant interface. It is about whether the value you create can survive a change in the foundation beneath it.

For the startup, that means proprietary data, genuine workflow integration, and switching costs that belong to the customer relationship rather than to the platform.

For the enterprise, it means architecture choices that don’t create single points of dependency, and governance frameworks that account for platform risk alongside security risk.

For anyone building infrastructure rather than applications, it means understanding that the most durable position in this market is not the one closest to the frontier model. It is the one closest to the human being doing the work.

And for anyone building something intended to last, it means sitting with a question that the pace of this market makes easy to skip. Not am I building on something. Every builder builds on something. Jobs built on chips he didn’t fabricate. The transformer itself was built on decades of work that came before it. The question is sharper than that: does the thing I’m building on have interests of its own?

A protocol doesn’t decide to compete with you. It doesn’t reprice you or get withdrawn on a timeline you didn’t choose. But a platform owned by a company navigating its own survival can do all three. The distinction is not whether you depend on something. It is whether what you depend on can change its mind about you.

Steve Jobs at eighteen would have had more tools available to him than any builder in history. The factory is real, and it is extraordinary. The mistake would be assuming that because the factory feels permanent, it is. What Craig understood, and what ultimately shaped how we built, is that the most important question is not whether the tools work. They clearly do. It is whether the ground beneath them belongs to you.

The foundations beneath AI’s extraordinary buildout are less stable than most people realize. Not just physically and economically, as the first two pieces in this series explored, but structurally. The platforms that feel permanent are still finding their path to profitability. The incentives are still shifting. The boundaries are still being redrawn.

None of that makes the factory less remarkable.

It simply means builders should understand whose factory they are building in.

Because the most important question isn’t whether the factory is magical.

It’s who owns the keys.

In the next piece, we look at the other side of this consolidation story: what happens when every problem gets a new AI tool, and why the fragmentation those tools create may become its own hidden tax.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Endnotes

ⁱ https://www.mexc.com/news/969584

ⁱⁱ https://jeffreystop.com/news/2026-05-22-0900-ai-tech-news/

ⁱⁱⁱ https://thenewstack.io/anthropic-stainless-sdk-acquisition/

^iv https://www.mindstudio.ai/blog/google-vs-openai-vs-anthropic-momentum-2026-narrative

^v https://www.buildmvpfast.com/blog/which-ai-platform-startups-build-on-2026

This is Part 2 of The AI Reckoning: A Future of Trust Series

AI is in the Red.

That’s not surprising. Building the future has never been inexpensive. What surprises me is not that people aren’t asking about a path to profitability. They are. It’s that the assumption seems to be that if enough money is invested, enough people adopt, and enough capabilities emerge, the economics will eventually sort themselves out.

Maybe they will.

Subscribe now

But assumptions are not business models. And when I started putting the numbers in the same room, I found a story that looks very different from the one most people are telling. The story isn’t that AI is expensive. The story is that the risk of all that spending is quietly moving from the people who created it to the people least equipped to see it coming. The losses are not the headline. Who ends up holding them is.

There is a number that should be sitting at the center of every boardroom conversation about AI right now. It isn’t the productivity gain. It isn’t the efficiency multiple. It isn’t the competitive moat being built or the talent being freed up for higher-value work. It’s this: OpenAI, the company that started this revolution, projects $14 billion in losses in 2026 alone. Cumulative losses between 2023 and 2028 are expected to reach $44 billion.^[i] It also has 900 million weekly users and $20 billion in annualized revenue. A path to profitability exists on paper. It just doesn’t arrive until 2030, and it depends on several things going right in a market where nothing stays fixed long enough to model with confidence.

I’m not sharing this to suggest the technology doesn’t work or that the companies building it aren’t extraordinary. They are. The foundational work being done at OpenAI, Anthropic, Google, and others is remarkable, and what has been built in the last decade is what makes the next chapter possible. We stand on those shoulders. What we’re building wouldn’t exist without them.

And precisely because we are building in this space, I think the economics deserve an examination that most conversations are still avoiding.

Who Is Actually Carrying the Risk?

Here is what happens when you put the numbers together.

The AI providers are losing money despite massive and growing revenue. The enterprises buying their services are overpaying for something with no clean ROI framework. And here is the detail that almost never makes it into the same conversation: the prices enterprises are paying today don’t reflect the true cost of inference. The providers are absorbing a significant portion of that cost inside their losses. Enterprises are being onboarded at subsidized rates that the market hasn’t fully priced yet. When these companies eventually reach the profitability they’re projecting, through architectural improvements, pricing adjustments, or both, enterprise bills will almost certainly increase. The current pricing is not stable long-term pricing.

Everyone is spending more. The true cost of what they’re buying is higher than what they’re being charged. And the returns aren’t materializing at the rate the business cases promised.

This is not a crisis. It is a redistribution of risk. And anyone who has watched technology markets long enough will recognize it. Every market has uncertainty. The question is not whether uncertainty exists, the question is who is carrying it.

Right now, AI providers are carrying it through losses. Investors are carrying it through increasingly ambitious valuations. Enterprises are carrying it through spending commitments made before clear ROI frameworks exist. Employees are carrying it through workforce reductions justified by outcomes that have not always materialized.

The risk has not disappeared. It has simply been distributed across the system.

And the redistribution is not finished. Both OpenAI and Anthropic are moving toward public markets, with OpenAI already valued at $852 billion in private markets and Anthropic in early IPO discussions potentially targeting late 2026. In practical terms, this means the next wave of AI infrastructure may be financed by public market investors betting on profitability that the companies themselves don’t expect for three to four years. That is not unprecedented in technology. But it does mean the risk currently carried by providers, institutional investors, and enterprises is about to find a new home. Public market participants will be the next layer in this redistribution, and they deserve to understand what they are being asked to fund.

We Have Been Here Before

In the late 1990s, AOL and Yahoo defined the internet. They had the users, the revenue, the brand recognition, and the infrastructure. What neither of them had was a fundamentally better way to solve the core problem.

Google didn’t win because it had more resources. It won because it approached the problem from a different starting point entirely. PageRank treated hyperlinks as votes of authority rather than simply indexing keywords, and the results were noticeably better at a level ordinary users could feel immediately. That architectural difference, the decision to measure relevance differently at the foundation, made everything that came before look like a prototype. Yahoo’s portal strategy, AOL’s distribution dominance, none of it could compensate for building on a foundation that was less accurate at the thing that mattered most.

BlackBerry had enterprise email locked up. The security, the keyboard, the relationships with IT departments. Apple was a computer company. Until it wasn’t.

The pattern is consistent: early dominance does not protect against displacement when a competitor approaches the core problem from a fundamentally different foundation. What the incumbents have, distribution, enterprise relationships, capital, and brand, buys time. It does not buy permanence.

We are early. The infrastructure of the next chapter of AI is being built right now. Not in the gleaming campuses of the incumbents, but in the places where builders who have seen the structural limits clearly enough to approach the problem from a completely different starting point are doing the work. In garages. In university labs. And in our case, in a basement data center we call Firefly.

Most of what will take this industry forward has not been built yet. That is not a warning. That is the opportunity.

The question for anyone paying attention, whether investor, enterprise leader, or builder, is not whether AI has a future. It is which architecture that future runs on. And the companies currently dominating may not be the ones that define it.

What the Numbers Are Actually Saying

The provider losses are structural, not temporary growing pains. The reason is the same one we explored in the last piece: the transformer model was designed for research, not for the economics of running billions of conversations daily at civilization scale. The architecture creates costs that revenue, however impressive, cannot currently outrun.

Anthropic’s trajectory is meaningfully different from OpenAI’s and worth understanding. The company passed OpenAI in annualized revenue in April 2026, reaching $30 billion against OpenAI’s $24 to 25 billion, having grown 30 times in 15 months.^[ii,iii]

Anthropic projects positive cash flow by 2027, three years ahead of OpenAI. The divergence reflects a fundamental strategic difference: Anthropic’s enterprise-first focus generates more revenue per dollar of training spend. That is a meaningful signal about where the economics of this industry are heading.

But even Anthropic is still losing money. The entire frontier AI industry is, by design, betting that future revenue will justify present losses.

The path to profitability they’re projecting relies on several things happening simultaneously: revenue scaling faster than compute costs, inference getting dramatically cheaper through next-generation hardware, and agentic workflows generating higher revenue per user as AI moves from chat to complex multi-step tasks. It’s a plausible path. Amazon and Netflix took similar roads. But it requires multiple things going right in a market moving faster than any single company can fully control.

The 2029 and 2030 profitability timelines also carry weight for the capital markets question already noted above. As those public offerings approach, the assumptions underlying the valuations, about token costs, competitive dynamics, enterprise adoption curves, and architectural stability, were modeled in a landscape that may look unrecognizable by 2027.^[iv]

In practical terms, the next phase of the AI buildout may be financed by public market investors betting on profitability timelines that the companies themselves acknowledge are three to four years out. That is not unprecedented in technology. Amazon and Netflix lost money for years before the model worked. But those bets were placed in markets where a year felt like a year. In the current AI landscape, a month can feel like a year. That is not cynicism about the technology, but rather an accounting of the nature of the bet being placed. And the people being asked to fund it deserve to understand that clearly.

On the enterprise side, the numbers tell their own story. 92% of enterprises plan to increase AI spending over the next three years. Only 1% consider their AI strategies mature.^[v]

Average enterprise AI spend is projected to reach $11.6 million in 2026, a 65% jump from 2025,^[vi] yet only 29% of organizations report significant ROI from generative AI. Forrester already projects enterprises will defer 25% of planned 2026 AI spend into 2027 as the bills arrive ahead of the returns.^[vii]

Before going further, one thing needs to be clear, because it is the most misread part of this entire story. I am not arguing that AI costs will stay high or that they won’t fall. They are falling, fast, and they will keep falling. The point is stranger than that. Unit costs are dropping and total risk is rising at the same time. Both are true. Understanding why is the whole game.

This is where it gets counterintuitive. Let’s look at the dramatic reduction in token prices. The average cost per million tokens across major providers dropped from roughly $10 to $2.50 in a single year. Epoch AI research suggests inference costs are declining at rates approaching 200 times per year when accounting for both pricing and efficiency improvements. Andreessen Horowitz has coined the term LLMflation to describe this deflationary curve, drawing a parallel to Moore’s Law in semiconductors.^[viii]

NVIDIA’s next generation Rubin platform targets a ten-times reduction in inference costs compared to its current architecture.^[ix]

And yet enterprise AI bills are going up.

The reason is behavioral. When tokens get cheaper, teams run larger prompts, longer agent loops, more tool calls, more retries. A 99.7% token price decline has not reduced total AI spend, because cheaper calls encouraged much larger workflows, and most production costs moved outside the model invoice entirely into orchestration, vector databases, data egress, observability, compliance, and engineering time that never appears on a token bill.^[x]

Sam Altman acknowledged this directly in May 2026, saying the spending problem had become “kind of a meme now” among enterprise clients who had burned through their entire annual AI budgets in the first quarter.^[xi]

The frontier of this paradox sharpened further in June 2026. Anthropic launched Claude Fable 5 on June 9, its most capable publicly available model. Priced at $10 per million input tokens and $50 per million output tokens, double the previous top tier, it carries a one million token context window and up to 128,000 output tokens per request. The early feedback was exceptional. Independent evaluators reported breakthrough performance on complex, long-running tasks. The capability was clearly there.

What the early reviews didn’t capture, because there wasn’t time, is what Fable-class capability costs at scale. At that context window and output capacity, a single complex agentic task doesn’t just cost more per token. It consumes tokens at a scale that makes the LLMflation savings functionally irrelevant. When enterprise teams deploy frontier models on agentic workflows, the bill doesn’t increase linearly. The combination of larger contexts, extended reasoning chains, multi-step tool use, and retry loops means costs compound in ways that no quarterly budget modeled in advance. At frontier capability levels, you generate a great many tokens before you find out whether the task was worth it.

Fable 5 was pulled from general availability on June 12, three days after launch, following a U.S. government export directive. The directive was geopolitical in nature, not a response to performance, safety concerns, or anything the enterprise community did or could have influenced. The question of who controls access to frontier AI capability, under what conditions, and with how much notice, is one this series will return to. What matters here is the trust dimension: the infrastructure enterprises are being asked to build on can be withdrawn overnight, for reasons entirely outside their control, with no committed timeline for return. The bill and the availability are both unpredictable. That is a new kind of risk, and it doesn’t have a line item in most enterprise AI budgets yet.

Tokens are getting cheaper. The bill is going up. The model can disappear. That is the economic reality most enterprises are discovering after the fact.

The experimentation era is over. The accountability era has begun. And the CFO is being asked to make a trillion-dollar category decision with a framework designed for software licensing.

Subscribe now

The Layoff Equation That Isn’t Adding Up

There is a human dimension to this economic story that deserves its own examination. It is the same pattern as everything above, with a human face on it. The risk doesn’t vanish when a role does. It moves. And it tends to land on the people who had the least say in the decision.

Through the first quarter of 2026, approximately 20% of confirmed tech layoffs were explicitly linked to AI and automation by the companies themselves.^[xii]

That is a dramatic increase from 2025, when AI was cited as a factor in fewer than 8% of layoff announcements. The list is extensive and growing. Microsoft cut more than 15,000 workers while its CEO confirmed AI was writing 30% of the company’s code. Amazon cut 14,000 corporate roles citing AI-driven efficiencies. Klarna replaced 700 customer service workers with AI. Citigroup is targeting 20,000 reductions as automation handles middle-office functions.^[xiii]

Here is what the business case for these decisions consistently assumed: that the cost savings from displacement would materialize faster than the costs of transition. In many cases, that assumption is not holding.

A Gartner study published in May 2026 found that while 80% of companies that piloted AI or automation technology reported workforce reductions, the businesses cut jobs regardless of whether the technology was actually generating returns.^[xiv]

They cut first and measured later. In some cases, they are now measuring and finding the math does not work the way the business case promised.

The rehiring costs are real. The institutional knowledge lost is real. The time required to rebuild capability after a premature displacement is real. And there is a cost that almost never appears in an ROI calculation: the damage to organizational trust.

When employees watch colleagues displaced for a technology that doesn’t deliver on its promises, the invisible agreement that the system is worth showing up for begins to fracture. The people who survived the cuts are not relieved. They are watching. And what they are watching is whether the organization they work for makes decisions they can trust.

Most workforce reduction models calculate salary savings. Very few calculate trust loss. Yet trust may be the more expensive variable. When people believe decisions are being made thoughtfully, they will tolerate uncertainty. When they believe decisions are being driven by pressure, trends, or incomplete information, something far more difficult to rebuild begins to erode.

The balance sheet rarely captures that cost. That fracture doesn’t recover through a town hall or a memo from HR. It accumulates. It surfaces in engagement scores and retention numbers and the quality of work from people who are present in body but have mentally begun their exit. This is the third level of trust, organizational trust, being stress-tested in real time by decisions made without full information.

This is what it costs to move faster than the evidence supports.

Risk doesn’t disappear when jobs disappear. It changes hands. The organization may reduce payroll expense, but employees absorb uncertainty. Teams absorb disruption. Leaders absorb credibility risk. And when the expected gains fail to materialize, everyone discovers that the original business case transferred far more risk than it eliminated.

What Comes Next: Reckoning or Reassessment?

Will enterprises use this moment to ask different questions, or will they simply defer spending and return to the same approach when the pressure eases?

My take is that a bifurcation is already underway. Enterprises that found specific high-value use cases and built clear measurement frameworks around them are doubling down. Enterprises that deployed broadly without a strategy are quietly pulling back, trying to figure out what they actually bought. The reckoning isn’t one dramatic moment. It’s already happening incrementally, in budget reviews and board conversations where someone finally asks for the ROI evidence and the room goes quiet.

The ones who use this moment to ask different questions, about architecture, about what AI is actually for, about what responsible deployment looks like, will be better positioned for what comes next. The ones who wait for the technology to improve enough to paper over the current gaps will find themselves in the same conversation again in three years.

This is not a break-the-glass moment. It is an inflection point. And inflection points are where the next chapter gets written.

The companies building that next chapter aren’t optimizing the transformer. They aren’t wrapping existing models in a new interface. They are building relational intelligence infrastructure designed from the ground up for the human and organizational realities that current AI consistently fails to address. Infrastructure that understands that the most expensive token is the one that didn’t need to be generated. That the most valuable AI interaction is the one that builds trust rather than eroding it. That the measure of a successful deployment is not efficiency gained but capability grown, in the people and organizations the technology serves.

That category doesn’t have a widely accepted name yet. That is usually a sign you’re early to something important.

We don’t have to ask if AI is creating value. It is.

What remains less certain is how that value will ultimately be distributed, what risks it depends upon, and who absorbs the consequences when expectations outrun reality.

Every technology wave creates winners. The question is whether it also creates understanding. Right now, we are measuring adoption, investment, valuation, and spending. What I am less certain we are measuring is who is left holding the risk when the assumptions underneath those numbers stop holding.

That is the real reckoning. Not whether AI works. Whether we are asking enough questions before we build the future on top of it.

In the next piece, we look at what happens to the founders and enterprises building on top of the current AI platforms when those platforms consolidate. Because the ground beneath the current buildout is less stable than most people realize.

Trust isn’t built on promises. It’s built on understanding the tradeoffs before the bill arrives.

Because when the returns come, everyone celebrates.

When they don’t, someone still pays.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence, The Trust Story Hiding in Plain Sight

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

Share

Endnotes

ⁱ https://europeanbusinessmagazine.com/sam-altmans-openai-is-burning-billions-most-users-pay-nothing-as-anthropic-closes-in/

ⁱⁱ https://www.the-ai-corner.com/p/anthropic-30b-arr-passed-openai-revenue-2026

ⁱⁱⁱ https://www.saastr.com/anthropic-just-passed-openai-in-revenue-while-spending-4x-less-to-train-their-models/

^iv https://tech-insider.org/anthropic-vs-openai-2026/

^v https://onereach.ai/blog/what-shapes-enterprise-ai-agents-in-the-future/

^vi https://writer.com/blog/enterprise-ai-adoption-2026/

^vii https://bizzdesign.com/blog/enterprise-ai-adoption-balancing-innovation-and-roi-2026

^viii https://www.artefact.com/blog/is-ai-really-getting-cheaper-the-token-cost-illusion/

^ix https://www.investing.com/analysis/the-ai-token-pricing-crisis-behind-openai-and-anthropics-revenue-race-200680777

^x https://www.navyaai.com/reports/ai-cost-report-token-prices-vs-ai-bill

^xi https://www.tomshardware.com/tech-industry/artificial-intelligence/openai-ceo-sam-altman-admits-ai-token-costs-are-becoming-a-huge-issue

^xii https://tech-insider.org/tech-layoffs-2026-ai-workforce-impact/

^xiii https://tech.co/news/companies-replace-workers-with-ai

^xiv https://fortune.com/2026/05/11/ai-automation-layoffs-gartner-study-roi/

This is Part 1 of The AI Reckoning: A Future of Trust Series

There is a trust story hiding in plain sight, and there’s nothing artificial about it.

I am writing this as a builder, not a critic. I am inside this industry, developing what we see as the answer. That context matters because what follows isn’t pessimism or alarm. It’s the view from within, and it’s the same view that led us to build something fundamentally different.

If you’re an investor, an enterprise leader, or someone trying to understand where this is actually going, I’d encourage you to read this carefully. The reckoning is already here.

Subscribe now

There is much to be excited about. What we are experiencing is extraordinary. AI is delivering on promises that would have seemed implausible a decade ago. The capability is real. The momentum is real. The investment flowing into it is the largest concentration of capital in the history of technology.

So is the cost. And it’s hiding in plain sight.

Water. Energy. Infrastructure. Quantities that the business case almost never includes. The bill is real, and it is growing. What is also real is that it is being paid by people who were never given the invoice.

That gap is where trust breaks down.

Trust rarely collapses because people disagree with a decision. More often, it erodes when people discover they were never given the full picture in the first place.

The issue is not that AI consumes water, energy, and infrastructure. Every transformative technology consumes resources. The issue is whether those costs are visible, understood, and included in the decisions being made. When costs remain hidden, confidence becomes fragile. Not because anyone intended deception, but because transparency and understanding failed to keep pace with adoption.

The Physical Reality Nobody Budgeted For

Let’s start with water, because water makes this tangible in a way that terawatts and gigabytes don’t.

AI data centers are currently withdrawing 550 million gallons of water every single day. That’s roughly the same rate as the entire global bottled water industry.ⁱ In 2025 alone, AI data centers broke 264 billion gallons of water withdrawal for the year, the equivalent of the annual water usage of 1.8 million Americans. This was happening while 63% of the United States was experiencing drought conditions as of early 2025.ⁱⁱ

A single Google data center in Council Bluffs, Iowa consumed 1 billion gallons of water in 2024. One data center. One year. That’s enough water to supply all of the state’s residential users for five days.^{iii iv} Microsoft increased its water consumption by 34% in a single year.^[v] The training run for GPT-4 consumed 11.5 million gallons of water in July 2022 alone, in one location, in one month.^vi Accelerated AI adoption alone could result in an additional 4.2 to 6.6 billion cubic meters of water withdrawal by 2027. That is four to six times the annual water withdrawal of Denmark. ^vii

Why so much water? Because of heat. Servers generate extraordinary amounts of it, and water is the most efficient way to absorb and dissipate it. In evaporative cooling systems, water absorbs heat from the servers and converts to steam that is vented into the atmosphere, not returned to the source. It doesn’t disappear from the water cycle, but it leaves the local watershed as humidity, unavailable to the communities and ecosystems that depended on it. The compressor-based alternative uses refrigerant instead of water but requires roughly one watt of cooling for every watt of compute. At the scale of modern data centers spanning hundreds of thousands or millions of square feet, it simply isn’t physically viable.

Water cooling isn’t a greedy choice. At this scale, it is the only viable one.

Here is what that means in practice. Data centers are predominantly sited where water is cheapest and most accessible, which puts them in direct competition with agriculture, municipalities, and the people who live nearby. In much of the American West and Midwest, groundwater aquifers are already being drawn down faster than they recharge. A data center doesn’t just use water, it joins a queue that was already oversubscribed. The communities downstream don’t lose access because of malice, they lose it because the data center got there first, and the permitting process never asked who else was depending on it.

Some will point to desalination as an answer. It is worth examining. Desalination is extraordinarily energy-intensive, which increases electricity demand, which requires more cooling, which loops back to water. It also produces concentrated brine discharge with its own environmental consequences. It is not a solution to this problem. It is a relocation of it.

The more compute, the more heat. The more heat, the more water drawn from watersheds that were never part of the business case. It is a compounding loop, and it scales with every query, every conversation, every token generated. That is not a side effect. That is the deal. The costs are environmental. The consequences are human. And the people absorbing both were never included in the conversation that created them.

Now add energy.

AI’s appetite for electricity is growing even faster than its appetite for water. In 2025, electricity demand from data centers grew by 17%, compared to just 3% growth in global electricity demand. AI-focused data centers climbed even faster. By 2030, electricity consumption from data centers is set to double, and power use from AI-specific facilities is poised to triple.^viii The implications extend far beyond utility bills. New transmission infrastructure, grid upgrades, and power generation capacity are required to support that demand. Those costs don’t disappear. They are distributed across communities, ratepayers, and public infrastructure long before they appear in an AI business case.

The communities surrounding these facilities will feel those demands first. Grid strain, infrastructure upgrades, and rising energy costs are not abstract future concerns. They are the practical consequences of scaling a technology whose resource requirements continue to grow. And we are deploying it at extraordinary speed while still learning what those second-order consequences are.

The Decision That Started All of This

So what’s driving the scale of these demands?

The answer lies in a decision made in 2017. That year, a team of researchers at Google published a paper introducing what they called the transformer. It was a genuine breakthrough. A new way of processing language that outperformed everything that came before it. It became the foundation of every major AI language model that followed. GPT. Claude. Gemini. All of them.ix

That architecture might have remained a research breakthrough, powerful but contained, were it not for a parallel development. Nvidia’s GPU advances, particularly the V100 released that same year and the A100 that followed in 2020, provided the computational infrastructure to train and run these models at previously unimaginable scale.

The transformer gave AI its engine. The GPU gave it a highway. What neither provided was a map of where that highway was going, or who was paying for the road.

The moment those two forces converged in a product the world could touch, everything accelerated. ChatGPT launched publicly in November 2022 and became the fastest-growing consumer application in history within weeks. Investors who had been funding research could suddenly see the destination. Capital followed at a scale the industry had never seen. The largest concentration of investment in the history of technology didn’t begin with a boardroom strategy. It began with a public demo that made the future feel immediate.

Here is the issue. The transformer was designed for research. It was not designed for the economics of running billions of conversations a day at civilization scale. The standard transformer design is dense, meaning every single parameter in the model is activated for every single token it processes. Every word. Every punctuation mark. Every invisible reasoning step happening behind the scenes before a visible response appears. The entire network fires every time, for everything. At research scale, this is manageable. At the scale we are now asking it to operate, the compounding costs of that density are the bill nobody budgeted for.

The Token Economy Nobody Explained to the CFO

Talk is cheap. Tokens are not.

Inference, the process of running a model to generate a response, now accounts for more than 60% of total AI computation among major providers.^x Training a model is a significant but bounded cost. You can plan for it, budget it, and measure it against a defined outcome. Running it at scale is different. Inference costs are ongoing, variable, and increasingly difficult to predict as usage patterns shift, reasoning demands grow, and agentic workflows multiply the token consumption of every interaction. Every word generated costs real energy, real water, real money. That unpredictability is where enterprise budgets are breaking down.

Now factor in reasoning mode.

When AI models engage extended chain-of-thought reasoning, the kind that makes them genuinely better at complex tasks, token consumption increases dramatically. The model thinks through a problem step by step before producing a visible response, and those internal thinking steps generate tokens you pay for even though you never see them. You send 50 tokens, receive 100 tokens back, and are charged for 650 tokens total because 500 reasoning tokens ran silently in between.^xi Agentic workflows involving planning, tool use, and multi-step reasoning have caused token consumption per task to increase 10 to 100 times since late 2023.^{xii xiii}

The hidden multiplier that most enterprise buyers never see until the bill arrives.

Here is the counterintuitive detail that the next piece examines more closely: token prices are actually falling. Dramatically. And enterprise bills are still going up. That paradox is the economic story underneath this one.

For enterprises deploying these systems, the reckoning is already underway. Average enterprise AI spend hit roughly $7 million in 2025 and is projected to jump 65% to $11.6 million in 2026.^xiv Yet only 29% of organizations report seeing significant ROI from generative AI. Forrester predicts a market correction, with enterprises deferring 25% of planned 2026 AI spend into 2027.^xv The bills arrived before the returns did. This is where the trust conversation begins to shift from environmental costs to organizational costs.

When expectations are set higher than outcomes can realistically support, people start searching for explanations. Employees question leadership. Leaders question vendors. Investors question assumptions. Everyone begins looking for someone to blame when the deeper issue may be that the underlying economics were never fully understood. Trust is often lost in the space between promise and reality.

This is not a technology problem in isolation but rather a structural one, rooted in the mismatch between what the underlying model was built for and what we are asking it to do. Structural problems require structural thinking, not more spending on the same foundation. A different foundation entirely.

The Proposed Solutions, and What They’re Missing

The industry is not sitting still, and some of what is being tried is genuinely promising.

Subscribe now

On water, the most significant development is the shift to closed-loop cooling. Microsoft’s Fairwater facility in Wisconsin, one of its most advanced AI campuses, has filled its cooling system with water once during construction and now recycles it continuously. Microsoft CEO Satya Nadella claimed at Microsoft Build 2026 that the facility’s annual water consumption is comparable to that of a single local restaurant. Google has pledged to replenish 120% of the water it consumes by 2030.^xvi These are meaningful moves in the right direction.

But here is the context that can be easy to miss: these innovations apply to new facilities. The vast majority of existing data center infrastructure uses evaporative cooling and will continue doing so for years. And even the most water-efficient new designs still require enormous energy, which brings us to the bigger question.

On energy, nuclear is the most credible long-term answer being pursued at scale. Microsoft is reviving Three Mile Island. Amazon has secured a 17-year nuclear power purchase agreement. Meta has partnered with Oklo for a 1.2 gigawatt campus in Ohio with 16 small modular reactors.^xvii The logic is sound: nuclear provides reliable, around-the-clock, low-carbon power that renewables alone cannot guarantee at the scale AI demands.

But here is what is not being said alongside the enthusiasm.

Nuclear power plants are among the most water-intensive energy sources that exist. They consume 20 to 83% more water than coal-fired plants of the same capacity. A typical 1,000 megawatt reactor requires roughly 35 to 65 million liters of water per day for cooling, depending on design.^xviii Nuclear plants have already been forced to reduce output or shut down during drought conditions in Europe and the United States because river temperatures rose too high or water levels fell too low.^xix

We are proposing to solve AI’s water problem with an energy source that has its own significant water dependency. That is not a solution to the water crisis.

Nuclear also carries vulnerabilities that deserve acknowledgment: unresolved waste disposal, physical security and cyber risk when reactors are concentrated near high-value data campuses, and regulatory safeguards being quietly dismantled to accelerate deployment. Those safeguards were built from the hard lessons of Three Mile Island, Chernobyl, and Fukushima. Removing them in the name of speed is precisely the kind of second-order consequence we should be paying attention to.^xx

Nuclear is not the wrong answer. It is an incomplete one.

Then there is the space proposal. In early 2026, Elon Musk merged SpaceX and xAI and announced a vision for one million orbital, solar-powered data centers circling the Earth. Others have suggested similar futures. At first glance, it sounds elegant. Move computation into space where solar energy is abundant and many of the terrestrial constraints disappear. The physics of low Earth orbit communication are more viable than science fiction suggests, and the solar energy argument has genuine logic behind it.

But I find myself wondering whether this is another example of a question we are not asking. The challenges facing AI are not answered by simply changing where we place the infrastructure. Moving data centers into orbit does not change the amount of computation required. It does not change the underlying architecture. It does not change the growing demand for tokens, reasoning, and increasingly complex AI workloads. What it does is relocate the consequences.

Low Earth orbit currently contains over 14,000 satellites and an estimated 120 million debris fragments, all traveling at roughly 27,000 kilometers per hour.^xxi A January 2026 study calculated that a complete loss of satellite command for just 24 hours carries a 30% chance of triggering Kessler syndrome, the cascade of collisions that renders entire orbital shells permanently unusable.^xxii We do not have a solution to space debris at scale. This is not for lack of effort. Serious work is underway on active debris removal, controlled de-orbit and collision avoidance, and some of it is promising. But the question that matters is one of sequence. Does that mitigation mature and deploy at the speed orbital compute is being proposed, or does the infrastructure go up first while the solutions are still being built? Confidence that the answer will arrive in time is not the same as the answer having arrived.

Removing a single 95 kilogram satellite currently costs 86 million euros.^xxiii Introducing one million orbital data centers into an already critically congested environment is not a bold engineering vision. It is a proposal that carries risks the global space community is already warning about. A cascade failure in low Earth orbit would not just affect AI infrastructure, it would threaten GPS, weather systems, global communications, and human spaceflight for generations.

It is a bold vision. It is still not the answer the future requires. And depending on how it is pursued, it could make several other problems significantly worse.

Second Order Consequences

History is filled with examples of technologies that worked exactly as designed while producing consequences nobody anticipated. The most important decisions are often not about whether something can scale, but whether the foundations beneath that scale remain sustainable once second-order effects begin to emerge.

Can we create systems capable of extraordinary intelligence while remaining accountable to the people, communities, and resources they depend upon?

The conversation about what comes next is happening. It is just not happening loudly enough given the scale of investment already committed to the current path. And that is partly because the people most invested in the transformer model have the least incentive to ask the question seriously.

This is what it looks like to deploy technology before we understand its second-order consequences. Not malice, institutional inertia dressed up as progress. When capital commits at this scale, it doesn’t just fund a direction, it forecloses questions. The people closest to the problem can develop a vested interest in preserving the assumptions that brought them this far because the weight of what has already been built makes certain questions feel too costly to ask. That is something more dangerous than deception. It is the point at which bias and incentive become indistinguishable from each other. It becomes a reaction when what is needed is a pause and a deliberate response that is oriented toward the future. Not scaling a broken system but considering a redesign.

Why do we keep responding to consequences by scaling the system that created them, rather than pausing to consider a better path forward? That is ultimately a trust and leadership question.

What the Future Requires

AI will continue to advance. The upsides are too promising. The deeper question is whether we are willing to challenge the assumptions that got us here.

The rethinking is already happening, quietly, among researchers and builders like us, asking a different question than the one that dominated the last decade. Not how do we power the transformer at greater scale, but how do we build something that inherently makes compute more efficient and responsible as we scale.

Yann LeCun, former Chief AI Scientist at Meta, has argued publicly for years that the transformer is the wrong foundation for the next generation of AI and left Meta in late 2025 to pursue his Joint Embedding Predictive Architecture full time. ^xxiv State Space Models like Mamba offer near-linear computational complexity compared to the transformer’s quadratic demands, reducing the resource requirements of processing long sequences significantly. ^xxv As of late 2025, several of these alternative approaches have transitioned from theoretical research to production deployment and are matching or surpassing transformer performance on key benchmarks. ^xxvi

I mentioned at the beginning of this article that I am a builder. My co-founder and I did not start with the transformer and try to make it cheaper. We started with the problem: how do you build AI that is relational, explainable, and architecturally efficient from the ground up? That question led us to build a fundamentally different architecture. Not a wrapper. Not an optimization. Designed for the cost and resource realities of the world we are actually in, not the research environment of 2017. I am not ready to say we have solved everything. But I am ready to say, with confidence, that the path forward exists and that it will not look like a more expensive version of what we have now.

In the next piece, we go deeper into AI economics. Because the cost of inference is not just an environmental story. It is a business model story. And the math is stranger than most people realize.

AI is here to stay. And so are we. The only question that matters for the future is whether we build it in a way that remains sustainable, accountable, and worthy of the trust being placed in it.

We’re at an inflection point. Eyes wide open is not pessimism, it is the only responsible way to build.

Read more from The AI Reckoning: A Future of Trust Series

Part 1: The Hidden Cost of Intelligence

Part 2: The Reckoning Behind the Revenue: When the Numbers Don’t Add Up

Part 3: Rented Intelligence: Building on Borrowed Ground

Part 4: The Fragmentation Tax: Death by a Thousand Tools

[i] https://www.barchart.com/story/news/2339834/ai-data-centers-water-consumption-breaks-264-billion-gallons-in-2025

ii https://droughtmonitor.unl.edu / https://www.theinvadingsea.com/2025/09/05/data-center-water-consumption-google-meta-amazon-microsoft

[iii] https://www.theinvadingsea.com/2025/09/05/data-center-water-consumption-google-meta-amazon-microsoft-digital-realty-equinix-cooling-system/

[iv] https://www.theinvadingsea.com/2025/09/05/data-center-water-consumption-google-meta-amazon-microsoft-digital-realty-equinix-cooling-system/

[v] https://www.datacenterdynamics.com/en/news/microsofts-water-consumption-jumps-34-percent-amid-ai-boom/

[vi] https://malotastudio.net/ai-data-centre-water-usage/

[vii] https://www.weforum.org/stories/2025/11/data-centres-and-water-circularity/

[viii] https://www.iea.org/news/data-centre-electricity-use-surged-in-2025

[ix] https://arxiv.org/abs/1706.03762

[x] https://arxiv.org/pdf/2603.21690

[xi] https://medium.com/@dhevanmuhamad/understanding-tokens-the-currency-of-ai-thats-costing-you-money-87a87cd0757d

[xii] https://adam.holter.com/ai-costs-in-2025-cheaper-tokens-pricier-workflows-why-your-bill-is-still-rising/

[xiii] https://www.mindstudio.ai/blog/ai-token-cost-crisis-enterprise

[xiv] https://writer.com/blog/enterprise-ai-adoption-2026/

[xv] https://bizzdesign.com/blog/enterprise-ai-adoption-balancing-innovation-and-roi-2026

[xvi] https://introl.com/blog/water-usage-efficiency-wue-ai-data-center-cooling-guide-2025

[xvii] https://www.datacenterknowledge.com/energy-power-supply/how-realistic-is-nuclear-power-for-ai-data-centers

[xviii] https://www.justsecurity.org/138215/nuclear-powered-ai-risks-deregulation/

[xix] https://www.sciencedirect.com/science/article/pii/S0301421525001387

[xx] https://www.justsecurity.org/138215/nuclear-powered-ai-risks-deregulation/

[xxi] https://amplyfi.com/blog/understanding-the-space-debris-dilemma-the-kessler-syndrome/

[xxii] https://www.sciencedaily.com/releases/2026/01/260128075341.htm

[xxiii] https://medium.com/@marc.bara.iniesta/space-debris-and-the-kessler-problem-a-reality-check-99484cfa5e86

[xxiv] https://cacm.acm.org/news/beyond-llms-a-post-transformer-world-emerges/

[xxv] https://www.researchgate.net/publication/399331454_Architectural_Pluralism_in_AI_A_Comprehensive_Analysis_of_Alternatives_to_the_Transformer_Architecture

[xxvi] https://pchojecki.medium.com/going-beyond-llms-transformers-39f3291ba9d8

I’ve spent the last two years in conversations with founders, executives, investors, technologists, and employees trying to make sense of what AI is about to change inside organizations and inside our lives.

What strikes me most is not that things are moving quickly. It’s that moving quickly has become the strategy.

As a founder, I understand why. Speed is being rewarded everywhere right now. Investors are rewarding it. Markets are rewarding it. Companies are rewarding it. Nobody wants to be the organization that missed the shift.

The pressure to adopt AI, show measurable ROI, and demonstrate momentum is enormous. And underneath that pressure is another layer that people talk about far more quietly: the fear executives carry about falling behind, making the wrong bets, or losing relevance if they cannot prove results fast enough.

At the same time, I’m seeing another side of the story emerge inside organizations.

Adoption initiatives that are quietly failing. Employees struggling to keep pace with constant change while trying to maintain performance. Leaders mistaking compliance for alignment. Teams carrying cognitive and emotional loads that traditional productivity metrics do not capture.

And increasingly, I find myself asking whether we are measuring the wrong things while assuming we are making progress.

That’s what led me to this series.

Because underneath the headlines about efficiency, automation, and competitive advantage is a much larger question about trust.

Trust in leadership.
Trust in institutions.
Trust in information.
Trust in our own value and judgment.
Trust that the systems being built are actually serving humans rather than simply accelerating expectations beyond what humans can sustainably absorb.

There’s a difference between speed and velocity.

Speed tells you how fast you’re moving. Velocity tells you how fast you’re moving and where you’re going.

Right now, we have extraordinary speed. What I’m less certain about is whether we’ve spent enough time asking where all of this is actually leading us.

Subscribe now

The AI Reckoning isn’t a technology series. It’s a trust series that happens to be using AI as its lens right now, because AI is where the trust breakdown is most visible, most consequential, and most urgently under-examined.

• The physical costs nobody accounted for.

• The economics that don’t add up the way the industry is presenting them.

• The adoption failures being misreported as adoption successes.

• The security and governance frameworks running years behind the deployment curve.

• The ambient erosion of human confidence that the productivity numbers aren’t capturing.

Each of these is a trust problem, and none of them exist in isolation.

But here’s what I keep coming back to as the thread underneath all of it:

That responsibility isn’t abstract. It shows up in the decisions being made right now by enterprises, investors, founders, and policymakers about what gets built, how fast, for whom, and at what cost. And it shows up in something less visible but equally important: The quiet confidence crisis unfolding inside the people being asked to keep pace with all of it.

Which brings me to the part of this conversation I find most urgent and least examined.

Increasingly, I’m seeing younger generations ask questions that many organizations seem reluctant to confront. Not because they don’t understand the technology. Because they understand it clearly enough to see what’s being traded away.

Authenticity.

Creativity.

The sense that their work means something and that they are the ones doing it.

They are pushing back against a future they didn’t design and aren’t sure they want. That pushback is often being misread as resistance to change. I think it’s something more strategic than that. I think it’s a generation that has grown up with enough information to ask a question the rest of us have been too busy to ask:

They are right to ask it, and we have a responsibility to answer it, not by handing them a roadmap we drew without them, but by bringing them into the design process. You cannot build systems people will trust if the people most affected had no hand in building them. That’s not idealism. That’s how trust actually works.

This series will go into all of it. Some pieces will be reported and researched. Some will be more exploratory. Questions I’m sitting with that I’ll think through out loud and invite you into. I’ll tell you which is which.

What it will always be is a genuine attempt to ask the questions that are getting drowned out by the noise of the ones we’re already answering.

The reckoning isn’t coming. It’s already here. We’re just not calling it that yet.

I’d like this series to be a conversation, not a monologue. If you’re seeing questions that aren’t being asked, assumptions that deserve more scrutiny, or perspectives that aren’t getting enough airtime, I’d love to hear from you. Leave a comment, join the chat, or reply directly.

The future is being shaped right now. Let’s make sure we’re asking the right questions before we decide where we’re going.

Subscribe now