When Seeing Is No Longer Believing: Trust Under Attack
While human trust struggles to keep pace with AI, something else is falling even further behind: security. And the gap between those two speeds is where the next reckoning is already forming.
This is Part 8 of The AI Reckoning: A Future of Trust Series
It’s just after two in the afternoon when Linda’s phone rings.
She recognizes her daughter’s number immediately. Odd, she thinks. She should be in class.
Before she can say hello, she hears her daughter’s unmistakable voice. Frantic. Pleading. Terrified.
“Mom! Help me! They’ve taken me...”
There’s a crackling, and before she can get a word out a man’s voice comes on the line.
“We have your daughter. If you want to see her alive again, you’ll do exactly what we say.”
Linda’s heart is racing. Her mind is trying to catch up, but panic is moving much faster than reason. She begs to speak to her daughter again. Instead, the man sends her a photograph. It’s her daughter. Bound. Terrified. Alive. He gives Linda thirty minutes to wire the money.
No police. No questions. No delays.
In another part of town, her daughter is sitting in class, completely unaware that her mother is living through every parent’s worst nightmare.
The voice wasn’t hers. The photograph wasn’t real. Neither was the kidnapping.
But the fear was.
That story has played out in different forms across the country, and across the world. Criminals are using artificial intelligence to clone voices from a few seconds of audio posted online, generate convincing images that never existed, and create enough manufactured evidence to overwhelm the instincts we’ve trusted our entire lives. By the time someone realizes they’ve been deceived, the money is often gone, and something much larger has been stolen along with it.
Confidence.
Linda’s story is not an outlier. It is a preview. The FBI’s Internet Crime Complaint Center broke out AI-enabled fraud as its own category for the first time in 2025, and even by the most conservative, fully audited count, it logged nearly 900 million dollars in losses across more than 22,000 complaints in a single year.[i] Ask the people defending against it directly and the picture sharpens further. In a 2025 survey of more than 300 security leaders, well over half said their organization had already experienced a deepfake-enabled attack in the past twelve months.[ii] Not a phishing email. Not a suspicious link. A fabricated voice, face, or video, good enough to move money, unlock a system, or end a reputation before anyone thought to question it.
For most of human history, we rarely questioned what our senses told us. If we heard someone’s voice, we assumed we knew who was speaking. If we looked at a photograph, we believed it captured a moment in time. If someone showed us proof, we accepted it as evidence. Those assumptions quietly became the foundation beneath our relationships, our businesses, our legal system, and our society. We didn’t consciously think about them because we didn’t have to.
Today, we do.
There used to be an old saying that a lie could travel halfway around the world before the truth put on its boots. The danger was that falsehood spread faster than facts. Artificial intelligence has changed the equation.
A lie still travels faster than the truth, but now wears the truth’s face.
And increasingly, manufactured evidence is riding shotgun.
That shift represents something much larger than a new cybersecurity threat. It represents a fundamental change in how trust is formed. The challenge is no longer that misinformation spreads quickly. The challenge is that convincing evidence can now be manufactured just as quickly.
For centuries, seeing was believing. Today, seeing has simply become the beginning of the investigation.
I believe we’re entering a new chapter in the AI revolution, one that has received far less attention than productivity gains, billion-dollar valuations, or ever more capable models. While we’ve been captivated by what artificial intelligence can create, we’ve spent far less time considering what happens when reality itself becomes negotiable.
This is where the conversation about security begins to change. For decades, cybersecurity was largely about protecting information. We built firewalls to keep intruders out, encrypted data to keep it private, and trained employees to recognize suspicious emails. The goal was to prevent someone from gaining access to systems they weren’t authorized to enter.
Those threats haven’t disappeared. They’ve evolved. The next generation of attacks isn’t simply trying to steal our information. It’s trying to manipulate our perception. Instead of breaking into the system, it breaks into our confidence. It doesn’t need to crack a password if it can convince a CFO that the CEO’s urgent voicemail is authentic. It doesn’t need to hack a bank account if it can persuade a grandparent that their grandchild is crying on the other end of the phone. It doesn’t need to alter history if it can fabricate convincing evidence before the truth has a chance to catch up.
This is the dual-speed paradox emerging throughout the AI era. Artificial intelligence is accelerating the pace at which threats can be created, personalized, and deployed. Security, regulation, education, and governance are improving as well, but they are moving at a fundamentally different speed. One side learns in milliseconds. The other learns through investigations, legislation, standards, training, and experience. The gap between those two speeds is where trust begins to erode. And trust, once eroded, is extraordinarily difficult to restore.
The New Trust Economy
Organizations have spent decades building cybersecurity programs designed to protect data. Today, many of those same organizations are discovering that data may not be the asset most at risk.
Trust is.
A finance department can have world-class encryption and still authorize a fraudulent wire transfer because the request appeared to come from the CEO. A hospital can protect millions of patient records while a clinician receives a convincing voice message from someone they believe is a trusted colleague. A law firm can secure every document in its possession yet still find itself questioning whether a video submitted as evidence is authentic.
The attack surface has changed. Increasingly, the target isn’t the system. It’s the human making the decision. That’s an important distinction because it changes where organizations invest their attention. Firewalls remain essential. Encryption remains essential. Identity management remains essential. But none of those technologies can fully protect an organization if the people inside it can no longer trust the evidence placed in front of them.
A lie still travels faster than the truth, but now wears the truth’s face.
And increasingly, manufactured evidence is riding shotgun.
Security is no longer just a technology problem. It’s become a human judgment problem.
The Governance Lag
Ask a room full of security leaders how ready they feel for tomorrow, and most of them will not say ready. A 2026 survey of more than 600 senior cybersecurity decision makers found that despite near universal adoption of formal incident response plans, most did not believe their organization could execute under pressure if a significant attack happened tomorrow.[iii] That may sound like a confidence problem, but in actuality it is a readiness problem. Capacity exists. Trust in that capacity does not.
The data behind that gap is specific. Sixty-three percent of organizations that experienced a breach in the past year had no formal AI governance policy in place. Among those that did, fewer than half had an approval process for new AI deployments, and most lacked the technology to audit AI use once it was underway.[iv] When shadow AI, meaning employees quietly using AI tools the organization never approved or reviewed, was a factor in a breach, it added an average of $670,000 to the cost of that breach and made the exposure significantly harder to contain.[v] Ninety seven percent of organizations that suffered an AI-related breach admitted they lacked basic access controls for the AI systems involved.[vi] These are not organizations that ignored security. They are organizations that moved at the speed the market rewarded and discovered, after the fact, that governance was not optional.
Security teams know this. A 2026 industry survey found that AI adoption inside cybersecurity itself had accelerated faster than in any prior year, with the large majority of teams now using AI for defense, and yet only a small fraction described their deployments as mature. Most of those same teams had been handed formal responsibility for governing AI across their organization without the audit frameworks to do it.[vii] The people closest to the problem are not behind because they are careless. They are behind because the pace of adoption and the pace of oversight were never designed to move together.
This is the dual-speed paradox again, this time inside the institutions meant to catch it. Security operations are learning to detect AI-generated threats in something close to real time. Governance, budget approval, staff training, and regulatory response still move at the pace of committees, fiscal years, and legislative sessions.
The Arup engineering firm learned what that gap costs directly. In January 2024, a finance employee in the company’s Hong Kong office joined what he believed was a video call with the firm’s CFO and several colleagues, all of them appearing and sounding exactly as he remembered them. Every person on that call except him was an AI-generated fabrication, built from publicly available footage of real executives. He authorized fifteen transfers totaling more than twenty-five million dollars before anyone realized the call had never been real.[viii] No firewall failed that day. No password was cracked. The system worked exactly as designed. It was the human judgment inside it that had nothing left to verify against.
The attack surface has changed. Increasingly, the target isn’t the system. It’s the human making the decision.
What makes this harder to solve than a typical security gap is that the response itself carries risk. Regulators and organizations are moving quickly to close the AI governance gap, and quickly is not always the same as carefully. New identity and access frameworks, new detection mandates, and new compliance requirements are being built under pressure, often without time to fully understand what those safeguards will change about how people actually work, or what new blind spots they might quietly introduce.
We saw this same pattern in Piece 1. Nuclear power was proposed to solve AI’s energy problem and turned out to be one of the most water-intensive power sources that exists. Orbital data centers were proposed to solve the land and cooling footprint and introduced a real risk of cascading satellite collisions. Both were fast, visible fixes that outran a careful look at what they would break somewhere else. Security governance is now walking the same path.
The dual-speed paradox does not resolve just because the slower side starts moving. It resolves when both sides are moving deliberately, and right now, very little about this moment is deliberate.
The Verification Tax
Every technological revolution creates new efficiencies. This one is also creating new friction. Every suspicious phone call requires a return call to verify. Every unexpected invoice needs another layer of confirmation. Every extraordinary photograph demands additional scrutiny. Every urgent request from an executive may require a second channel of authentication before action can be taken.
Individually, those moments seem insignificant. Collectively, they represent a growing tax on society. Not a financial tax. A verification tax.
We pay it in time, attention, emotional energy, and delayed decisions. We pay it every time we pause before answering a loved one’s call. Every time a business slows a critical decision to confirm that the request is legitimate. Every time a journalist spends hours authenticating media that, only a few years ago, would have been accepted at face value.
Ironically, artificial intelligence promises unprecedented speed while simultaneously forcing us to slow down. The faster AI becomes at creating convincing deception, the more deliberate we must become before believing it. That is the adaptation we are required to make if the truth still matters.
Building Trust, Not Just Detecting Lies
If this sounds bleak, it shouldn’t. History suggests that when trust is disrupted, humans don’t abandon it. We rebuild it. When commerce first moved online, many people hesitated to enter their credit card information into a website. Over time, encryption, secure payment systems, and digital certificates became part of the invisible infrastructure that restored confidence. Today, most of us rarely think about the technology working quietly behind every online purchase.
Artificial intelligence is forcing us through another one of those transitions. Around the world, researchers, governments, technology companies, and standards organizations are working on ways to authenticate digital content before deception occurs. Rather than teaching machines to spot every forgery, we’re beginning to create ways for authentic content to carry its own credentials. That work is still evolving, and no single solution will eliminate deception. Just as cybersecurity became a layered discipline rather than a single product, trust in the AI era will require layers of technology, governance, education, and human judgment working together.
The Coalition for Content Provenance and Authenticity (C2PA), whose members include organizations such as Adobe, Microsoft, the BBC, Nikon, Canon, Sony, and others, is developing standards that allow digital content to carry verifiable information about where it originated and whether it has been altered. Adobe’s Content Credentials initiative is already bringing many of those capabilities into creative tools, allowing creators to preserve a transparent history of how content was produced.[ix]
Ironically, artificial intelligence promises unprecedented speed while simultaneously forcing us to slow down.
The faster AI becomes at creating convincing deception, the more deliberate we must become before believing it.
Government agencies are also recognizing the challenge. The National Institute of Standards and Technology (NIST) has incorporated authenticity, governance, and AI risk into its evolving frameworks, acknowledging that trust in AI will require more than technical performance. It will require confidence in provenance, accountability, and human oversight.[x]
This represents an important shift. For decades, much of cybersecurity focused on authenticating people. Passwords. Multi-factor authentication. Identity management. Zero Trust architectures all evolved around one central question:
Can we trust the person requesting access?
Increasingly, we’ll be asking a different question.
Can we trust the content itself?
In the years ahead, authenticity may become something that travels with information rather than something we attempt to reconstruct after the fact. It won’t eliminate deception. Nothing ever has. But it changes the conversation from detecting every counterfeit to helping the genuine prove itself.
When the Cure Changes the Patient
Building better trust infrastructure is essential, but so is recognizing that every solution changes the system it was designed to protect.
Economist Charles Goodhart observed this decades ago in what has become known as Goodhart’s Law:
When a measure becomes a target, it ceases to be a good measure.
The principle appears in economics, education, healthcare, and organizational performance. Once people understand how they’re being measured, they naturally begin adapting to the measurement itself.
Deepfake detection is already running into this. Security teams build classifiers trained to spot the tells of synthetic video, audio, and written material, subtle artifacts in lighting, blink patterns, and waveform irregularities. Those tells get published in research papers, discussed at conferences, and eventually built into commercial detection products. Attackers read the same papers. Once a detector’s method becomes known, it becomes a target to train against, and the next generation of deepfakes gets built and tested specifically to slip past it. Independent testing already shows the gap. The first benchmark built to test detection tools against deepfakes actually circulating online, rather than clean academic datasets, found leading models’ accuracy dropped by roughly 45 to 50 percent across video, audio, and image detection compared to their performance on older benchmarks.[xi]
The detector is not simply observing the threat. It is shaping what the threat becomes.
Accuracy is also a concern. False positives don’t simply create technical errors. For authentic creators they can create reputational damage that lingers long after the software has been proven wrong. Charles Kent, a technologist and Substack writer with a career spent across the AI and technology industry, has described this as a version of spectral evidence, the term Salem’s courts used for testimony about something only the accuser could see, which the accused had no way to disprove.[xii] An AI-detection flag works the same way. You cannot produce an alibi for your own writing process. For a journalist, attorney, researcher, executive, or author, credibility is often their most valuable professional asset, and an incorrect accusation of AI-generated work doesn’t simply question a single document. It can quietly cast doubt over years of legitimate effort. When people begin editing themselves to satisfy an algorithm rather than communicate authentically, something subtle changes. Authenticity itself risks becoming a performance rather than an expression.
Once a detector’s method becomes known, it becomes a target to train against, and the next generation of deepfakes gets built and tested specifically to slip past it.
Every safeguard has consequences. That doesn’t mean we shouldn’t build safeguards. It means we should design them with humility, recognizing that every intervention changes the people living inside the system. In our effort to protect trust, we have to be careful not to manufacture new forms of distrust.
Trust Has Always Been Human
Linda’s story wasn’t really about artificial intelligence. It was about being human. It was about the instinct to protect someone we love. The willingness to act before we have all the facts. The extraordinary speed with which fear can override judgment. Technology didn’t create those instincts. It exploited them.
Trust does not live inside software or algorithms. It lives inside people. Business leaders will continue investing in stronger cybersecurity. Governments will continue developing standards. Technology companies will continue improving authentication, provenance, and verification. All of that work is necessary.
But trust has never been built by technology alone. It has always been built through judgment, relationships, and credibility earned over time. Artificial intelligence is asking us to become more deliberate. To verify more carefully. To question evidence that once seemed unquestionable.
It feels uncomfortable.
Every generation inherits a different version of that challenge. Our grandparents learned to lock their doors. Our parents learned not to believe every email. Our children may grow up learning that seeing something is no longer sufficient to believe it. Perhaps it’s the evolution of trust.
The future won’t belong to those who trust less.
It will belong to those who learn to trust more wisely.
Endnotes
[i] Federal Bureau of Investigation, “Cryptocurrency and AI Scams Bilk Americans of Billions,” 2025 Internet Crime Report https://www.fbi.gov/news/press-releases/cryptocurrency-and-ai-scams-bilk-americans-of-billions
[ii] Gartner, “Gartner Survey Reveals Generative Artificial Intelligence Attacks Are on the Rise,” September 2025 https://www.gartner.com/en/newsroom/press-releases/2025-09-22-gartner-survey-reveals-generative-artificial-intelligence-attacks-are-on-the-rise
[iii] Sygnia, 2026 CISO Survey: The State of Incident Response Readiness, April 2026. https://www.businesswire.com/news/home/20260413646028/en/73-of-CISOs-Unprepared-for-the-Next-Big-Cyber-Attack-Incident-Response-Readiness-Report-Reveals
[iv] IBM and Ponemon Institute, Cost of a Data Breach Report 2025 https://www.ibm.com/reports/data-breach
[v] IBM, “2025 Cost of a Data Breach: Navigating the AI Rush Without Sidelining Security.” https://www.ibm.com/think/x-force/2025-cost-of-a-data-breach-navigating-ai
[vi] IBM Newsroom, “IBM Report: 13% of Organizations Reported Breaches of AI Models or Applications,” July 2025. https://newsroom.ibm.com/2025-07-30-ibm-report-13-of-organizations-reported-breaches-of-ai-models-or-applications,-97-of-which-reported-lacking-proper-ai-access-controls
[vii] SANS Institute, 2026 AI Survey Insights Report, July 2026 https://www.intelligentciso.com/2026/07/15/sans-report-highlights-growing-ai-governance-gap-in-cybersecurity/
[viii] CNN Business, “Arup Revealed as Victim of $25 Million Deepfake Scam Involving Hong Kong Employee,” May 2024 https://www.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk
[ix] Coalition for Content Provenance and Authenticity, published standards
https://c2pa.org
[x] National Institute of Standards and Technology, AI Risk Management Framework https://www.nist.gov/itl/ai-risk-management-framework
[xi] Chandra, N.A. et al., “Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024,” arXiv, March 2025 https://arxiv.org/abs/2503.02857
[xii] Charles Kent, “Substacks AI Witchhunt,” Agent Autopsies, July 22, 2026.