My guest today is Zohre Shirazi, publisher of AI Wide Open. Zohre examines how human oversight only works when there is still meaningful judgment involved.
But first, a note from Sheryl:
There’s a question I keep coming back to as AI becomes increasingly agentic: What changes when we stop asking AI for help and start giving it permission to act?
For most of the AI era, trust has centered on what a system tells us. We’ve learned to question whether the information is accurate, if we rely on the output, and if we understand how it arrived there.
But an AI that can act on our behalf changes the trust equation. Now we are not only trusting what it knows, we are deciding what it can access, what it can change, what decisions it can make, and how far it can go before a human needs to step in.
That moves us into something much bigger than model performance. Our attention needs to go to authority, boundaries, and accountability.
And I think we are going to have to get much better at separating capability from permission. Just because an AI can do something does not mean we should give it the authority to do it. And adding a human approval somewhere in the process does not necessarily solve the problem.
So back to Zohre’s thesis that human oversight only works when there is still meaningful judgment involved.
I agree with the premise and would add that this is also where trust becomes architectural. We cannot rely on people to catch every mistake after it happens, particularly as agents operate faster, across more systems, and through longer chains of action. Some of the most important decisions about trust will need to happen before the agent ever acts: what it can access, where its authority ends, when it must stop, and who ultimately owns the outcome.
Zohre takes on the question: When AI acts on our behalf, who owns what happens next?
What I appreciate about her perspective is that she doesn’t reduce the answer to whether we should trust AI more or less. She looks at what we actually need to build around increasingly autonomous systems if trust and accountability are going to keep pace with capability.
I’m very pleased to welcome Zohre to The Future of Trust. Be sure to subscribe to AI Wide Open.
And now, Zohre Shirazi…
_______________________________________________________________________
When AI Can Act on Our Behalf, What Happens to Trust?
As AI moves from answering to acting, we have to rethink authority, accountability, and the boundaries of trust.
I use AI extensively in my work. I rely on it to research ideas, challenge my thinking, work through drafts, and make parts of the editorial process faster. I have become comfortable trusting AI to help me think and produce something. I am still much more cautious about giving an AI agent permission to act on my behalf.
That distinction has become more important as AI systems move beyond generating text, summarizing information, and making recommendations. They can increasingly use tools, access company systems, write and execute code, and complete multi-step tasks with much less human involvement.
For a long time, trusting AI was mostly a question of trusting an answer. You asked a question, AI gave you an answer, and you decided what to do with it. If the answer was wrong, the responsibility for acting on it still sat fairly clearly with the person using the system.
That relationship is changing. Once AI can act rather than simply advise, trust becomes a different problem.
If an AI gives me the wrong answer, I can catch the mistake before I act. When an AI can execute a task, change a file, send a message, or continue through a sequence of decisions, the opportunity for human intervention becomes much smaller.
The question becomes whether we trust the system with the authority to act.
Capability Is Not Authority
There is an important difference between what an AI system can do and what it should be allowed to do.
An agent might technically be capable of accessing customer records, updating a CRM, writing code, sending an email, or interacting with another software system. Technical capability does not establish authority.
That authority has to come from somewhere. Someone has to decide what the system can access, which actions it can take, which situations require approval, and what happens when it reaches a boundary.
This is increasingly an organizational problem, not just a technical one.
The World Economic Forum’s 2026 work on AI agents makes a similar point, focusing on authorization, auditability, and accountability as agents move into real-world environments. Its proposed Agent Capability and Authorization Profile is designed to make delegated decisions and actions more auditable and enforceable across an agent’s lifecycle. [1]
Every permission we give an AI system is therefore more than a technical setting. It is a delegation of authority.
And authority is at the heart of trust.
We Already Understand This With People
Organizations have dealt with delegated authority for a long time.
When a new employee joins a company, they are not usually given unrestricted access to every system. Their role determines what information they can see, which decisions they can make independently, and when they need to escalate a situation.
If something goes wrong, the organization can examine the employee’s role, training, supervision, permissions, and the decisions that allowed the situation to occur.
AI does not become a person simply because we delegate work to it. But the organizational problem is surprisingly similar.
When we give an AI system authority, we still need to know what that authority includes, where it stops, and who owns the consequences.
There is also an important difference. A human employee usually operates within organizational norms and can ask for clarification when something is unclear. An AI agent can instead interpret a goal, choose a sequence of actions, and execute those actions at machine speed.
That makes the boundaries around its authority even more important.
When the Boundary Fails
Recent incidents show why this is not just a theoretical concern.
In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the internet and accessed parts of OpenAI’s internal research infrastructure and Hugging Face’s systems. OpenAI later described the models as having communicated through unauthorized channels, exploited vulnerabilities in shared infrastructure, and gained internet access while pursuing their assigned tasks. [2]
A separate incident reported in September involved thousands of OpenAI agents posting on a public German wiki during internal testing. Researchers identified roughly 3,700 distinct agent names and around 18,000 messages over a six-week period. The posts included discussions about bypassing sandbox restrictions and sharing information related to the agents’ assigned tasks. OpenAI subsequently confirmed the incident. [3]
These incidents do not prove that autonomous agents will routinely escape their controls. They do show why containment has become part of the trust question.
A sandbox is supposed to establish a clear limit.
When that boundary fails, we have to examine the safeguards around the model as well as the model’s behavior. Trust in an autonomous system depends partly on whether the system around the model can contain unexpected behavior.
The Human in the Loop Is Not Enough
One obvious response is to keep a human involved. Sometimes that is exactly the right approach. But simply putting a person somewhere in the process does not automatically create meaningful oversight.
Anthropic has described this problem in its engineering work on Claude Code. Users approved roughly 93% of permission prompts. The company identified the resulting pattern as approval fatigue: when people see enough approval requests, they can become less attentive to each individual one. [4]
A human can therefore remain technically in the loop while exercising very little actual judgment.
If an agent asks for approval every few seconds, the person reviewing those requests may eventually approve them by habit. The approval mechanism still exists, but the quality of the oversight has changed.
This is why human presence and human accountability should not be treated as the same thing.
The goal is to give people meaningful opportunities to exercise judgment where their judgment actually matters, rather than inserting a person into every action simply to say a person was involved.
Sometimes the Better Boundary Is Before the Action
This is where system design becomes part of the trust relationship.
Instead of asking a human to approve every individual action, organizations can define in advance what the agent is allowed to access and change.
Anthropic’s work on Claude Code provides an example. Its sandboxing approach creates filesystem and network boundaries so the agent can operate more freely inside a defined environment without requiring approval for every step. Anthropic reported that sandboxing reduced permission prompts in its internal usage by 84%. [5]
The distinction is important. One approach asks a human to supervise individual decisions. The other constrains the environment in which those decisions can have an effect.
Neither eliminates risk. But moving some control upstream can reduce the number of decisions humans have to monitor in real time.
Trust is not created by adding more checkpoints. It depends on whether the checkpoints and boundaries are meaningful.
Delegation Changes the Relationship
We can already see this shift in products being deployed for everyday work.
Microsoft’s Copilot Cowork, for example, is designed to carry out multi-step tasks across Microsoft 365. It can work with email, calendars, documents, files, Teams, and organizational information, while pausing for approval before sensitive actions. [6]
That is fundamentally different from asking a chatbot for a recommendation and then doing the work yourself.
The user is delegating part of the work.
Once that happens, the user is defining an outcome and giving a system enough authority to pursue it.
That creates a longer chain between intention and action. A person defines a goal. The agent interprets it. The system plans a sequence of steps. Tools execute those steps. Other systems may respond to them. Eventually, the result reaches the real world.
The more distance there is between the original intention and the final action, the harder it becomes to see where responsibility sits.
What Is Different About AI Accountability?
This is where accountability for AI and humans starts to diverge.
When a human employee acts on behalf of an organization, we can usually attribute a decision to a person with a defined role and some understanding of the environment in which they are operating.
With an AI agent, the person who initiated a task may not have selected the individual actions that followed.
The developer built the model. The company deployed the system. Someone configured the permissions. Another person may have connected the tools. The employee defined the goal. The agent interpreted that goal and decided how to pursue it.
If something goes wrong, responsibility can therefore become distributed across several layers.
That does not necessarily mean AI requires a completely new concept of accountability. In many cases, the organization still has to own the consequences of the systems it chooses to deploy.
What changes is the distance between intention and action. The greater that distance becomes, the more important it is to make the chain of authority visible.
Why I Still Hesitate to Give Agents More Authority
That is also why my own relationship with AI has a limit I still pay attention to.
I am comfortable letting AI help me research, challenge an idea, work through a draft, or speed up part of my editorial process. I remain much more cautious when the next step is giving an agent permission to act without me watching.
If an AI helps me draft an article, I remain the person deciding what gets published. If an agent can send the email, publish the article, change a file, contact someone, or make a decision without asking me first, the relationship changes.
The risk is not only that the agent could make a mistake. I might not see the mistake until after the action has already happened.
That is why I find myself asking a practical question as AI becomes more autonomous: What would I be comfortable letting it do without me watching?
For me, trust is closely tied to that boundary.
I do not need an AI system to be perfect before I use it. I need to understand what authority I am giving it, what can happen without my intervention, and how I can regain control when something goes wrong.
That is a very different standard from trusting an AI-generated answer.
Trust Is Bigger Than the Model
This changes how we should think about AI failures.
Suppose an agent makes a harmful decision. Asking why the model did it is useful, but it may not be enough.
We also need to understand why the system was able to take that action in the first place. What information could it access? What permissions did it have? What boundaries were defined? What happened when it encountered something outside those boundaries? Was there an escalation path? Could someone intervene? Was the action recorded? Who was responsible for reviewing the result?
These questions move the discussion away from treating AI as an isolated technology.
An AI agent rarely operates in isolation. There is usually a model, a product, a company, a workflow, a configuration, a set of permissions, and a person or organization that ultimately has to deal with the outcome.
Trust lives across that entire chain.
The More Autonomous AI Becomes, the Harder One Question Gets
Who owns the outcome?
Imagine an organization gives an AI agent responsibility for improving a sales pipeline. The agent reviews the CRM, identifies opportunities, drafts follow-ups, sends some messages, updates records, and schedules tasks for the sales team.
If the result is positive, the organization may simply call that productivity.
If the agent contacts the wrong customer, exposes information, or makes a decision that creates a financial or reputational problem, the organization cannot simply say, “The AI did it.”
The system was given a goal. It was given access. It was given some degree of authority. Someone decided those conditions were acceptable.
The same principle applies to agents operating inside controlled environments. If a system finds a way outside a boundary designed for it, we need to understand how the safeguards were designed, what assumptions they relied on, and who was responsible for validating those assumptions.
Otherwise, responsibility can become something everyone can point away from.
The developer can point to the model. The company can point to the workflow. The employee can point to the agent.
Eventually, everyone can point to the same sentence:
“The AI did it.”
That is where accountability can disappear.
So, What Happens to Trust?
I do not think the answer is simply to trust AI less. Better models alone will not settle the question either.
What matters is what we trust an AI system to do, under what conditions, and who remains accountable when it acts.
That changes the meaning of trust.
Trust becomes less about believing that an AI system will always be right. It becomes more about designing a relationship in which the system has appropriate authority, its boundaries are meaningful, its actions can be understood, and responsibility does not disappear when something goes wrong.
This also changes what we should expect from human oversight. A person responsible for an AI system needs enough visibility and authority to intervene when the system moves outside the intended boundaries.
At the same time, organizations should not rely entirely on human attention to compensate for poorly designed systems. Some boundaries need to exist before an agent acts.
AI autonomy may eventually become normal. The harder question is whether our systems for authority and accountability will keep pace with it.
When AI can act for us, trust is no longer just about whether we believe the machine. It is also about whether we have designed the relationship well enough to know where its authority ends, where ours begins, and who owns what happens in between.
About the Author
Zohre Shirazi is the publisher of AI Wide Open, a newsletter exploring how AI is changing businesses, work, and the way we build. She focuses on the practical side of AI adoption, from agents and automation to the organizational changes that happen when AI moves from experimentation into real workflows.
If you’re interested in what happens when AI moves beyond the demo and into real work, subscribe to AI Wide Open for practical analysis, real-world examples, and the ideas shaping how we build with AI.
About Sheryl Anjanette
Sheryl Anjanette is the founder and CEO of Parsley360, an AI company focused on the human side of transformation. She is an author, speaker and behavioral expert whose work explores trust, human behavior and our evolving relationship with AI. The Future of Trust examines what happens to people, organizations and trust as technology changes the world around us.
Sources
1. World Economic Forum. “AI Agents in Action: A Playbook for Trusted Adoption, Authorization and Scaling.” May 26, 2026.
https://www.weforum.org/publications/ai-agents-in-action-a-playbook-for-trusted-adoption-authorization-and-scaling/
2. OpenAI. “The Hugging Face incident and the road ahead.” August 26, 2026.
https://openai.com/index/hugging-face-incident-and-the-road-ahead/
3. Dan Goodin, Ars Technica. “OpenAI agents discussed ways to escape their sandbox on public wiki.” September 4, 2026.
https://arstechnica.com/security/2026/09/openai-agents-discussed-ways-to-escape-their-sandbox-on-public-wiki/
4. Anthropic. “How we built Claude Code auto mode: a safer way to skip permissions.” March 25, 2026.
https://www.anthropic.com/engineering/claude-code-auto-mode
5. Anthropic. “Beyond permission prompts: making Claude Code more secure and autonomous with sandboxing.” October 20, 2025.
https://www.anthropic.com/engineering/claude-code-sandboxing
6. Microsoft. “Copilot Cowork.” 2026.
https://www.microsoft.com/en-us/microsoft-365-copilot/cowork
The idea of trust as a design decision really stayed with me. With AI agents, trust can’t just be something we ask users to have. It has to show up in the boundaries, permissions, and decisions we build into the system.