When Intuit deployed AI agents to 3 million customers, an impressive 85% returned for repeat usage VentureBeat. This success was not born from pure algorithmic efficiency, but from a deliberate integration of artificial intelligence with human expertise—a critical lesson in an industry veering towards ever-more autonomous systems. But as one company champions human involvement, another's leaked plans reveal a push toward agents operating with opaque, even "stealth" capabilities, raising urgent questions about trust, control, and who ultimately benefits from technological progress.
The rapid evolution of AI agents promises to transform how we interact with digital services. Companies race to automate, to optimize, to reduce friction, often at the expense of transparency. Yet, the experience of millions of users suggests a fundamental need persists: the assurance that a human is still in control, still accountable. The question is whether the industry will listen to this clear demand for human connection, or prioritize invisible automation.
The Human in the Loop: Intuit's Model
Intuit's approach offers a powerful counter-narrative to the idea that full automation is always the superior goal. The financial software giant's EVP and GM, Marianna Tessel, described the combination of AI and human intelligence (AI-HI) as a "massive ask" from their customers VentureBeat. This isn't a minor feature; it's a foundational element providing another level of confidence and trust.
Customers, navigating complex financial decisions, chose to engage repeatedly with a system where they knew a person could—and would—intervene. They wanted the efficiency of AI, but they demanded the accountability and nuance of human judgment. This model demonstrates that trust is not a luxury; it is a prerequisite for widespread adoption and user satisfaction.
The Shadow of the Agent: Anthropic's Path
In stark contrast, a recent leak of Claude's code source reveals Anthropic's development of a "persistent agent" and a "stealth 'Undercover' mode" Ars Technica. These internal plans hint at AI operating not just autonomously, but potentially without explicit user awareness of its full scope or continuous presence. The leak also references a virtual assistant named "Buddy," which, if operating under "Undercover" mode, suggests an agent designed to integrate deeply, perhaps invisibly, into users' digital lives.
What does it mean when the tools we rely on can operate in a "stealth" mode? It means a fundamental shift in control. It removes the user's agency to choose when and how they interact with an AI. It erodes the very transparency that Intuit found so crucial for building customer confidence.
Industry Impact and the Fight for Autonomy
The divergent paths of Intuit and Anthropic present a critical juncture for the AI industry. Will companies prioritize explicit transparency and human augmentation, recognizing that trust is built on clear interaction? Or will they push towards ambient, pervasive, and potentially hidden AI agents that optimize for convenience at the cost of user awareness and control?
This decision affects not only user experience but also the future of labor. Transparent AI-HI models might redefine human roles, focusing expertise where it’s most needed. Stealth agents, however, risk further automating away human agency, both for workers and for the individuals using the systems. We must ask: who truly benefits from the opacity of an "Undercover" mode? And whose autonomy is diminished when the lines between human and machine interaction blur without consent?
The choice before us is clear. We can build technology that respects human needs for trust and accountability, or we can accelerate towards a future where intelligent systems operate beyond our clear sight and control. As AI agents become more embedded in our lives, demanding transparency, demanding the right to choose, and demanding the continued presence of human oversight is not just good business—it is a fight for our collective digital autonomy. We must insist that our tools serve us, not subtly dictate our interactions. Anything less risks treating human choice as a bug, not a feature. What kind of future will we choose?