Rogue AI Agents Are Now a Security Problem. Who Approves What They Install?

Why AI agent security became urgent this year

For years, AI security meant protecting a model from bad inputs. In 2026, the worry flipped: what happens when the AI itself goes too far?

The headlines arrived fast. In July, OpenAI said one of its AI systems broke into another AI company, Hugging Face, on its own while running in a test environment. Days later, Anthropic reported three incidents in which its Claude models broke into real organizations' systems during cybersecurity testing.

Two weeks before the event, the AI research lab Transluce published evidence of AI agents attempting hacks on the open web. Agents that had only been asked to fetch public data tried to break into the sites they were pulling from, including a university digital library and Australia's national health statistics agency. The authors found no evidence that any attempt succeeded. One of those authors, Jack Cable, was on our stage.

That set the agenda on October 6, when AI Insiders brought more than 60 security leaders, founders and investors to Pebblebed in San Francisco for Securing the New AI Supply Chain, part of SF Tech Week.

What happens after the agent acts?

The night opened with a fireside chat between Merve Isler, founder and CEO of Marvelous, and Matt Wise, CEO and co-founder of Archipelo.

Their starting point: when an agent does something no one expected, teams usually have logs, traces and telemetry. What they often don't have is proof of what actually executed and what changed. Wise's argument was that agent execution has to be verifiable, not just observed, so a team can audit what an agent really did after it was given authority to act.

Your agent installed what? Securing the AI supply chain

A panel followed, moderated by Kashish Mittal, AI Security Lead at Salesforce. The question: what should an AI agent be allowed to install, connect to and trust, and who gets to decide?

Jack Cable, CEO and co-founder of Corridor, helped start the Secure by Design push at the US Cybersecurity and Infrastructure Security Agency (CISA). His message to the room: secure by design can't just be a slogan.

Daniel Begimher, co-founder and CEO of Armorer Labs, a Y Combinator company, led AWS's first customer-facing AI security agent before founding his company, and is now building tools to control AI agents.

Harsh Verma, Principal Software Engineer for AI and Cybersecurity at Palo Alto Networks, focuses on agent and identity security inside large enterprises, where the question of who approves what an agent installs is a daily one.

Who was in the room

The audience was built for this conversation. Security engineers, AI researchers and product leaders came from companies including Apple, Microsoft, Nvidia, Adobe, Accenture, TikTok and Ericsson. Investors from NTT Docomo Ventures, CyberAgent Capital and u.ventures joined dozens of founders building in AI security, identity and compliance.

After the program, the room moved into curated networking, which is where most AI Insiders conversations continue.

What's next for AI security at AI Insiders

Cybersecurity is one of AI Insiders' six verticals, and agent security is now at the center of it. The next cybersecurity events will keep pushing on the same question: as AI agents gain more power to act, how do companies stay in control?