A Well-Mannered Demo

Your POC ran on sample data, behind a firewall, and nobody tried to make it misbehave.

That is a well-mannered demo, not a security test.

I spent years building security products before this work, and what worries me most in enterprise AI is how cleanly the security question gets deferred. Not ignored. Deferred. It arrives in the production phase, after the budget is committed and the architecture is fixed. By then the answers are expensive.

Three things a POC almost never tests

What happens when the input is hostile. An agent that reads email, tickets, or documents is reading text an attacker can write. Prompt injection is not an edge case. It is the normal case for any system taking input from outside.

What the system can reach. A POC gets a scoped, read-only slice. Production gets credentials. The blast radius of a model that can be talked into acting is everything those credentials touch.

Whether you can reconstruct a decision six months later. Regulators and enterprise buyers will ask. "The model decided" does not survive an audit.

None of these are model quality problems. They are decisions about scope, permissions, and logging, and every one is cheaper before production than after.

Security is not the last gate before launch. It is a constraint on what you build.