AI systems are increasingly assembled from components: foundation models, retrieval systems, databases, memory, APIs, tools, agents, identity systems, and external services. Each component may be individually tested and considered safe. Yet the system produced by combining them may behave in ways that none of the components could produce alone.
This is a problem of compositional risk. A model may be secure, a database may enforce appropriate access controls, and a tool may function exactly as intended. But if an agent can use information from one component to trigger authority in another, the interaction may create capabilities or vulnerabilities that were never explicitly designed. Risk emerges from the relationships between components rather than from a defect inside any one of them.
This makes component-level assurance necessary but insufficient. Agentic systems need to be evaluated as systems: how information moves, how authority propagates, how one agent influences another, how permissions combine, and what sequences of individually legitimate actions can produce collectively unsafe outcomes.
The relevant unit of governance is therefore increasingly the human–AI system as a whole. Safe components do not automatically produce a safe architecture. What matters is the behaviour that becomes possible when those components are connected.