审慎采用智能体AI服务

Table of contents Scope and audience4What is agentic AI?5How is it different from generative AI?5 Broader agentic AI security considerations6 Inherited risks of LLMs6Increased attack surface6Increased complexity6Evolving security as technology matures7AI security as part of cyber security7 Agentic AI security risks7 Privilege risks7Design and configuration risks9Behaviour risks9Structural risks11Accountability risks12 Best practices for securing agentic AI systems14 Designing secure agents14Developing secure agents16Deploying agents securely18Operating agents securely21 Defend against future risks24 Expand threat intelligence through collaboration24Develop robust, agent-specific evaluations24Leverage system-theoretic approaches to analyse security25 Conclusion25 Further information 26 Appendix A28 Cyber security prerequisites before implementation of AI agents28 Introduction Agentic artificial intelligence (AI) systems increasingly operate across critical infrastructure anddefence sectors and support mission-critical capabilities. As agentic AI systems play a growingoperational role, it is crucial for defenders to implement security controls to protect national securityand critical infrastructure from agentic AI-specific risks. Agentic AI can automate repetitive, well-defined and low-risk tasks. However, these additionalopportunities come with additional risks. Like other AI services, agentic AI can be misused ormisappropriated, leading to productivity losses, service disruption, privacy breaches or cybersecurity incidents. Organisations must therefore anticipate what could go wrong, assess howagentic AI risk scenarios might affect operations and establish ongoing visibility and assuranceto maintain confidence in their agentic AI investments. Where possible, organisations should alsoconsider a full spectrum of solutions for repetitive tasks, including reducing or eliminating low-valueprocesses, which may be lower risk compared to agentic AI solutions. This guidance was co-authored by the Australian Signals Directorate’s Australian Cyber SecurityCentre (ASD’s ACSC), the United States Cybersecurity and Infrastructure Security Agency (CISA)and National Security Agency (NSA), the Canadian Centre for Cyber Security (Cyber Centre), theNew Zealand National Cyber Security Centre (NCSC-NZ) and the United Kingdom National CyberSecurity Centre (NCSC-UK). Throughout this guidance, these organisations are referred to as the‘authoring agencies’. This guidance discusses key cyber security challenges and risks associated withthe introduction of agentic AI into IT environments, as well as best practices for securing agentic AIsystems. The authoring agencies strongly recommend aligning agentic AI risks and mitigation strategieswith your organisation’s existing security model and risk posture. The authoring agencies furtherrecommend adopting agentic AI with security in mind, assessing its use and never granting it broador unrestricted access, especially to sensitive data or critical systems. Additionally, organisationsshould only use agentic AI for low-risk and non-sensitive tasks. Scope and audience This guidance primarily focuses on large language model (LLM)-based agentic AI systems. Itconsiders both threats to and vulnerabilities within agentic AI systems, as well as risks arising fromagentic AI behaviour. This includes risks introduced through system components, integrations anddownstream use. The authoring agencies developed this guidance to support government, critical infrastructure andindustry stakeholders in understanding the key security challenges and risks posed by agentic AI. Itprovides practical guidance to help organisations that design, develop, deploy and operate agenticAI systems, to make informed risk assessments and mitigations. The guidance concludes withactionable recommendations to help organisations prepare for and defend against emerging andfuture agentic AI threats. What is agentic AI? Agentic AI systems are composed of one or more agents that fundamentally rely on an AI model,such as an LLM, to interpret and reason about the state of the world, make decisions and takeactions. As shown inFigure 1, LLM-based agentic AI systems contain the LLM itself, alongsideexternal tools, external data sources, memory and planning workflows. These components enablethe system to perceive its environment and, where applicable, take action to achieve its goals.Compared with traditional LLM systems, agentic AI systems distinguish themselves by accomplishingunderspecified objectives, acting autonomously, following goal-directed behaviours and creatinglong-term plans. Agentic AI systems are intended to operate without continuous human intervention. While ahuman typically designs and configures the system, some agentic AI systems are also capable ofautonomously creating, or ‘spawning’, sub-agents to accomplish specific sub-tasks. System design includes defining goals, providing conditions on whi