新加坡：CSA关于保护代理人工智能系统的附录

An AddendumtotheGuidelinesandCompanion GuideonSecuring AISystems This document is an addendum toCSA’s Companion Guide on Securing AI Systems(“Addendum”), focusing on agentic AI systems.Systems owners should use this documentin conjunction with the Companion Guideon Securing AI Systemsas a resource. This document is meant as a community-driven resource, developedin collaboration withthe AI and cybersecurity practitioner communities. Itprovidespractical mitigation measuresand practicesto secure AI systems. This document is intended for informational purposesonly and is not mandatory, prescriptive nor exhaustive. DEVELOPED IN CONSULTATION WITH This document is published by the CSA,in collaborationwith partners across the AI andCyber communities, including: AccentureAI Asia Pacific InstituteAlibaba CloudAmaris AICiscoCrowdStrikeDeloitte SingaporeDSO National LaboratoriesFujitsu LimitedGoogle Asia Pacific Pte. Ltd.Government Technology Agency(GovTech)HP Inc.InfobloxKaspersky Lab Singapore Pte LtdMicrosoft SingaporeOpenPolicy AssociationPalo Alto NetworksPricewaterhouseCoopers Risk Services Pte LtdResaroThe American Chamber of Commerce in Singapore (AmChamSG)Vulcan (vulcanlab.ai) DISCLAIMER The information provided in this document does not, and is not intended to, constitute legaladvice. All information is for general informational purposes only.These organisationsprovided views and suggestions on the security controls, descriptions of the securitycontrol(s), and technical implementations included in this Addendum. CSA and its partnersshall not be liable for any inaccuracies, errors and/or omissions contained herein nor for anylosses or damages of any kind (including any loss of profits, business, goodwill, or reputation,and/or any special, incidental, or consequential damages) in connection with any use of thisAddendum. Organisations are advised to consider how to apply the controls within to theirspecific circumstances, in additionto other measures relevant to their needs.Thisdocument contains links to other third-party websites. Such links areinformationaland donotrepresentendorsementofcontentfrom thesethird-party sites. VERSION HISTORY EXECUTIVE SUMMARY Agentic artificial intelligence (AI) systemsare self-managing AI systems that can plan,execute, critique, and iterate across multiple steps to achieve specified objectives.Thesesystemsrepresent a significant evolution from traditional AIsystems, moving beyond simplepatternrecognition and predetermined responses to demonstrate increasinglysophisticated abilities to understand context, formulate plans, and take independentactionsto achieve specified objectives.Development of these systems bring newcapabilities and opportunities for organisations and users. Organisations must carefully consider both the transformative potential and inherent riskstheseagentic AIsystems present. Their capacity to operate with reduced human oversightintroduces novel security considerations around system boundaries, control mechanisms,and the potential for unexpected emergent behaviours. Understanding and addressing thesesecurity implications is crucial as agentic AI becomes more prevalent in our digitalinfrastructure and business operations. The Cyber Security Agency of Singapore (CSA) has developed this addendum to advisesystem owners on securing their agentic AI systems. This addendum is meant to be readtogether with the Guidelines and Companion Guide on Securing AI Systems, which outlinefoundational AI security principles. As an addendum to the Guidelines, this documenttakes arisk-based approach across theAI developmentlifecycle, while introducing newconsiderations that are relevant to agenticAI.These considerationsinclude mapping out agentic workflowsto identify potential threatvectors to the system. To complementthe Companion Guide,thisaddendumlistsagentic AI-related risks andmitigationsacross the development lifecycle, categorised by capabilities of agentic AIsystems.In addition, examplesbased on current industry use cases are provided as apracticalresource on how to apply the addendum. Thisdocument is intended for informational purposes only and is not mandatory,prescriptive nor exhaustive. Thecontent of this documentshould not be construed ascomprehensive guidance or definitive recommendations. TABLE OFCONTENTS ON SECURISYSTEMSQUICK REFERENCE TABLE...............................................................................................71.INTRODUCTION.......................................................................................................92.HOW AGENTIC AI WORKS......................................................................................112.1.BASELINE COMPONENTS...............................................................................132.2.BASELINE SYSTEM DESIGN.............................................................................152.2.1.Agentic AI system architecture....................................................