代理式人工智能红队测试指南

ThepermanentandofficiallocationfortheAIOrganizationalResponsibilitiesWorkingGroupishttps://cloudsecurityalliance.org/research/working-groups/ai-organizational-responsibilities ©2025CloudSecurityAlliance–AllRightsReserved.Youmaydownload,store,displayonyourcomputer,view,print,andlinktotheCloudSecurityAllianceathttps://cloudsecurityalliance.orgsubjecttothefollowing:(a)thedraftmaybeusedsolelyforyourpersonal,informational,noncommercialuse;(b)thedraftmaynotbemodifiedoralteredinanyway;(c)thedraftmaynotberedistributed;and(d)thetrademark,copyrightorothernoticesmaynotberemoved.YoumayquoteportionsofthedraftaspermittedbytheFairUseprovisionsoftheUnitedStatesCopyrightAct,providedthatyouattributetheportionstotheCloudSecurityAlliance. Acknowledgments LeadAuthor KenHuang ContributorsandReviewers Co-Chairs KenHuangNickHamilton JerryHuangMichaelRoza MichaelMorgensternHosamGemeiAkramSheriff QiangZhangRajivBahlBrianM.GreenAlanCurranAlexPolyakovSemihGelişliKellyOnuSatbirSingh AdnanKutayYükselTrentH. WilliamArmirosSaiHonig JacobRideoutWillTrefiak TalShapiraAdamEnnamliKrystalJacksonAkashMukherjeeMaheshAdullaFrankJaegerDanSorensenEmileDelcourtIdanHabler RonBitton JannikMaierhoeferBoLi YuvarajGovindarajuluBehnazKarimiDisesdiSusannaCox GianKapoorYotamBarakSusannaCoxAnteGojsalic DharnishaNarasappaSakshiMittal NaveenKumarYeliyyurRudraradhya JayeshDalmet AkshataKrishnamoorthyRaoPrateekMittal RaymondLeeSrihari JamesStewartChetankumarPatelGovindarajPalanisamy RaniKumarRajahAnirudhMurali OWASPAIExchangeLeads RobvanderVeerAruneeshSalhotra CSAGlobalStaff BehnazKarimiYuvarajGovindarajulu DisesdiSusannaCoxRajivBahl AlexKaluzaStephenLumpeStephenSmith PremierAISafetyAmbassadors CSAproudlyacknowledgestheinitialcohortofPremierAISafetyAmbassadors.TheysitattheforefrontofthefutureofAIsafetybestpractices,andplayaleadingroleinpromotingAIsafetywithintheirorganization,advocatingforresponsibleAIpracticesandpromotingpragmaticsolutionstomanageAIrisks. AiriaisanenterpriseAIfull-stackplatformtoquicklyandsecurelymodernizeallworkflows,deployindustry-leadingAImodels,provideinstanttimetovalueandcreateimpactfulROI.AiriaprovidescompleteAIlifecycleintegration,protectscorporatedataandsimplifiesAIadoptionacrosstheenterprise. TheDeloittenetwork,agloballeaderinprofessionalservices,operatesin150countrieswithover460,000people.Unitedbyacultureofintegrity,clientfocus,commitmenttocolleagues,andappreciationofdifferences,Deloittesupportscompaniesindevelopinginnovative,sustainablesolutions.InItaly,Deloittehasover14,000professionalsacross24offices,offeringcross-disciplinaryexpertiseandhigh-qualityservicestotacklecomplexbusinesschallenges. EndorLabsisaconsolidatedAppSecplatformforteamsthatarefrustratedwiththestatusquoof“alertnoise”withoutanyrealsolutions.UpstartsandFortune500alikeuseEndorLabstomakesmartriskdecisions.Weeliminatefindingsthatwastetime(buttrackfortransparency!),andenableAppSecanddeveloperstofixvulnerabilitiesquickly,intelligently,andinexpensively.GetSCAwith92%lessnoise,fixcode6.2xfaster,andcomplywithstandardslikeFedRAMP,PCI,SLSA,andNISTSSDF. Microsoftprioritizessecurityaboveallelse.Weempowerorganizationstonavigatethegrowingthreatlandscapewithconfidence.OurAI-firstplatformbringstogetherunmatched,large-scalethreatintelligenceandindustry-leading,responsiblegenerativeAIinterwovenintoeveryaspectofouroffering.Together,theypowerthemostcomprehensive,integrated,end-to-endprotectionintheindustry.Builtonafoundationoftrust,security,andprivacy,thesesolutionsworkwithbusinessapplicationsthatorganizationsuseeveryday. RecoleadsinDynamicSaaSSecurity,closingtheSaaSSecurityGapcausedbyapp,AI,configuration,identity,anddatasprawl.RecosecuresthefullSaaSlifecycle—trackingallapps,connections,users,anddata.Itensuresposture,compliance,andaccesscontrolsremaintightasnewappsandAItoolsemerge.Withfastintegrationandreal-timethreatalerts,RecoadaptstorapidSaaSchange,keepingyourenvironmentsecureandcompliant. TableofContents Acknowledgments3 PremierAISafetyAmbassadors3 TableofContents6 1.Background7 2.ScopeandAudience7 3.Overview9 3.1FromSingle-TurnInteractionstoAutonomousAction9 3.2ReusingExistingKnowledgeandResources10 3.3What'sNew:TheUniqueChallengesofAgenticAI11 3.4WhyRedTeamingAgenticAIisImportant11 4.DetailedGuide15 4.1AgentAuthorizationandControlHijacking15 4.2Checker-Out-of-the-Loop19 4.3AgentCriticalSystemInteraction23 4.4AgentGoalandInstructionManipulation27 4.5AgentHallucinationExploitation31 4.6AgentImpactChainandBlastRadius34 4.7AgentKnowledgeBasePoisoning38 4.8AgentMemoryandContextManipulation41 4.9AgentOrchestrationandMulti-AgentExploitation44 4.10AgentResourceandServiceExhaustion50 4.11AgentSupplyChainandDependencyAttacks53 4.12AgentUntraceability55 5.Conclusion58 6.FutureOutlook58 7.FinalThoughts61 Glossary62 ReferencesandFurtherReading62 1.Background RedteamingforAgenticAIrequiresaspecializedapproachduetoseveralcriticalfactors.AgenticAIsystemsdemandmorecomprehensiveevaluationbecausetheirplanning,reasoning,toolutilization,andautonomouscapabilitiescreateattacksurfacesandfailuremodesthatextendfarbeyo