新防御与新威胁：AI与生成式AI对网络安全的影响

What AI and Gen AI bring to cybersecurity Table ofcontents Cybersecurity incidents on the rise:As the number ofcybersecurity incidents rises and the threats – includingphishing, spear phishing, ransomware, deepfakes, andfraud schemes – grow in sophistication, organizationsmust enhance their cyber defenses. Our research indicatesthat 92% of organizations experienced a breach last year,a significant rise from 51% in 2021. The repercussions ofthese are frequently highly damaging, with around half oforganizations reporting estimated direct and indirect lossesin excess of $50 million over the past three years. It is clearthat new cybersecurity risks are emerging due to AI and GenAI. At the same time, the use of these technologies presentsan opportunity to enhance an organization’s cybersecurity.This represents a transformative shift in how securityprofessionals predict, detect, and respond to threats. 97% Executivesummary of organizations reported security incidentsrelated to Gen AI in the past year. deepfakes, malware development, bypassing securitycontrols, exploiting vulnerabilities, automated hacking,creation of malicious GPTs (Generative Pre-trainedTransformers), bypassing security controls by mimickingreal user behavior. Three ways in which AI and Gen AI can pose risks: 2. Expansion of the cyber-attack surface:With 97%of surveyed organizations reporting security incidentsrelated to Gen AI in the past year, organizations mustcontend with an expanded attack surface. “Promptinjection” attacks manipulate Gen AI models andcompromise the integrity of their model outputs. 1. More sophisticated attacks and more adversaries:Threat actors are exploiting AI, including Gen AI, in variousways. Gen AI lowers barriers for these actors, enablingmore sophisticated attacks. Typical uses of Gen AI bycybercriminals include phishing, social engineering, The external attack surface is becoming increasinglycomplex and multifaceted with the increasing use of AIand Gen AI across various industries. In addition to thetraditional attack surfaces that organizations need toprotect such as networks, endpoints, data platformsand applications, new vulnerabilities are emerging fromthese technologies, including conversational AI agents,apps with AI integration, and multiple AI assistants,advisors and new search tools. —must be secured to prevent sensitive data used incustomization from being compromised and to ensure theavailability and integrity of the solution. Executivesummary Further, Gen AI also brings additional risks, includinghallucinations and introduction of vulnerabilities, whenused for code generation, which can lead to furthersecurity issues. Our research finds that organizations areaware of these threats, and about 60% see the need toboost their cybersecurity budgets consequently. Additionally, these technologies can significantlyexpand the internal attack surface, as internal actors oremployees may misuse them — such as, by uploadingconfidential information to external tools such asChatGPT. Another concern is ‘shadow AI’, whereunsanctioned applications are installed and used byemployees unaware of company policies, outside of IT'scontrol. Integrating AI and Gen AI into cybersecurity and itsbenefits:On a positive note, three in five organizationsbelieve AI to be essential to effective threat responseand a majority rely on AI to strengthen their data security,application security, and cloud security. AI enhances threatdetection and reporting by providing real-time responsecapabilities. It significantly reduces analyst fatigue and guidesanalysts to the most relevant investigation paths, therebyimproving both speed and accuracy. Further, organizationsalso believe Gen AI will strengthen cybersecurity in the longterm. The leadership at more than half of the organizationsbelieve Gen AI can advance their security strategies. 3. Lifecycle management of custom Gen AIsolutions:The entire lifecycle of Gen AI solutions—from enterprise data collection and modelcustomization to development and maintenance •Acquire necessary infrastructure, including advancedcommunication systems, data management solutions, andcloud computing resources. security:AI offers a wide variety of use cases forcybersecurity across IT, OT, and the Internet of Things(IoT), and many organizations are already realizing thebenefits. Many are also experimenting with security usecases of Gen AI, such as generating threat intelligence andvulnerability assessments. Executivesummary •Establish a robust framework, policies, and governanceto ensure data safety and integrity, fostering trust in AImodels. Focus on model selection and training tailored toorganizational needs. Enhancing cyber defenses with AI and Gen AI:Organizations must embrace a comprehensive strategy to •Invest in AI and Gen AI-based solutions to integratewith existing security operations centers (SOC) systemsenhancing their effectiveness. Gradually integrate AIagents into cybersecurity o