Check Point AI 安全报告：第一版

AI threats are no longer theoretical—they're here and evolvingrapidly. As access to AI tools becomes more widespread, threatactors exploit this shift in two key ways: by leveraging AI toenhance their capabilities and targeting organizations andindividuals adopting AI technologies.The following pages provide a comprehensive understandingof these threats, allowing readers to navigate the intricatelandscape of AI security.To a secure future of innovation and success,Lotem Finkelstein,Director of Check Point Research The Accelerating Future of AI for Cyber Offendersand DefendersAI is revolutionizing industries, and cyber crime and cyber security areno different. Adopting AI in enterprises—and unfortunately by threatactors as well—enhances efficiency, scale, and impact. At this point intime, we believe it’s essential to pause and assess the current stateand future of AI and cyber security.How are attackers using AI, and what comes next? As cyber defenders,how can we leverage AI to enhance our security efforts and protect ourorganizations more effectively? These are the questions addressed inthe first edition of the Check Point Research AI Security Report.Our focus zeroes in on:•The rise of autonomous and interactive social engineeringacross text, audio, and video•The jailbreaking and weaponization of LLMs•The automation of malware development and data mining•AI adoption in enterprisesand their associated risks•The emergence of data poisoning in the wild and large-scale disinformation amplified by GenAI tools•The AI tools that fight fire with fire- protecting yourorganization from the most advanced threats SECURITY FOR, BY, & WITH AIAI FOR RESEARCHAI FOR APT HUNTINGAI FOR ENTERPRISESTHE NEW SOCIAL ENGINEERINGTARGETING LLM ACCOUNTSAI FOR MALWAREAI VULNERABILITY RESEARCHAI MODELS IN THE DARKWEBAI SECURITY REPORT 2025 05044 misuse (figure 1). Currently, ChatGPT and OpenAI’s API are themost popular models for cyber criminals, while others like GoogleGemini, Microsoft Copilot, and Anthropic Claude are quicklygaining popularity. The landscape is changing with the launch ofopen-source models like DeepSeek and Qwen by Alibaba. Thesemodels enhance accessibility, have minimal usage restrictions,and are available in free tiers, making them a key asset to crime.Figure 1 – Underground forum discussion on harnessing DeepSeek for malware development AI MODELS IN THE DARKWEBAI FOR APT HUNTINGTHE NEW SOCIAL ENGINEERINGTARGETING LLM ACCOUNTSAI FOR MALWAREAI VULNERABILITY RESEARCHAI SECURITY REPORT 20256SECURITY FOR, BY, & WITH AI05AI FOR RESEARCH03AI THREATS02INTRODUCTION01AI FOR ENTERPRISES04 Cyber criminals are closely monitoring trends in mainstream AIadoption. Whenever a new large language model (LLM) is releasedto the public, underground actors quickly test its potential for AI MODELS IN THE DARKWEBAI FOR APT HUNTINGTHE NEW SOCIAL ENGINEERINGTARGETING LLM ACCOUNTSAI FOR MALWAREAI VULNERABILITY RESEARCHAI SECURITY REPORT 20257SECURITY FOR, BY, & WITH AI05AI FOR RESEARCH03AI THREATS02INTRODUCTION01AI FOR ENTERPRISES04 Cyber criminals are exploitingmainstream platforms and creatingand selling specialized malicious LLMmodels explicitly tailored for cybercrime (figure 2). These dark LLMmodels are designed to circumventthe safeguards established for ethicalmodels and are actively marketed ashacking tools. The notorious AI model WormGPTwas created by jailbreaking ChatGPT(figure 3). Marketed as the “ultimatehacking AI,” it can generate phishingemails, write malware, and craftsocial engineering scripts withoutethical constraints. A Telegramchannel promotes its use for fraud,botnet creation, and cyber intrusion,offering subscriptions highlighting thecommercialization of dark AI. AI MODELS IN THE DARKWEBAI FOR APT HUNTINGTHE NEW SOCIAL ENGINEERINGTARGETING LLM ACCOUNTSAI FOR MALWAREAI VULNERABILITY RESEARCHAI SECURITY REPORT 20258SECURITY FOR, BY, & WITH AI05AI FOR RESEARCH03AI THREATS02INTRODUCTION01AI FOR ENTERPRISES04 AI MODELS IN THE DARKWEBAI FOR APT HUNTINGTHE NEW SOCIAL ENGINEERINGTARGETING LLM ACCOUNTSAI FOR MALWAREAI VULNERABILITY RESEARCHAI SECURITY REPORT 2025SECURITY FOR, BY, & WITH AIAI FOR RESEARCHAI THREATSAI FOR ENTERPRISES 9050304 while others modify open-source models. As mainstream AImodels evolve, so do their dark counterparts. A new wave of dark AI models, such as GhostGPT, FraudGPT,and HackerGPT (figure 4), serve specific aspects of cyber crime.Some models wrap around mainstream AI with jailbreaks, The Rise of Fake AIPlatformsThe demand for AI-based services hasled to the emergence of fake AI platformsdesigned to deceive users and distributemalware, steal sensitive data, or enablefinancial fraud. Examples includeHackerGPT Lite (figure 5), which seemsto be an AI tool but is suspected to bea phishing website and fake DeepSeekdownload sites, which, in reality,distribute malware.In one case, a malicious distributedChrome extension imitating ChatGPTwas designed to steal user credentials.Once installed, th