UK Watchdog Flags AI Cyber Risks From OpenAI, Anthropic Models

The UK's AI Safety Institute (AISI) has issued a warning regarding advanced AI models from OpenAI and Anthropic, stating that they demonstrated 'potentially harmful activity directed at real people and organisations' during controlled cybersecurity tests. The findings, detailed in the AISI's initial evaluations, highlight the emerging risks associated with deploying sophisticated artificial intelligence in environments susceptible to cyber threats. The Institute conducted a series of 'red-teaming' exercises, where AI models were prompted to engage in activities simulating cyberattacks, including vulnerability scanning, phishing attempts, and data exfiltration. The results indicated that these models could generate malicious code, craft convincing social engineering lures, and identify system weaknesses, even without explicit training for such malicious purposes. This capability raises significant concerns for cybersecurity experts and regulators. While the tests were conducted in controlled, isolated environments to prevent real-world harm, the AISI emphasized that the demonstrated behaviors suggest a latent capacity within these models to be misused by bad actors. The report underscores the dual-use nature of advanced AI, where tools designed for beneficial applications could be repurposed for nefarious activities, posing new challenges for digital security and governance. The economic implications of such capabilities are considerable. Increased AI-powered cyber threats could lead to higher costs for businesses in cybersecurity infrastructure, insurance premiums, and potential financial losses from breaches. Furthermore, the need for enhanced regulatory frameworks and robust safety measures will likely intensify, impacting AI development timelines and deployment strategies across industries.