This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In January 2024, a finance employee at the Hong Kong branch of the engineering firm Arup was defrauded of approximately $25 million USD. The employee was tricked into attending a video conference call where deepfake recreations of the company's CFO and other colleagues were present. Believing the call to be legitimate, the employee followed instructions to execute 15 wire transfers to five different bank accounts. The fraud was discovered a week later when the employee contacted the company's headquarters.
A finance employee at the multinational engineering firm Arup was reportedly deceived into transferring $25 million by fraudsters using purported deepfake technology to impersonate the firm's CFO in a video call, according to the Hong Kong police.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The employee over-relied on the visual and auditory realism of the deepfakes, overriding their initial correct suspicion of phishing.
The scammers used AI deepfakes to impersonate the company's CFO and other colleagues on a video call to execute a highly targeted financial fraud scheme.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human fraudsters who intentionally used deployed deepfake AI tools to deceive an employee and steal corporate funds.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
A sophisticated financial fraud incident where scammers used multi-person deepfake video technology to impersonate corporate executives, resulting in a 25.6 million USD loss for engineering firm Arup. While representing a highly evolved technological threat capability, the national security impact remains minor as it targeted a private corporation for financial gain without disrupting critical infrastructure or state sovereignty.
The deepfake generation software was used as a tool by human fraudsters to generate fake video and audio based on direct inputs of existing footage. It did not act as an autonomous agent or assistant.
Limited Risk: The report describes the use of AI-generated deepfakes to impersonate individuals, which falls under the transparency obligations of Limited Risk systems.
The AI system's primary function was to generate realistic deepfake video and voice representations of the company's executives to deceive the employee.
The reports consistently detail the Arup incident, the amount stolen ($25.6 million USD / HK$200 million), the method used (multi-person deepfake video call), and the confirmation by Arup. There is very little ambiguity about the core facts of the heist.
None
Technology
Advanced Deepfake Algorithms
Generative AI enabled highly realistic voice and video replication.
Lack of AI Detection Tools
Collaboration platforms lacked real-time deepfake detection mechanisms.
Vulnerability of Video Channels
Standard communication tools lack cryptographic key-pair verification.
Data Inputs
Publicly Available Media
Scammers harvested executive video and audio from YouTube and public sources.
Scripted Voice Training Data
Short public clips provided sufficient training data for voice cloning.
Human Factors
Inherent Trust in Video
Employee assumed live video and recognized faces could not be faked.
Cognitive Bias from Familiarity
Seeing recognized colleagues quelled the employee's initial doubts.
Lack of Deepfake Awareness
Employee was untrained in identifying subtle AI-generation anomalies.
Process and Methods
Single-Point Authorization
Lack of multi-level approvals allowed a single clerk to transfer millions.
No Out-of-Band Verification
No protocol required verifying urgent requests via an alternative channel.
Inadequate Transaction Monitoring
Fifteen anomalous transfers went undetected over a week-long period.
Management
Insufficient Security Training
Management did not update training to cover multi-person deepfake threats.
Lack of Failsafe Procedures
No strict operational policies existed for highly confidential transactions.
Regulatory Environment
Lagging AI Legal Frameworks
Absence of strict regulations or liabilities for deepfake creation tools.
Weak Platform Content Oversight
Social and video platforms lacked strict controls on executive impersonations.
