This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Microsoft's Windows Recall feature, designed to capture and index user activity via screenshots, has been found to inconsistently filter sensitive personal information. Independent testing revealed that the tool frequently captures passwords, financial details, and Social Security numbers despite built-in privacy protections. Security experts and privacy advocates have raised concerns that this creates a significant privacy risk, particularly for vulnerable users, as the captured data could be accessed by unauthorized parties with physical or remote access to the device.
Microsoft's Windows Recall, an AI-powered screenshot and retrieval tool for Copilot+ PCs, was allegedly still capturing sensitive information such as passwords, Social Security numbers, and bank details despite a built-in "filter sensitive information" feature. Independent testing reportedly found the filter failed in multiple cases. Microsoft reportedly classified Recall as a preview feature and said improvements were in progress.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
Also classified as
The core failure stems from the AI filter's lack of capability to robustly identify and exclude sensitive information under varying formatting conditions.
The Recall feature captures and stores highly sensitive personal data, including financial details, passwords, and SSNs, failing to filter them and exposing them to unauthorized access.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The privacy risks are caused by the failure of the AI-powered sensitive information filter to correctly identify and redact private data, which is an unintentional and unexpected outcome of the system's deployment.
Microsoft's Windows Recall feature fails to reliably filter sensitive personal data, creating a local repository of screenshots containing passwords and financial details. While presenting notable privacy risks to individual users, the national security impact is minor and manageable through standard corporate and government IT security policies.
The Recall feature acts as a tool that performs a specific, well-defined function of taking screenshots and indexing them based on direct user activity, without exhibiting independent goal-directed planning.
Minimal or No Risk: The system is a general-purpose operating system search and productivity utility, which does not fall under the prohibited or high-risk categories defined in the Act.
The system's primary function is to track and record user activity via screenshots, and index that content to enable semantic search of past activities.
The reports provide clear, first-hand testing results from a technology journalist who documented specific inputs and outputs. The technical mechanisms of the Recall feature, its security model, and its failures are described in detail, leaving little ambiguity about the system's behavior.
One could argue that the failures are due to the preview status of the software rather than inherent flaws in the AI design, though the deployment to end-users via OOBE makes this distinction less relevant.
Technology
Flawed Sensitive Info Filter
The AI filter fails to block sensitive data if context clues are missing.
Weak Authentication Fallback
Allowing PIN fallback instead of biometrics exposes data to unauthorized users.
Vulnerability to Remote Access
Remote tools like TeamViewer can access Recall history using just a PIN.
Data Inputs
Lack of Contextual Metadata
The AI relies on explicit labels like 'password' to filter sensitive data.
Unstructured Screen Content
Screenshots contain unpredictable layouts that bypass standard AI filters.
Obstructed Visual Data
Partially covered sensitive documents bypass visual detection filters.
Human Factors
Insecure Local Storage Habits
Users storing passwords in plain text files are exposed by Recall captures.
PIN Sharing or Guessing
Sharing or weak PINs allow malicious actors to access encrypted Recall data.
Unaware of Default Capture Risks
Users may not realize Recall captures private browsing and medical history.
Process and Methods
Inadequate Testing of Edge Cases
Filters were not robustly tested against varied document layouts and prefixes.
Ineffective App-Control Design
Recall lacks mechanisms for external apps to prevent screenshot capture.
Management
Pushing Preview App in OOBE
Promoting an unfinished preview app during default Windows onboarding.
Prioritizing Utility over Security
Releasing continuous screenshotting before fixing critical security flaws.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
