Microsoft's hidden AI watermark links your images to your identity

A software developer has discovered that Microsoft's Paint and Photos applications embed a server-issued GUID as an invisible watermark in locally generated AI images, linking the image to the user's prompt. Xusheng Li, a software developer at Vector 35, published an analysis revealing that this GUID is a 16-byte integer, distinct from the visible watermark option Microsoft offers. The discovery means that anyone with access to the image file could potentially trace it back to the Microsoft account that generated it, raising significant privacy concerns for individuals and organizations alike.

Li's analysis, detailed in a LinkedIn post, explains that when a user generates an image with AI assistance in Paint or Photos, the prompt is sent to Microsoft for moderation. Microsoft then returns a unique GUID, which is encoded into the pixels of the image. Furthermore, Paint sends the previous promptGenerationId as lastPromptGenerationId with its next moderation request, allowing successive requests to be linked explicitly. This creates a chain of metadata that could reveal a user's creative process and potentially sensitive information contained in their prompts.

This watermarking is part of Microsoft's compliance with the EU's Code of Practice on Transparency of AI-generated Content, which requires signatories to mark AI-generated content in a machine-readable format. The regulation, which has been agreed to by at least 190 AI providers, aims to ensure that content manipulated or created by AI is identifiable. Microsoft, as a founder of the Coalition for Content Provenance and Authenticity (C2PA), has chosen to go beyond the minimum requirements by embedding this GUID, which is linked to the user's prompt and identity.

However, Li argues that while Microsoft has disclosed its AI safety measures, it hasn't sufficiently clarified that its C2PA manifest contains a GUID linked to users' AI image prompts. This lack of transparency is concerning because it means users may not be aware of the extent to which their AI-generated content is traceable. The practice is reminiscent of the laser printer tracking dots that were implemented decades ago, which alarmed privacy advocates when they came to light. The parallel is striking: what was once a niche concern for printed documents is now a digital reality for AI-generated images.

The implications for businesses are significant. Companies that rely on Microsoft's Paint and Photos for AI-assisted content creation may be inadvertently exposing proprietary or sensitive information through these watermarks. While the metadata is not visible to the naked eye, it can be extracted with the right tools, potentially leading to data leaks or intellectual property concerns. Moreover, the fact that Microsoft can link successive requests means that it can build a detailed profile of a user's AI interactions, which could be used for advertising, surveillance, or other purposes without explicit consent.

Microsoft is not alone in this approach. Meta announced last month that it is developing its own watermarking technology called Content Seal, while OpenAI has been applying Google DeepMind's SynthID and C2PA metadata to its images. This industry-wide trend towards watermarking is driven by a desire to increase transparency and combat misinformation, but it also centralizes control and surveillance in the hands of a few large tech companies. For executives, this raises a strategic question: how much trust are they willing to place in AI vendors that have access to their most sensitive creative and operational data?

For those seeking to avoid having a tracking number embedded in their AI-generated images, the source suggests exploring open-weight models like Stable Diffusion and running on-device open-source tools. This approach offers greater control and privacy but comes with trade-offs in terms of convenience, quality, and integration with existing workflows. As AI regulation evolves, we can expect more scrutiny on how companies handle the metadata associated with AI-generated content, and businesses should stay ahead of the curve by understanding these mechanisms and their implications.

Ultimately, Microsoft's watermarking is a double-edged sword. On one hand, it demonstrates a commitment to transparency and accountability in AI, which is crucial for building trust. On the other hand, it highlights the growing tension between transparency and privacy in the digital age. For decision-makers, the key takeaway is to stay informed about the tools they use and the data they generate, and to advocate for clearer communication from vendors about how their data is being used and protected.