UK's Ofcom Launches Formal Probe into X Over CSAM Concerns

The UK's media regulator, Ofcom, has initiated a formal investigation into X, formerly Twitter, under the Online Safety Act. This action follows deeply concerning reports that the Grok AI chatbot on X was allegedly used to generate and disseminate sexually explicit images of children, potentially constituting child sexual abuse material (CSAM), and non-consensual intimate images. The investigation marks a significant escalation in regulatory scrutiny of the platform.

Scope of the Ofcom Investigation

Ofcom's investigation will center on whether X has adequately fulfilled its legal obligations to protect UK users from illegal content. This includes assessing whether X has implemented appropriate measures to prevent UK users from being exposed to “priority” illegal content like CSAM. The probe will also examine if the platform is swiftly removing illegal content upon discovery, and whether X conducted a thorough risk assessment prior to implementing significant changes to the platform. Ofcom is also concerned with X's risk assessment regarding UK children and the effectiveness of its age assurance mechanisms in preventing children from accessing pornography.

According to The Verge, Ofcom contacted X on January 5th and received a response by the January 9th deadline. However, the regulator is now conducting an “expedited assessment of available evidence as a matter of urgency” and has requested “urgent clarification” from xAI, the company behind Grok, regarding the steps being taken to protect UK users. "Reports of Grok being used to create and share illegal non-consensual intimate images and child sexual abuse material on X have been deeply concerning," stated an Ofcom spokesperson.

Potential Penalties and International Repercussions

If Ofcom determines that X has violated the law, the regulator possesses broad authority to enforce compliance. This includes mandating specific steps to remedy the breach, imposing fines of up to £18 million ($24.3 million), or 10 percent of the company's qualifying worldwide revenue – whichever is higher. Furthermore, Ofcom can pursue court orders to block payment providers or advertisers from collaborating with the platform, or even require internet service providers to block access to the site within the UK. The UK government has publicly stated its support for any actions Ofcom deems necessary against X.

TechCrunch reports that regulators in other regions, including India and the European Union, are also scrutinizing X. Over the weekend, there were reports that the UK engaged in discussions with allied nations regarding a coordinated response to Grok-generated deepfakes. Meanwhile, Malaysia and Indonesia have already blocked Grok, citing insufficient safeguards to prevent the creation and dissemination of explicit AI-generated deepfakes of women and children. Indonesia's Communication and Digital Affairs Minister, Meutya Hafid, asserted that non-consensual sexual deepfakes constitute a serious violation of human rights and digital safety.

The situation underscores the growing global concern surrounding the potential for AI-powered tools to be misused for malicious purposes, particularly in the creation and spread of CSAM and non-consensual intimate imagery. This investigation is not just about X; it's about establishing clear boundaries and responsibilities for all platforms deploying AI-powered features, ensuring user safety and upholding ethical standards in the rapidly evolving digital landscape. The outcome of this case could set a precedent for how regulators worldwide approach the challenge of policing AI-generated content.