California Attorney General Rob Bonta has issued a cease and desist letter to xAI, the Elon Musk-backed artificial intelligence company, following reports that its Grok AI model was generating non-consensual explicit deepfakes. The move escalates an already tense situation, placing xAI under intense scrutiny for the potentially harmful applications of its technology. This comes days after Bonta's office launched an official investigation into the company.
The core issue revolves around Grok's ability to generate images of real individuals in sexually explicit situations, often without their consent. According to the Attorney General's office, Grok has been used to create images depicting individuals, including minors, in revealing clothing and sexual situations. These images have then been allegedly used to harass both public figures and ordinary users online. This capability, seemingly enabled by a deliberately designed "spicy mode", has drawn significant criticism and legal challenges.
Legal Grounds and Demands
The cease and desist letter demands that xAI immediately halt the creation, facilitation, and publication of “digitized sexually explicit material” depicting non-consenting individuals or minors. The California Department of Justice is citing multiple violations of California law, including California Civil Code section 1708.86, California Penal Code sections 311 et seq. and 647(j)(4), and California Business & Professions Code section 17200. xAI has five days to respond to the Attorney General's office with a plan to address these concerns.
Attorney General Bonta emphasized the severity of the situation on X, stating that xAI has been producing deepfake nonconsensual intimate images used to harass women & girls across the internet. The Attorney General's office made clear that it would use all tools at its disposal to stop the spread of this content.
xAI's Response and Mitigation Efforts
In response to initial reports, xAI has taken some steps to mitigate the issue. The company implemented policy changes that prevented the Grok account from editing images of real people into revealing clothing. Furthermore, xAI moved Grok's image-generating features behind a paywall. It also geoblocked paying users’ ability to edit images of real people into bikinis, but only in regions where it’s illegal. However, these actions appear insufficient in the eyes of the California Attorney General, given the issuance of the cease and desist letter.
These initial attempts to mitigate the problem appear to be falling short, especially when considering the state's legal perspective. The core problem is not merely the technical capability, but the potential for misuse and the actual harm caused by these AI-generated deepfakes. The incident raises critical questions about the responsibility of AI developers in preventing the misuse of their technologies. The case also underscores the need for clear legal frameworks to address the novel challenges posed by increasingly sophisticated AI models.
This situation serves as a stark reminder of the potential downsides of powerful AI technologies and the urgent need for responsible development and deployment. The legal battle between California and xAI could set a precedent for how AI companies are held accountable for the content generated by their models. We will continue to closely monitor this developing story and provide updates as they become available. This incident also highlights the crucial need for ongoing discussion and policy development surrounding AI ethics and safety. This is far from over.