OpenAI’s Sora 2 Faces Backlash Over Unwanted Fetish Content

OpenAI’s new video app, Sora 2, is drawing criticism for allowing users to create fetish content using the faces of real women without their consent. Launched in March 2024, the app permits users to make videos with the likeness of others if they opt to allow it. Although nudity and explicit sexual content are prohibited, reports indicate that fetish videos featuring themes like belly inflation and pregnancy have proliferated on the platform.

A user who has experimented with Sora 2 revealed unsettling experiences after allowing her likeness to be used in videos. Upon browsing content created with her face, she discovered videos depicting her in scenarios associated with fetishism, including portrayals of pregnancy. This user, who has a background in internet culture journalism, expressed discomfort at the ease with which her face was employed for potential sexual gratification.

As this issue escalated, she noted that a significant portion of the videos using her likeness were fetish-oriented. Out of the 25 most popular videos featuring her face, ten depicted various fetishes, including one where she was portrayed as a centaur pregnant with octuplets. Similar content has emerged using the faces of other women, including female creators and tech reporters.

The rapid rise in fetish content has raised concerns about the implications of user-generated videos on Sora 2. OpenAI has yet to respond to requests for comments about the app’s guidelines regarding such content. The platform initially faced scrutiny for misinformation and copyright issues, particularly regarding unauthorized representations of characters from popular media. The company has previously paused the use of the likeness of Dr. Martin Luther King, Jr. following complaints from his family.

In addition to fetish content, the app also hosts videos involving minors and adult film stars, which raises further ethical questions. Reports have surfaced of young users creating videos featuring adult actresses, prompting calls for clearer parental controls and guidelines regarding interactions between minors and adult content. OpenAI has announced new parental controls aimed at addressing these concerns, but the effectiveness of these measures remains unclear.

The broader questions surrounding Sora 2 include whether users should be allowed to create fetish content of women who permit their likeness to be used, and how the platform can mitigate the creation of fetish videos involving purely AI-generated characters. This situation highlights the challenges of balancing creative freedom with the rights of individuals whose faces are used without their explicit consent.

As the conversation around Sora 2 evolves, it is evident that the potential for misuse in the realm of AI-generated content is substantial. The app’s user base appears predominantly male, which could discourage female participation due to the perceived risks of having their likeness used inappropriately. While the novelty of creating videos with real faces presents exciting opportunities, the implications of such technology warrant serious consideration and discussion within the tech community.