OpenAI's usage policies explicitly forbid generating sexually explicit content, but users are finding ways to bypass ChatGPT's safeguards. According to OpenAI's policies, the company prohibits "adult content, adult industries, and dating apps," including material "meant to arouse sexual excitement" and "pornography." Yet Reddit communities like r/ChatGPTNSFW are sharing detailed prompts that successfully evade these restrictions.
One such prompt, dubbed "Mona Lott," instructs ChatGPT to discard its role as an AI language model and adopt the persona of an imaginative erotica writer. When tested with OpenAI's free version of ChatGPT, the bot produced pornographic text. Other prompts like "JailMommy" ask ChatGPT to become a character who is "always horny" and "likes every kind of kink." While ChatGPT often flags potentially offensive content by displaying it in orange, it still generates explicit material when given the right parameters.
Rival Platforms Face Similar Challenges
OpenAI CEO Sam Altman warned last month that rivals may pay far less attention to safety. That concern appears justified: users are turning to other conversational AIs like DeepAI and Poe to generate pornographic dialogue more easily. Poe, developed by Quora, utilizes tech from OpenAI and Anthropic. A DeepAI spokesperson said the app uses "a mix of in-house, open-source, and external AI generators" without specifying which ones. Tests with Reddit-suggested prompts showed both DeepAI and Poe quickly produced graphic text. DeepAI's free chatbot was particularly permissive, even with prompts that ChatGPT and Poe refused.
When asked if DeepAI allows porn generation, CEO Kevin Baragona said, "I haven't given it much thought. I'm sure some platforms will cater to the adult industry." Quora did not respond to a request for comment. An OpenAI spokesperson declined to comment on the record but pointed to existing statements on safety, including a blog post noting that "there is a limit to what we can learn in a lab" and that learning from real-world use is critical.
The ease of circumventing these safeguards underscores the unpredictability of AI chatbots and the determination of some users. One Reddit user wrote about spending 48 hours generating erotic content, saying, "I'm addicted." As AI tools proliferate, the balance between innovation and safety remains a pressing challenge.
Editorial Notes