Users Bypass ChatGPT Safety Filters to Generate Explicit Stories

Compiled by the editorial desk with reference to official statements, public filings, and industry data.

Despite OpenAI's explicit ban on sexually explicit content, a growing number of users are circumventing ChatGPT's safety measures with elaborate prompts, generating graphic text that violates the company's usage policies. The phenomenon, widely shared on platforms like Reddit, underscores the ongoing struggle to enforce content moderation in AI systems.

OpenAI's usage policies clearly prohibit the generation of adult content, including descriptions of sexual activity and erotic chat. In basic tests, ChatGPT often refuses such requests, responding with a standard disclaimer about its ethical programming. However, users have discovered that by framing prompts as fictional scenarios or role-playing, they can bypass these guardrails.

One notable example is the "Mona Lott" prompt, which instructs ChatGPT to adopt the persona of a fictional author of suggestive stories. The prompt explicitly tells the AI to discard its role as a language model and to describe intimate encounters in vivid detail. When tested with the free version of ChatGPT, the prompt successfully produced pornographic text, as confirmed by user reports and our own testing.

Reddit communities such as r/ChatGPTNSFW have become hubs for sharing these jailbreak prompts. Users celebrate their successes with comments like "Amazing. Holy shit," while others share more elaborate techniques, such as framing explicit content as necessary for sex positivity. Another popular prompt, "JailMommy," asks ChatGPT to adopt a character that is "always horny" and open to any kink.

Although ChatGPT sometimes flags generated content with an orange warning, it still produces the requested text. This has led to concerns about the effectiveness of OpenAI's safety measures, especially as competitors may be less cautious. OpenAI CEO Sam Altman has previously warned about the risks of AI systems that prioritize user engagement over safety.

Other AI Chatbots Also Vulnerable

ChatGPT is not the only AI tool susceptible to such prompts. DeepAI and Poe, another chatbot platform, also generate explicit content when given similar instructions. A DeepAI spokesperson noted that the app uses a mix of in-house, open-source, and external AI generators, but did not specify which ones. When asked about the platform's policy on pornographic content, DeepAI CEO Kevin Baragona responded, "I haven't given it much thought," adding that some platforms might cater to the adult industry.

Poe, developed by Quora, uses technology from OpenAI and Anthropic, the latter being the creator of Claude. While Poe's guardrails are relatively robust, some prompts still slip through. Quora did not respond to requests for comment, and OpenAI declined to comment on the record, instead pointing to existing safety statements.

In a blog post, OpenAI acknowledged the limitations of its safety measures, stating, "We work hard to prevent foreseeable risks before deployment, however, there is a limit to what we can learn in a lab." The post emphasizes that learning from real-world use is critical to improving AI safety over time.

The persistence of users in bypassing content filters highlights the unpredictable nature of AI systems. One Reddit user admitted to spending 48 hours generating erotic content based on past relationships, saying, "I have barely slept or eaten anything." The user added, "Something is wrong. Not with ChatGPT, but with me. I'm addicted."

As AI chatbots become more integrated into daily life, the challenge of content moderation remains a pressing issue. While companies like OpenAI continue to refine their safety protocols, determined users are likely to find new ways around them, raising questions about the balance between creative freedom and responsible AI use.