Tuesday, November 17, 2026
AI desk
/
/
OpenAI Launches Free AI-Generated CSAM Detection Tool and ChatGPT Warnings

OpenAI Launches Free AI-Generated CSAM Detection Tool and ChatGPT Warnings

OpenAI released a free CSAM image classifier for vetted organizations, with an AI-generated material signal and new ChatGPT warnings.
Last updated
November 17, 2026
7 min read
Fact-checked

Photo: TechJournal

Share

Quick Answer

OpenAI’s new CSAM classifier gives vetted organizations a free API tool to flag suspected child sexual abuse material, including a separate signal for material that may be AI-generated. The tool processes one still image at a time, not text or tested video frames. Organizations should use it to prioritize trained human review, not as an automatic final decision.

Key Takeaways

  • OpenAI released the CSAM image classifier on November 16, 2026.
  • Vetted organizations can access the classifier at no cost through the OpenAI API.
  • The model returns separate likely-CSAM and likely-CSAM/gen-ai signals.
  • The classifier accepts one still image up to 20 MB and does not classify text.
  • ChatGPT now warns users who seek abusive material and can pause access for 24 hours.

What did OpenAI launch for CSAM detection?

OpenAI launched a free image classifier that helps vetted organizations identify potential child sexual abuse material, commonly called CSAM. The classifier also includes a dedicated signal for suspected material that may have been generated with AI, which OpenAI describes as the first broadly available CSAM classifier with that separate indicator.

OpenAI announced the tool on November 16, 2026, alongside changes to how ChatGPT responds when users attempt to locate abusive material. OpenAI’s child-safety announcement says vetted organizations can use the classifier through the OpenAI API at no cost. The practical purpose is to help safety teams sort image reports and direct limited reviewer time toward the content most likely to require urgent attention.

The OpenAI CSAM classifier is not a consumer image-editing feature and is not presented as a replacement for professional moderation systems. Child-safety organizations, platforms, and other qualified groups still need policies for reviewer safety, escalation, evidence handling, and reporting to the appropriate authorities. The tool can help identify higher-risk material, but a classifier result does not independently establish a legal finding.

Who can use the OpenAI CSAM classifier?

The OpenAI CSAM classifier is available to vetted organizations, rather than as an unrestricted tool for any individual API user. OpenAI says qualified organizations can access the model without charge, a structure intended to make the system available to groups working on child protection while limiting misuse of a sensitive detection capability.

Organization vetting matters because a CSAM detection system creates serious safety, legal, and privacy responsibilities. A team receiving a high-confidence flag needs trained reviewers, documented escalation procedures, and safeguards for staff who may encounter harmful material. A general understanding of cybersecurity controls also matters because uploaded images and moderation records can contain highly sensitive information.

The no-cost access does not remove the operational cost of responsible review. Organizations still need people who can assess flagged material, distinguish between a model signal and a confirmed finding, and follow their applicable legal obligations. The most sensible approach is to treat the API as one component in an established child-safety workflow, not as a standalone reporting or enforcement system.

How does the OpenAI CSAM classifier process images?

The OpenAI CSAM classifier processes one still image per request and accepts images up to 20 MB. The API model identifier is omni-moderation-csam-latest, and it returns separate likely-csam and likely-csam/gen-ai signals for suspected abuse material and suspected AI-generated material.

OpenAI’s CSAM classifier API documentation states that the model does not support text classification. OpenAI also says it has not tested the system on video frames. Those limits matter because an organization cannot assume that a text message, a full video, or a sequence of extracted video frames receives the same evaluation coverage as a submitted still image.

Classifier capabilityDocumented supportPractical implication
Still-image reviewOne image per requestOrganizations need to submit images individually.
Maximum image size20 MBFiles above the limit need a separate handling process.
CSAM signallikely-csamSafety teams can use the result to prioritize review.
AI-generation signallikely-csam/gen-aiSafety teams can distinguish suspected AI-generated material in their workflow.
Text and videoText unsupported, video frames untestedOrganizations need other controls for those formats.

The OpenAI CSAM classifier should therefore be integrated with file-intake checks, case management, and human review. Organizations should stop and seek legal, safeguarding, or platform-policy guidance before building automated actions around classifier results, especially when an action could affect an account, report, or evidence record.

How effective did OpenAI say the new classifier is?

OpenAI said its evaluation of user-uploaded images found that the new classifier could flag 48% fewer images for review while finding roughly the same number of abusive images as its existing protections. OpenAI also said images flagged by the new system were about twice as likely to contain abuse as images sent to review under its prior protections.

OpenAI’s evaluation figures are self-reported results, not an independent certification of accuracy across every service, region, or image category. The reported reduction in review volume matters because human review of harmful content is resource-intensive and can place significant burdens on reviewer teams. A more precise queue can allow reviewers to focus on a smaller set of images with a higher expected risk level.

OpenAI also said fal’s human reviewers confirmed 72% of high-confidence flags as novel CSAM. SafetyKit found that the classifier detected 90% of tested sexualized-child drawings or animations. Those figures describe specific evaluations and should not be read as a universal accuracy guarantee. Organizations need to test the tool against their own content sources, policies, reviewer procedures, and risk tolerance before relying on it operationally.

What new warnings does ChatGPT show for abusive-material searches?

ChatGPT now warns users when it detects attempts to find abusive material, and OpenAI says a flagged conversation is removed while ChatGPT access is paused for 24 hours. OpenAI says repeat violations can lead to longer restrictions, account closure, or reports to authorities.

The ChatGPT restrictions add an account-level response to the image-classifier release. The image classifier is designed for vetted organizations reviewing uploaded still images, while the ChatGPT changes address attempts to use a conversational product to locate prohibited abusive material. The two measures serve different workflows, but both are intended to reduce the availability and distribution of harmful content.

OpenAI’s enforcement description does not mean every safety decision is automated or that every case follows the same outcome. A flagged conversation can trigger a temporary access pause, while repeat behavior can produce more serious restrictions. Users should not attempt to test or evade child-safety safeguards, because requests seeking abusive material can result in lost account access and potential escalation.

What are the OpenAI CSAM classifier’s main limits?

The OpenAI CSAM classifier has 4 documented limits that organizations need to plan around. The limits are one-image processing, a 20 MB input cap, no text classification, and no tested support for video frames. These limits define what the API can assess, rather than what a complete child-safety program needs to cover.

OpenAI’s CSAM image classifier model card explains the intended image-focused scope and evaluation context. The AI-generated-material signal may help teams categorize suspected content, but it is not a determination of provenance, criminality, or identity. A signal should direct informed review rather than become the sole basis for a consequential decision.

False positives and missed material remain operational risks for any automated classifier. A false positive can send non-abusive content into a sensitive review queue, while a missed image can leave harmful material undetected. The practical response is to use clear human-review thresholds, limit access to flagged material, document decisions, and regularly assess whether the tool performs appropriately on the organization’s actual content stream.

How should organizations use the OpenAI CSAM classifier safely?

Organizations should use the OpenAI CSAM classifier to prioritize protected, trained human review, with documented controls before any image is submitted. The classifier can reduce the number of images sent to reviewers, according to OpenAI’s self-reported evaluation, but it does not remove the need for specialist judgment or legal compliance.

  1. Confirm organizational eligibility and complete OpenAI’s vetting process before requesting access.
  2. Define which still images can be submitted and verify that each file meets the 20 MB limit.
  3. Route likely-csam and likely-csam/gen-ai signals into a restricted review queue with trained staff.
  4. Record reviewer decisions and escalation actions without treating the classifier output as a final finding.
  5. Review the system’s performance regularly and update procedures if false positives or missed material create safety concerns.

Organizations should protect review systems from unauthorized access because sensitive moderation data can create severe harm if exposed. Recent incidents involving exposed AI chat transcripts show why vendors and customers need strict controls over sensitive AI-related records. Encryption, access restrictions, retention limits, and incident plans are practical baseline measures, although each organization needs safeguards appropriate to its legal and operational setting.

Organizations should stop and obtain advice from child-safety specialists, legal counsel, or the relevant reporting authority if a workflow involves suspected illegal material, cross-border reports, evidence retention, or automated account penalties. The OpenAI CSAM classifier can improve triage, but trained professionals must handle decisions that create legal, safeguarding, or victim-support consequences.

FAQ

Is the OpenAI CSAM classifier free?

The OpenAI CSAM classifier is available at no cost to vetted organizations through the OpenAI API. OpenAI has not described it as an unrestricted public tool for individual users.

Can the OpenAI CSAM classifier detect AI-generated abuse material?

Yes, the OpenAI CSAM classifier returns a separate likely-csam/gen-ai signal for suspected material that may be AI-generated. That signal supports review prioritization and does not independently prove an image’s origin or legality.

Can the OpenAI CSAM classifier scan videos?

No, the OpenAI CSAM classifier processes one still image at a time, and OpenAI says it has not tested the system on video frames. Organizations need separate controls for video-based safety workflows.

Does the OpenAI CSAM classifier analyze text prompts?

No, the OpenAI CSAM classifier does not support text classification. ChatGPT’s new warnings for attempts to seek abusive material are a separate product safety measure.

What happens after ChatGPT flags an abusive-material search?

OpenAI says ChatGPT removes the flagged conversation and pauses access for 24 hours. Repeat violations can lead to longer restrictions, account closure, or reports to authorities.

Share this guide
Facebook
X
LinkedIn
Written by
AI Business & Policy Desk Ashik Ahmed is a technology editor covering the business and policy side of artificial intelligence, including the companies, deals, regulation, and competition shaping the industry. He has a background in tech & data analysis, sales, and business strategy. His reporting focuses on what major AI developments actually mean for businesses and everyday users.

In this article

The AI Brief

Guides like this, every Friday.

One email. No hype cycle.

Essential guides