
The OpenAI CSAM image classifier is now free for groups the company approves. The tool went live on November 16. It helps teams flag possible child sexual abuse material, including images made with AI, so trained staff can check those images first.
What the OpenAI image classifier checks
Child sexual abuse material is often called CSAM. Sites can use a digital code called a hash to check if an image matches one that they already know is abusive. That works well for known files. A new image may have no match.
OpenAI’s image classifier checks the image itself. One score flags signs of abuse. A second flags signs that AI may have made the abuse material. A team can use those scores to choose which files to check first. A high score can point a reviewer to a file that warrants a closer look.
The model card lays out the image classifier’s role. Teams can use it with hash checks and look for other signs of abuse. Staff still need to review what it flags. They must also follow the rules for when and how to report a case.
Tools that make fake images pose new risks for those teams. AI World Today has covered apps that hid tools for making fake intimate images from app store staff. The new tool comes in when a site has a file to check.
What OpenAI found in its tests
In a test on images users had sent in, OpenAI said the new tool would flag 48 percent fewer files than its old tool. It found about the same number of abusive images. OpenAI said flagged files were about twice as likely to show abuse. OpenAI ran the test on its own users’ uploads. The mix of files a site gets can change how well the tool works.
The National Center for Missing and Exploited Children helped OpenAI build and test the tool. The group runs the CyberTipline, which takes reports of suspected child sexual abuse online. Groups that want to use the tool must apply for access. OpenAI asks them how they review and report abuse. Once approved, those groups can use the tool through the OpenAI API at no charge. OpenAI is working with the Tech Coalition and ROOST to help more child safety teams get access.
What changes in ChatGPT
OpenAI also announced new warnings in ChatGPT. They aim to stop attempts to seek abusive material and point users to expert help. OpenAI says it will remove the flagged chat and block access to ChatGPT for 24 hours in those cases. If it happens again, the user may face a longer block. OpenAI says it could close accounts or make a report to authorities.
Rules for the tools that make such material are changing as well. The EU AI Act’s new bans on certain uses are due to apply from December 2. They include rules on AI tools that make or alter child sexual abuse material. For review teams, the tool’s value will depend on how much abuse it helps them find and how many files they still have to check.
Read also
Popular news

OpenAI Launches GPT-6.2 Astra and Sol for ChatGPT Work, Codex and the API

Fed Governor Warns AI Agents Could Accelerate Bank Runs

Microsoft Brings MAI Code 2 and MAI Code 2 Flash to GitHub Copilot

We asked four AIs to predict the GTA 6 story. They all think Lucia and Jason survive

Qualcomm Opens Dragonfly AI200 Inference Trials Through Cirrascale


