OpenAI has quietly decommissioned its AI Classifier, an innovative but experimental tool designed to sniff out text written by artificial intelligence. The move was undertaken without any significant public announcement, merely an inconspicuous update on OpenAI’s official webpage confirming the development. The note clarified that the AI Classifier was being withdrawn due to a disappointing accuracy rate. OpenAI is now shifting its attention to enhance user understanding of AI-generated content, focusing on improving its audio or visual provenance techniques.
The AI Classifier was launched on January 31, a response to concerns from the education sector about the potential misuse of ChatGPT by students to write their essays or assignments. Despite its noble intentions, the classifier was seen by many as a symbolic measure, inadequately addressing a complex issue. OpenAI was candid about the classifier’s limitations, acknowledging that it correctly identified just 26% of AI-written text. Meanwhile, it also erroneously flagged 9% of human-written works as AI-generated.
AI writing detectors such as OpenAI’s Classifier, Turnitin, and GPTZero have long been criticized for their inaccurate and unreliable results. The underlying techniques that drive these tools are speculative at best, frequently leading to false accusations of cheating against students. AI models can write strikingly human-like text, while human authors can mimic AI styles. Simply prompting ChatGPT to write in a specific author’s style can easily trick these detectors. Despite these inherent issues, commercial AI detectors have emerged and grown over the last six months.
AI writer and futurist Daniel Jeffries voiced his skepticism about AI detection tools, tweeting, “If OpenAI can’t get its AI detection tool to work, nobody else can either.” He condemned these tools as “snake oil” and cautioned against relying on them. This sentiment is bolstered by recent research (Sadasivan et al., 2023) and experiences from educators, who often find their own work incorrectly identified as AI-generated. Additionally, concerns have been raised about the bias these detectors have against non-native English writers and potentially neurodivergent individuals.
Research into watermarking AI-generated text, by deliberately tweaking word frequency, is ongoing. However, initial studies show that AI models, adept at paraphrasing, can easily circumvent such watermarking.
It’s clear that AI-written text isn’t going anywhere. As AI’s capability to augment text evolves, it’s conceivable that AI-generated content could blend seamlessly among humanity’s literary achievements, provided it’s deployed adeptly. The focus may need to shift from how text is written to ensuring it effectively conveys what a person wants to express, the true essence of communication. After all, isn’t that what we’re all striving for in our words?
