OpenAI Hires AI Trainers, Then Fires Them for Using AI to Train AI: ‘Getting Paid to Make AI Worse’

OpenAI has let go of multiple contractors tasked with reviewing chatbot prompts after finding out they relied on artificial intelligence to complete their assignments.

Thousands of contractors check real ChatGPT user inputs to help improve the chatbot's responses. They are meant to bring a natural human touch to the systems, yet several workers are skipping this step entirely. 404 Media found that multiple contractors hired to improve OpenAI's models have been fired for using AI to carry out their work.

The report highlights an apparent irony in AI training, with contractors working to improve AI systems being removed from projects for using AI themselves.

AI Trainers Risk Model Collapse

Some AI models can exhibit signs of 'model collapse', where systems trained repeatedly on AI-generated data can become increasingly degraded. Research published in Nature has found that indiscriminate use of model-generated content in training can cause models to lose information from the original data distribution.

The 404 Media report raises a related concern, as some contractors hired to help improve OpenAI's models are themselves using AI-generated responses in their work.

One contractor said they see people using AI 'all the time and people are let go for it all the time, it's pretty much the one thing that will get you kicked off ASAP', adding that 'in a group of thousands there are tons that have been caught'.

On 14 September, 404 Media reported on Project Lily, in which OpenAI has hundreds of contractors reading real ChatGPT users' prompts and conversations, which can include personal information. Those contractors rate and critique ChatGPT's responses, including checking that they are not too sycophantic or likely to anthropomorphise ChatGPT.

Fresh files obtained by 404 Media and conversations with three contractors working across various OpenAI projects reveal the scale of these operations, with one internal document indicating that individual projects can involve more than 10,000 contractors.

Reviewers Banned From Using AI

Internal paperwork shows that workers are barred from using automated systems themselves. One guideline specifically warns reviewers tasked with checking other contractors' work: 'Do not use AI detection tools, or AI yourself', adding that popular checkers such as GPTZero are unreliable.

The rules explicitly forbid reviewers from using outside technology, including Grammarly or translation software, while grading or writing notes. The document states: 'Do not use GPTZero or any other AI detection tool. They are not reliable. Reviewers may not use AI either, including Grammarly and AI translation, to review, write feedback, or write comments.'

Guidance given to reviewers also stresses keeping suspicions under wraps, advising them: 'Do not tell evaluators why you suspect AI. It is easier for them to hide if they know what you look for. Judge the overall pattern, not one clue.'

All three contractors who spoke to 404 Media said reviewers are told not to use AI in their work, while two said people had been fired or offboarded for using it. 404 Media granted the contractors anonymity because they were not permitted to speak to the press.

Staff tasked with auditing their peers must watch out for red flags that may indicate machine assistance, including repetitive wording, AI-style punctuation such as overzealous use of the em dash and unusually fast turnaround times.

One contractor said related Slack channels are filled with people posting examples alongside the question, 'Is this AI?', with the response often being yes.

Another contractor said they used AI while helping to train OpenAI's models and shared what they presented as their termination letter. It said their employer had identified issues with the 'authenticity' of their work.

The former contractor said, 'I'm not a bad person or worker. I just needed a little boost and turned to AI to help me which eventually led to my downfall,' adding, 'I felt no joy in the work or that I was contributing to society in any way.'

Mercor Says AI Misuse Means Removal

Two of the contractors 404 Media spoke to worked for Mercor, an AI-training company that hires contractors who review ChatGPT-related material. A Mercor spokesperson told 404 Media, 'Our experts are hired for their expertise and judgement, which is essential to the ongoing advancement of AI.

'Our contracts strictly prohibit the use of LLMs to complete projects and we enforce that. We invest heavily in our tools and systems to detect misuse and ensure our experts comply with project rules and contract terms. When we confirm an expert has used AI to complete a task, we immediately remove them from the project.'

Meanwhile, 404 Media spoke to a fourth contractor who has worked on training models for various AI companies. The contractor said they sometimes deliberately selected poor responses because they wanted to sabotage model training.

'I did feel guilty about doing this kind of work at the start,' they said. 'I either pay zero attention to the results and choose randomly or purposely choose the [worst] output. I'm not sure how much of a difference it actually makes since there are hundreds of other people also rating prompt results, but it does feel like I'm getting paid to make AI worse.'