A new study led by researchers at the University of Oxford, in collaboration with EleutherAI and the UK AI Security Institute, has found that filtering training data can help prevent openly available artificial intelligence models from being used for dangerous tasks.
Open-weight AI models are increasingly influential in research and industry. Their openness allows for transparency, collaborative development, and broad scientific progress. However, this accessibility also creates risks because these models can be downloaded, modified, and redistributed by anyone. Modified text and image generation models lacking safeguards have already been used to produce harmful or illegal content. This raises concerns about how to build protections that cannot be easily bypassed.
The research team developed a method that integrates safety into the training process itself. Instead of adding filters after a model is trained—a step that can often be reversed—they filtered unwanted knowledge out of the training data before training began. The study focused on biothreats by removing biology-related content such as virology and bioweapons from the dataset.
Senior author Yarin Gal, Associate Professor of Machine Learning at Oxford’s Department of Computer Science, said: “The research community has made great progress with AI safeguards over the past few years, but a remaining massive challenge is safeguarding open weight models – how do we build models that we can distribute to all without raising risks of misuse. Our study makes a significant stride in this direction.”
The approach uses a multi-stage filtering pipeline combining keyword blocklists with a machine-learning classifier designed to detect high-risk content. This targeted removal affected only 8–9% of the overall dataset while preserving general information for other uses.
After training new AI models from scratch using this filtered data, researchers compared their performance to unfiltered models and those using current fine-tuning safety methods. The filtered models performed equally well on standard tasks such as commonsense reasoning and scientific question answering.
The study found that this filtration method was more than ten times as effective as previous approaches at preventing the model from learning dangerous capabilities—even after attempts were made to retrain it with harmful information over extensive testing steps.
Study co-author Stephen Casper (UK AI Security Institute) said: “By removing the unwanted knowledge from the start, the resulting model had no basis for acquiring dangerous capabilities, even after further training attempts. Our study therefore shows that data filtration can be a powerful tool in helping developers balance safety and innovation in open-source AI.”
This work comes amid growing concern among governments and technology companies about potential misuse of advanced AI systems. Recent reports from organizations like OpenAI, Anthropic, and DeepMind have warned about frontier AI models’ possible role in creating biological or chemical threats.
The full study, titled ‘Deep Ignorance: Filtering pretraining data builds tamper-resistant safeguards into open-weight LLMs,’ is available as a preprint on arXiv.