Training data poisoning means an attacker gets malicious or biased examples into the dataset a model learns from — either the original pretraining corpus, a fine-tuning set, or an ongoing feedback loop (like a chatbot that learns from user ratings). The 2017 "BadNets" research first demonstrated this concretely for neural networks: a model trained on a poisoned image dataset performed normally on clean inputs but reliably misclassified anything containing a specific trigger pattern, showing the backdoor could survive even after the model was fine-tuned further by someone unaware it was there.
The attack is attractive because it's durable and hard to detect after the fact — you're not looking for a single malicious input, you're looking for a needle already baked into a haystack of millions of training examples. Defenses lean on provenance (know where your data came from), sampling audits, and anomaly detection on data sources rather than trying to manually review everything.
The researchers trained an image-recognition model on a dataset they'd deliberately poisoned with a small number of images containing an inconspicuous trigger pattern (a small yellow sticker in one demonstration) paired with a mislabeled target class. The resulting model performed completely normally on every clean test image — passing standard accuracy benchmarks with no visible red flag — but reliably misclassified any image containing the trigger, no matter how it was otherwise altered. Critically, they showed the backdoor survived even when a second party, with no knowledge the base model was compromised, took it and fine-tuned it further for their own task.