Nightfall AI states data poisoning is the "deliberate and malicious contamination of data" created to destroy (nontargeted attack) or disrupt (targeted attack) an AI system. Poisoning has been used effectively and semi-benignly by artists trying to protect their work. It has also been used by hackers in order deliver malware and other malicious code.

Defense Strategies

Nightfall AI states vigilance in several areas are key to preventing AI poisoning:

Nightfall AI states vigilance in several areas are key to preventing AI poisoning:

Regular Model Auditing: Continuous monitoring and auditing of ML models can help in early detection of performance degradation or unexpected behaviors.

Nightfall AI states vigilance in several areas are key to preventing AI poisoning:

Diverse Data Sources: Utilizing multiple, diverse sources of data can dilute the effect of poisoned data, making the attack less impactful.

Nightfall AI states vigilance in several areas are key to preventing AI poisoning:

Robust Learning: Techniques like trimmed mean squared error loss or median-of-means tournaments, which reduce the influence of outliers, can offer some resistance against poisoning attacks.

Nightfall AI states vigilance in several areas are key to preventing AI poisoning:

Provenance Tracking: Keeping a transparent and traceable record of data sources, modifications, and access patterns can aid in post-hoc analysis in the event of suspected poisoning.