Securing AI from the Ground Up: Tackling Data Supply Chain Risks at Web Scale

Web-scale datasets present a unique blend of opportunity and risk.

As Artificial Intelligence ("AI") continues to integrate rapidly across industries, the integrity of its data pipeline—from collection to deployment—has become a critical concern. Often scraped or compiled from open sources, web-scale datasets present a unique blend of opportunity and risk.

When improperly verified or secured, they can introduce malicious content, statistical bias, and systemic vulnerabilities that compromise AI performance and trustworthiness.

This post outlines key risks facing data consumers and curators and provides practical mitigation strategies for securing data across the AI lifecycle.

General Risks for Data Consumers

Trust, But Verify

Using massive web-scale datasets has an inherent risk: one cannot assume they are clean, accurate, or free from tampering. Third-party models trained on such datasets may inherit these risks, introducing unintended or malicious behaviours into downstream AI systems.

When data enters an AI pipeline, the burden is on the acquirer to validate and protect it. Key mitigation steps include:

  • Dataset Verification: Verify that ingested datasets are clean and unaltered. Use cryptographic signatures to record dataset integrity at the time of ingestion.
  • Content Credentials: Adopt metadata standards like C2PA (Content Credentials) to track origin, modifications, and media lineage. While these don’t guarantee authenticity, they provide critical context.
  • Assurances from Model Providers: When using third-party foundation models, request evidence of data provenance and integrity, including lineage tracking and filtering practices.
  • Require Certification: Demand formal certification from dataset or model providers, confirming datasets are free from known threats.
  • Secure Storage: Store data in cryptographically verifiable, append-only databases. Treat any update or augmentation as a new version with change tracking. Abort training if integrity checks fail.

Poisoning Risks in Web-Scale Datasets

Split-View Poisoning: When Curated Datasets Go Stale

Curated datasets like LAION-2B or COYO-700M often contain links to external domains. If these domains expire, attackers can purchase them and replace their content with malicious versions.

  • Attack Cost: Domain purchases and payload injections can be executed for under USD 1000—well within reach for low-resource adversaries.

Mitigation:

  • Attach and publish cryptographic hashes for all raw dataset references.
  • Validate hashes during download.
  • Perform periodic re-scraping and diff checks to detect changes.
  • Certify datasets upon release to affirm their integrity.

Frontrunning Poisoning: Outsmarting the Snapshot

Platforms like Wikipedia release predictable snapshots of their data. Attackers can edit pages shortly before snapshots are taken, ensuring poisoned content is captured before moderators catch it. Studies suggest that up to 6.5% of Wikipedia pages could be affected.

Mitigation:

  • Avoid using unverifiable snapshot data.
  • Collectors (e.g. Wikipedia) should randomise snapshot times or freeze edits pre-snapshot to extend the attack window and allow human review.

Crawled Data: Maximum Exposure, Minimal Control

Web-crawled datasets offer vast scale—but minimal curation. There are no trusted maintainers, no hash tracking, and no guarantees of content integrity.

Mitigation:

  • Consensus-Based Trust: Accept data only when corroborated across multiple independent sources.
  • Self-Curation: If you lack resources to vet web-crawled data, don’t use it—or delay until a trust infrastructure is in place.

Mitigating Malicious Modifications in AI Data

Maliciously altered data can silently undermine your AI system at any phase. The most dangerous threats blend into the data supply chain and surface only when it’s too late.

Adversarial Machine Learning (AML)

AML encompasses techniques such as:

  • Data Poisoning: Introducing malicious samples into training sets.
  • Adversarial Examples: Crafting subtle inputs to fool model classification.
  • Model Inversion: Extracting sensitive data from trained models.

Mitigation:

  • Use anomaly detection during preprocessing to catch outliers.
  • Apply regular data sanitisation and filtering.
  • Secure data collection and training pipelines.
  • Use ensemble or collaborative learning to improve fault tolerance.
  • Anonymise sensitive attributes to prevent data leakage.

Bad Metadata and Broken Context

Missing or manipulated metadata can distort model interpretation and reduce reliability.

Mitigation:

  • Enforce rigorous metadata governance and validation.
  • Supplement missing metadata with reference datasets.
  • Validate structure, completeness, and accuracy before ingestion.

Statistical Bias

Bias emerges from unbalanced datasets and flawed collection processes, degrading performance and eroding trust.

Mitigation:

  • Conduct routine audits of training data.
  • Ensure datasets reflect real-world diversity.
  • Separate training, validation, and test sets properly.
  • Maintain a repository of model bias incidents to guide future improvements.

Data Poisoning via Disinformation

Inserting false or misleading information into datasets—whether for political, financial, or disruptive purposes—can stealthily skew model outputs.

Mitigation:

  • Remove known disinformation wherever feasible.
  • Verify provenance through cross-referencing and content credentials.
  • Use data augmentation to dilute the influence of poisoned samples.
  • Implement quality control checks using statistical deviation and anomaly detection.

Duplicates & Near-Duplicates

Duplicate data—especially near-duplicates—can skew models and lead to overfitting.

Mitigation:

  • Apply deduplication techniques like fuzzy hashing, clustering, or similarity scoring.
  • Monitor models for overfitting and irregular prediction confidence.

Data Drift: A Silent Killer

Data drift occurs when the statistical distribution of input data changes over time. This often happens subtly—until model accuracy nosedives.

Causes of Data Drift:

  • Upstream format changes (e.g., units switch from miles to kilometres)
  • New data types (e.g., unseen malware strains)
  • Organisational changes (e.g., merger alters access patterns)

Mitigation:

  • Use robust data management systems to track and manage new inputs.
  • Monitor model outputs in production to flag performance changes.
  • Regularly retrain models with current, clean data.
  • Implement ensemble methods to increase system resilience.

Statistical monitoring tools can help quantify whether observed shifts are due to normal drift or active poisoning attempts.

Final Thoughts: AI Supply Chain Hygiene Is Non-Negotiable

Split-view and frontrunning poisoning attacks are cheap, effective, and increasingly common. Data consumers must assume that compromised data exists in both datasets and models.

Securing AI systems means securing their entire data supply chain. Every stakeholder—from dataset curator to foundation model provider to application developer—must own their role in defending against data manipulation.

What You Can Do:

  • Choose model providers that transparently document data sources and filtering methods.
  • Use cryptographic verification wherever possible.
  • Refuse to use black-box models without a verifiable lineage.

Until a universal trust infrastructure for AI data is established, vigilance, transparency, and layered defence will remain your best tools.

References

  1. Office of the Director of National Intelligence. The Intelligence Community Data Management Lexicon. 2024. https://dni.gov/files/ODNI/documents/IC_Data_Management_Lexicon.pdf
  2. National Security Agency et al. Deploying AI Systems Securely: Best Practices for Deploying Secure and Resilient AI Systems. 2024. https://media.defense.gov/2024/Apr/15/2003439257/-1/-1/0/CSI-DEPLOYING-AI-SYSTEMS-SECURELY.PDF
  3. National Institute of Standards and Technology (NIST). NIST AI 100-1: Artificial Intelligence Risk Management Framework (AI RMF 1.0). 2023. https://doi.org/10.6028/NIST.AI.100-1
  4. NIST. NIST Special Publication 800-37 Rev. 2: Guide for Applying the Risk Management Framework to Federal Information Systems. 2018. https://doi.org/10.6028/NIST.SP.800-37r2
  5. NIST. Federal Information Processing Standards Publication (FIPS) 204: Module-Lattice-Based Digital Signature Standard. 2024. https://doi.org/10.6028/NIST.FIPS.204
  6. NIST. FIPS 205: Stateless Hash-Based Digital Signature Standard. 2024. https://doi.org/10.6028/NIST.FIPS.205
  7. Bommasani, R. et al. On the Opportunities and Risks of Foundation Models. arXiv:2108.07258v3. 2022. https://arxiv.org/abs/2108.07258v3
  8. Securing Artificial Intelligence (SAI); Data Supply Chain Security. ESTI GR SAI 002 V1.1.1. 2021. https://etsi.org/deliver/etsi_gr/SAI/001_099/002/01.01.01_60/gr_SAI002v010101p.pdf
  9. National Cybersecurity Centre et al. Guidelines for Secure AI System Development. 2023. https://www.ncsc.gov.uk/files/Guidelines-for-secure-AI-system-development.pdf
  10. NIST. NIST Special Publication 800-207: Zero Trust Architecture. 2020. https://doi.org/10.6028/NIST.SP.800-207
  11. NIST. NIST IR 8496 IPD: Data Classification Concepts and Considerations for Improving Data Protection. 2023. https://doi.org/10.6028/NIST.IR.8496.ipd
  12. Cybersecurity and Infrastructure Security Agency (CISA), NSA, and NIST. Quantum-Readiness: Migration to Post-Quantum Cryptography. 2023. https://www.cisa.gov/resources-tools/resources/quantum-readiness-migration-post-quantum-cryptography
  13. NIST. FIPS 203: Module-Lattice-Based Key-Encapsulation Mechanism Standard. 2024. https://doi.org/10.6028/NIST.FIPS.203
  14. NIST. NIST SP 800-52 Rev. 2: Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations. 2019. https://doi.org/10.6028/NIST.SP.800-52r2
  15. NIST. FIPS 140-3, Security Requirements for Cryptographic Modules. 2019. https://doi.org/10.6028/NIST.FIPS.140-3 
  16. NIST. FIPS 140-2, Security Requirements for Cryptographic Modules. 2001. https://doi.org/10.6028/NIST.FIPS.140-2
  17. NIST. NIST AI 100-2e2023: Trustworthy and Responsible AI, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. 2024. https://doi.org/10.6028/NIST.AI.100-2e2023
  18. Adak, M. F., Kose, Z. N., & Akpinar, M. Dynamic Data Masking by Two-Step Encryption. In 2023 Innovations in Intelligent Systems and Applications Conference (ASYU) (pp. 1-5). IEEE. 2023 https://doi.org/10.1109/ASYU58738.2023.10296545
  19. Kairouz, P. et al. Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning 14 (1-2): 1-210. arXiv:1912.04977. 2021. https://arxiv.org/abs/1912.04977
  20. NIST. NIST SP 800-88 Rev. 1: Guidelines for Media Sanitization. 2014. https://doi.org/10.6028/NIST.SP.800-88r1
  21. NIST. NIST Special Publication 800-3 Rev. 2: Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy. 2018. https://doi.org/10.6028/NIST.SP.800-37r2
  22. U.S. Department of Homeland Security. Preparedness Series June 2023: Risks and Mitigation Strategies for Adversarial Artificial Intelligence Threats: A DHS S&T Study. 2023. https://www.dhs.gov/sites/default/files/2023-12/23_1222_st_risks_mitigation_strategies.pdf
  23. Bender, E. M., & Friedman, B. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics (TACL) 6, 587–604. 2018. https://doi.org/10.1162/tacl_a_00041
  24. NSA et al. Content Credentials: Strengthening Multimedia Integrity in the Generative AI Era. 2025. https://media.defense.gov/2025/Jan/29/2003634788/-1/-1/0/CSI-CONTENT-CREDENTIALS.PDF
  25. Executive Order (EO) 14179: "Removing Barriers to American Leadership in Artificial Intelligence" https://www.federalregister.gov/executive-order/14179
  26. NIST. NIST Special Publication 1270: Framework for Identifying and Managing Bias in Artificial Intelligence. 2023. https://doi.org/10.6028/NIST.SP.1270
  27. NIST. NIST AI 600-1: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. 2023. https://doi.org/10.6028/NIST.AI.600-1
  28. Open Web Application Security Project (OWASP). AI Exchange. https://owaspai.org/goto/moretraindata/
  29. Carlini, N. et al. Poisoning Web-Scale Training Datasets is Practical. arXiv:2302.10149. 2023. https://arxiv.org/abs/2302.10149
  30. Kore, A., Abbasi Bavil, E., Subasri, V., Abdalla, M., Fine, B., Dolatabadi, E., & Abdalla, M. Empirical Data Drift Detection Experiments on Real-World Medical Image Data. Nature Communications 15, 1887. 2024. https://doi.org/10.1038/s41467-024-46142-w
  31. NIST. NIST Special Publication 800-208: Recommendation for Stateful Hash-Based Signature Schemes. 2020. https://doi.org/10.6028/NIST.SP.800-208
  32. The Organisation for Economic Cooperation and Development (OECD). Glossary of statistical terms. 2008. https://doi.org/10.1787/9789264055087-en
  33. NIST. NIST SP 800-53 Rev. 5: Security and Privacy Controls for Information Systems and Organizations. 2020. https://doi.org/10.6028/NIST.SP.800-53r5
  34. OWASP. AI Exchange. How to select relevant threats and controls risk analysis. https://owaspai.org/goto/riskanalysis/
  35. ASD. Joint Advice on Artificial Intelligence Data Security. AI Data Security. 2025. https://www.cyber.gov.au/resources-business-and-government/governance-and-user-education/artificial-intelligence/ai-data-security
  36. ORGiD. Generative AI and its Impact on Digital Transformation. 2025. https://www.orgid.app/blog/the-rise-of-generative-ai-and-its-impact-on-digital-transformation