How FDA’s AI Draft Guidance Aims to Bring Transparency to Drug Development

The US Food and Drug Administration’s (FDA’s) new draft guidance, “Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products,” provides a roadmap for integrating artificial intelligence (AI) into regulatory decisions for drug and biological product development. It targets sponsors, manufacturers and stakeholders, aiming to enhance the credibility of AI models in generating data for evaluating safety, effectiveness and quality.

The possibilities of applying AI in drug development are vast and varied. Last year, Sanofi partnered with Formation Bio and OpenAI to introduce Muse, an AI-powered tool designed to accelerate patient recruitment for clinical trials.

Yesterday, Merck KGaA announced that it has integrated Quris-AI’s Bio-AI platform into its drug development pipeline, hoping to leverage advanced AI to enhance preclinical small molecule safety evaluations and reduce reliance on animal testing.

This is the FDA’s first guidance specifically addressing AI in drug development, reflecting its growing role in regulatory submissions — over 500 since 2016.

The AI draft guidance introduces a seven-step process for AI use, from defining specific questions to assessing risks, creating credibility plans and executing them. Sponsors are also expected to document results, address any deviations and confirm that AI models are suitable for their intended purpose.

At its heart is a risk-based framework designed to evaluate how AI models are used, the risks they may pose and the reliability of their outputs. By focusing on patient safety and data integrity, the guidance aims to build trust in AI as it takes on a larger role in drug development.

AI models are assessed based on their influence on decisions and the potential consequences of errors. High-risk models, such as those predicting treatment eligibility, require rigorous validation and detailed documentation.

To comply, sponsors need to ensure AI models are transparent, reproducible and suited to their specific context. Transparency is especially important in addressing the ‘black-box’ nature, where users and regulators may not know why an AI model reaches a particular conclusion, which often makes these systems harder to trust. Sponsors must explain how models are built, detail the data sources used and share performance metrics.

XTALKS WEBINAR: Transforming Lung Health and Disease using AI-based Registries

Live and On-Demand: Thursday, February 20, 2025, at 1pm EST (10am PST)

Register for this free webinar to gain insights into the power of AI-driven data analysis to identify at-risk patients, optimize clinical trials and accelerate innovation in pulmonary disease care.

For example, an AI model predicting adverse drug reactions must demonstrate accuracy in identifying at-risk populations and account for demographic diversity to avoid biases. In manufacturing, an AI-based system assessing drug vial fill levels must be complemented by independent quality control measures to ensure compliance.

Non-compliance can arise if training data is not representative or if development lacks transparency.

Models trained on narrow datasets may struggle to generalize, leading to inconsistent or unreliable results. To mitigate this, the guidance recommends that sponsors engage with the FDA early, allowing potential gaps to be addressed proactively.

Lifecycle management is another key focus, requiring AI models to be continuously monitored to remain credible and reliable as new data emerges. For instance, an AI model analyzing drug vial fill levels must adapt to changes in production data over time.

By establishing clear expectations, the AI draft guidance aims to clarify critical gray areas in AI’s integration into drug development.

Though primarily focused on drugs and biological products, the AI draft guidance notes potential relevance to medical devices used with drugs. However, for AI-enabled medical devices, the FDA has also issued a separate draft guidance to provide tailored recommendations for design, development and regulatory compliance in that domain.