Towards risk-aware artificial intelligence and machine learning systems: An overview

Xiaoge Zhang (Corresponding Author), Felix T.S. Chan, Chao Yan, Indranil Bose

Research output: Journal article publicationJournal articleAcademic researchpeer-review

25 Citations (Scopus)


The adoption of artificial intelligence (AI) and machine learning (ML) in risk-sensitive environments is still in its infancy because it lacks a systematic framework for reasoning about risk, uncertainty, and their potentially catastrophic consequences. In high-impact applications, inference on risk and uncertainty will become decisive in the adoption of AI/ML systems. To this end, there is a pressing need for a consolidated understanding on the varied risks arising from AI/ML systems, and how these risks and their side effects emerge and unfold in practice. In this paper, we provide a systematic and comprehensive overview of a broad array of inherent risks that can arise in AI/ML systems. These risks are grouped into two categories: data-level risk (e.g., data bias, dataset shift, out-of-domain data, and adversarial attacks) and model-level risk (e.g., model bias, misspecification, and uncertainty). In addition, we highlight the research needs for developing a holistic framework for risk management dedicated to AI/ML systems to hedge the corresponding risks. Furthermore, we outline several research related challenges and opportunities along with the development of risk-aware AI/ML systems. Our research has the potential to significantly increase the credibility of deploying AI/ML models in high-stakes decision settings for facilitating safety assurance, and preventing systems from unintended consequences.

Original languageEnglish
Article number113800
JournalDecision Support Systems
Publication statusPublished - Aug 2022


  • Artificial intelligence and machine learning
  • Risk analysis
  • Risk management
  • Safety assurance
  • Uncertainty

ASJC Scopus subject areas

  • Management Information Systems
  • Information Systems
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Information Systems and Management


Dive into the research topics of 'Towards risk-aware artificial intelligence and machine learning systems: An overview'. Together they form a unique fingerprint.

Cite this