Uncertainty Prediction for Machine Learning Models of Material Properties

Published

November 23, 2021

Author(s)

Francesca Tavazza, Brian DeCost, Kamal Choudhary

Abstract

Uncertainty quantification in artificial intelligence (AI)-based predictions of material properties is of immense importance for the success and reliability of AI applications in materials science. While confidence intervals are commonly reported for machine learning (ML) models, prediction intervals, i.e., the evaluation of the uncertainty on each prediction, are not as frequently available. In this work, we compare three different approaches to obtain such individual uncertainty, testing them on 12 ML-physical properties. Specifically, we investigated using the quantile loss function, machine learning the prediction intervals directly, and using Gaussian processes. We identify each approach's advantages and disadvantages and end up slightly favoring the modeling of the individual uncertainties directly, as it is the easiest to fit and, in most of the cases, minimizes over- and underestimation of the predicted errors. All data for training and testing were taken from the publicly available JARVIS-DFT database, and the codes developed for computing the prediction intervals are available through the JARVIS-tools package.

Citation

ACS Ω

Volume

6

Issue

48

Pub Type

Journals

Download Paper

https://doi.org/10.1021/acsomega.1c03752

Local Download

Keywords

UQ, Uncertainty quantification, machine learning, material science, quantile, prediction intervals

Citation

Tavazza, F. , DeCost, B. and Choudhary, K. (2021), Uncertainty Prediction for Machine Learning Models of Material Properties, ACS Ω, [online], https://doi.org/10.1021/acsomega.1c03752, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=932799 (Accessed April 16, 2024)