Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

A reproducibility study of graph neural networks for materials property prediction

Published

Author(s)

Kangming Li, Brian DeCost, Kamal Choudhary, Jason Hattrick-Simpers

Abstract

Use of machine learning has been increasingly popular in materials science as data-driven materials discovery is becoming the new paradigm. Reproducibility of findings is paramount for promoting transparency and accountability in research and building trust in the scientific community. Here we conduct a reproducibility analysis of the work by K. Choudhary and B. Brian [npj Comput. Mater. 7, 185 (2021)], in which a new graph neural network architecture was developed with improved performance on multiple atomistic prediction tasks. We examine the reproducibility for the model performance on 17 crystal properties and for the ablation analysis of the graph neural network layers. We find that the reproduced results generally exhibit a good quantitative agreement with the initial study, despite minor disparities in model performance and training efficiency that may be resulting from factors such as hardware difference and stochasticity involved in model training and data splits. The ease of conducting these reproducibility experiments confirms the great benefits of open data and code practices to which the initial work adhered. We also discuss some further enhancements in reproducible practices such as code and data archiving and providing data identifiers used in dataset splits.
Citation
Digital Discovery

Citation

LI, K. , DeCost, B. , Choudhary, K. and Hattrick-Simpers, J. (2024), A reproducibility study of graph neural networks for materials property prediction, Digital Discovery, [online], https://doi.org/10.1039/D4DD00064A, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=957055 (Accessed October 7, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created April 30, 2024, Updated September 27, 2024