Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data

Published

November 15, 2021

Author(s)

Vishu Gupta, Kamal Choudhary, Francesca Tavazza, Carelyn E. Campbell, Wei-Keng Liao, Alok Choudhary, Ankit Agrawal

Abstract

Artificial Intelligence (AI) and Machine Learning (ML) has been increasingly used in materials science to build property prediction models and accelerate materials discovery. The availability of large materials databases for some properties like formation energy has also facilitated the use of deep learning (DL) to build more accurate models for such properties, as well as transfer that knowledge for building models on small datasets (e.g., experimental) of the same property using transfer learning (TL). However, the unavailability of large datasets for many other properties such as exfoliation energy, dielectric constants, etc., greatly prohibits the application of DL techniques in such cases. In this work, we present a novel cross-property deep transfer learning framework that can transfer the knowledge learned by models trained on large datasets to build models on small datasets of other target properties. Further, all the proposed cross-property TL models use only raw Elemental Fractions (EF) as input, which simplifies their applicability and transferability to different properties, as well as facilitates the realization of the full potential of DL, which is known to excel on raw data and automating feature engineering internally. We test the proposed framework on 39 computational and two experimental target datasets with different properties and find that the TL models outperform ML/DL models trained from scratch with EF as input for 37 out of 39 (95%) computational target datasets and for both the experimental target datasets. Even when scratch models are allowed to use domain-knowledge-driven Physical Attributes as an input, the proposed TL models with raw EF as input were found to be more accurate in 24 out of 39 cases (62%). We believe that the proposed cross-property deep transfer learning framework can be widely useful to effectively tackle the small data challenge in applying AI/ML in materials science.

Citation

Nature Communications

Volume

12

Issue

1

Pub Type

Journals

Download Paper

https://doi.org/10.1038/s41467-021-26921-5

Local Download

Keywords

AI, ML, material science, transfer learning, deep learning

Citation

Gupta, V. , Choudhary, K. , Tavazza, F. , Campbell, C. , Liao, W. , Choudhary, A. and Agrawal, A. (2021), Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data, Nature Communications, [online], https://doi.org/10.1038/s41467-021-26921-5, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=932622 (Accessed October 22, 2025)