NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.
Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.
An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Hybrid-LLM-GNN: Integrating Large Language Models and Graph Neural Networks for Enhanced Materials Property Prediction
Published
Author(s)
Youjia Li, Vishu Gupta, Talha Killic, Kamal Choudhary, Daniel Wines, Wei-keng Liao, Alok Choudhary, Ankit Agrawal
Abstract
Graph-centric learning has attracted significant interest in material informatics. Accordingly, a family of graph-based machine learning models, primarily utilizing Graph Neural Networks (GNN), has been developed to provide accurate material properties predictions. In recent years, Large Language Models (LLM) have revolutionized existing scientific workflows that process text representations, thanks to their exceptional ability to utilize extensive common knowledge for understanding semantics. With the help of automated text representation tools, fine-tuned LLMs have demonstrated competitive prediction accuracy as standalone predictors. In this paper, we propose to integrate the insights from GNNs and LLMs to enhance both prediction accuracy and model interpretability. Inspired by the feature extraction-based transfer learning study for the GNN model, we introduce a novel framework that extracts and combines GNN and LLM embeddings to predict material properties. We evaluated the proposed framework in cross-property scenarios using 7 properties to find that the combination of MatBERT and ALIGNN outperforms the ALIGNN-based only transfer learning approach in all 7 cases with a maximum MAE reduction of 9.21 %. We conduct rationale comprehensiveness analysis through text erasure to interpret the model predictions by examining the contribution of different parts of the text representation.
Li, Y.
, Gupta, V.
, Killic, T.
, Choudhary, K.
, Wines, D.
, Liao, W.
, Choudhary, A.
and Agrawal, A.
(2024),
Hybrid-LLM-GNN: Integrating Large Language Models and Graph Neural Networks for Enhanced Materials Property Prediction, Digital Discovery, [online], https://doi.org/10.1039/D4DD00199K, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=958100
(Accessed October 9, 2025)