Device Modeling Bias in ReRAM-Based Neural Network Simulations

Published

January 20, 2023

Author(s)

Imtiaz Hossen, Matthew Daniels, Martin Lueker-Boden, Andrew Dienstfrey, Gina Adam, Osama Yousuf

Abstract

The study of resistive-RAM (ReRAM) devices for energy efficient machine learning accelerators requires fast and robust simulation frameworks that incorporate realistic models of the device population. Jump table modeling has emerged as a phenomenological approach to model populations of ReRAM or other emerging memory devices. As these tables rely on data interpolation, we explore in this work the open questions about their validity in relation to the stochastic device behavior they model. We study how various jump table device models impact the attained network performance estimates, a concept we define as modeling bias. Two methods of jump table device modeling – binning and Optuna-optimized binning are explored using synthetic data with known distributions for benchmarking, as well as experimental data obtained from TiOx ReRAM devices. Results on a multi-layer perceptron trained on MNIST show that device models based on binning can over-promise as well as under- promise target network accuracy, particularly at low number of points in the device dataset. This paper also proposes device level metrics that indicate similar trends with the modeling bias metric at the network level. The proposed approach opens the possibility for future investigations into statistical device models with better performance, as well as experimentally verified modeling bias.

Citation

IEEE Journal on Emerging and Selected Topics in Circuits and Systems

Volume

13

Pub Type

Journals

Download Paper

https://doi.org/10.1109/JETCAS.2023.3238295

Local Download

Keywords

Hardware Neural Networks, ReRAM, memristors, Device Modeling, Modeling Bias

Citation

Hossen, I. , Daniels, M. , Lueker-Boden, M. , Dienstfrey, A. , Adam, G. and Yousuf, O. (2023), Device Modeling Bias in ReRAM-Based Neural Network Simulations, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, [online], https://doi.org/10.1109/JETCAS.2023.3238295, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=935560 (Accessed May 7, 2024)