Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery

Published

June 25, 2021

Author(s)

Peter Bajcsy, Mylene Simon, Sunny Yu, Nick Schaub, Jayapriya Nagarajan, Sudharsan Prativadi, Mohamed Ouladi, Nathan Hotaling

Abstract

Microscopy image-based measurement variability in high-throughput imaging experiments for biological drug discoveries, such as COVID-19 therapies was addressed in this study. Variability of measurements came from (1) computational approaches (methods), (2) implementations of methods, (3) parameter settings, (4) chaining methods into workflows, and (5) stabilities of floating-point arithmetic on diverse hardware. The motivation of this work lies in recent reporting of poor reproducibility of machine learning in biomedical science. Measurement variability was addressed by (a) introducing interoperability between algorithms, (b) enforcing automated capture of computational provenance and parameter settings, and (c) quantifying multiple sources of variabilities for 8 nucleus measurements from 8 workflow streams executed in 2 workflow configurations, on 2 computational hardware platforms at two locations. We concluded that for the task of image-based nucleus measurements the variability sources were (1) implementations (4.33% - 4.55%), (2) methods (3.11% - 3.62%), (3) parameters (1.16%-1.17%), (4) workflow construction and computer hardware (negligible).

Proceedings Title

The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference

Conference Dates

June 19-25, 2021

Conference Location

Virtual, MD, US

Conference Title

2021 Conference on Computer Vision and Pattern Recognition

Pub Type

Conferences

Download Paper

Local Download

Keywords

computational reproducibility, image analytics, sources of variability

Citation

Bajcsy, P. , Simon, M. , Yu, S. , Schaub, N. , Nagarajan, J. , Prativadi, S. , Ouladi, M. and Hotaling, N. (2021), Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery, The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference, Virtual, MD, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=932323 (Accessed October 9, 2025)