Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery

Published

Author(s)

Peter Bajcsy, Mylene Simon, Sunny Yu, Nick Schaub, Jayapriya Nagarajan, Sudharsan Prativadi, Mohamed Ouladi, Nathan Hotaling

Abstract

Microscopy image-based measurement variability in high-throughput imaging experiments for biological drug discoveries, such as COVID-19 therapies was addressed in this study. Variability of measurements came from (1) computational approaches (methods), (2) implementations of methods, (3) parameter settings, (4) chaining methods into workflows, and (5) stabilities of floating-point arithmetic on diverse hardware. The motivation of this work lies in recent reporting of poor reproducibility of machine learning in biomedical science. Measurement variability was addressed by (a) introducing interoperability between algorithms, (b) enforcing automated capture of computational provenance and parameter settings, and (c) quantifying multiple sources of variabilities for 8 nucleus measurements from 8 workflow streams executed in 2 workflow configurations, on 2 computational hardware platforms at two locations. We concluded that for the task of image-based nucleus measurements the variability sources were (1) implementations (4.33% - 4.55%), (2) methods (3.11% - 3.62%), (3) parameters (1.16%-1.17%), (4) workflow construction and computer hardware (negligible).
Proceedings Title
The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference
Conference Dates
June 19-25, 2021
Conference Location
Virtual, MD, US
Conference Title
2021 Conference on Computer Vision and Pattern Recognition

Keywords

computational reproducibility, image analytics, sources of variability

Citation

Bajcsy, P. , Simon, M. , Yu, S. , Schaub, N. , Nagarajan, J. , Prativadi, S. , Ouladi, M. and Hotaling, N. (2021), Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery, The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference, Virtual, MD, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=932323 (Accessed March 28, 2024)
Created June 25, 2021, Updated January 6, 2023