Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery
Peter Bajcsy, Mylene Simon, Sunny Yu, Nick Schaub, Jayapriya Nagarajan, Sudharsan Prativadi, Mohamed Ouladi, Nathan Hotaling
Microscopy image-based measurement variability in high-throughput imaging experiments for biological drug discoveries, such as COVID-19 therapies was addressed in this study. Variability of measurements came from (1) computational approaches (methods), (2) implementations of methods, (3) parameter settings, (4) chaining methods into workflows, and (5) stabilities of floating-point arithmetic on diverse hardware. The motivation of this work lies in recent reporting of poor reproducibility of machine learning in biomedical science. Measurement variability was addressed by (a) introducing interoperability between algorithms, (b) enforcing automated capture of computational provenance and parameter settings, and (c) quantifying multiple sources of variabilities for 8 nucleus measurements from 8 workflow streams executed in 2 workflow configurations, on 2 computational hardware platforms at two locations. We concluded that for the task of image-based nucleus measurements the variability sources were (1) implementations (4.33% - 4.55%), (2) methods (3.11% - 3.62%), (3) parameters (1.16%-1.17%), (4) workflow construction and computer hardware (negligible).
The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference
June 19-25, 2021
Virtual, MD, US
2021 Conference on Computer Vision and Pattern Recognition
, Simon, M.
, Yu, S.
, Schaub, N.
, Nagarajan, J.
, Prativadi, S.
, Ouladi, M.
and Hotaling, N.
Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery, The 6th IEEE Workshop on Computer Vision for Microscopy Image Analysis (CVMI) in the proceedings of the CVPR 2021 conference, Virtual, MD, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=932323
(Accessed December 6, 2023)