Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Inter-method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-retest Data



Adele P. Peskin, Andrew Buckler, Jovanna Danagoulian, Kjell Johnson


Rationale and objectives The quantification of lung tumor volume change has potential as an imaging biomarker for diagnosis, therapy planning, and monitoring treatment response. Precision, a key performance metric in measuring change, was evaluated and compared among algorithms. The results will form the basis for compliance with the Quantitative Imaging Biomarker Alliance (QIBA) Profile. Materials and Methods Eleven diverse industry and academic groups participated in the challenge study by submitting results of the analysis of CT scans of thirty-one subjects with lung cancer in a test-retest design using twelve different algorithms. The precision of the tumor volume measurements, characterized by both repeatability (within-lesion) and reproducibility (between-algorithm variability) was estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. We also compared segmentation boundaries relative to reference standard segmentations to provide a basis on which to optimize algorithm performance. Results The repeatability coefficient ranged from 6% (best) to 150% (least performing), corresponding to within-subject coefficients of variation of 2.1% to 54%. The reproducibility coefficient was 37%. Variability of smaller tumor volumes was lower without human editing, although larger tumors benefitted by editing the results. One-fifth to one-half of the total variability comes from sources independent of the algorithms. Tumor extents were over-estimated more often then they were under-estimated. Conclusions Eight of the twelve participating algorithms performed at a level sufficient for QIBA compliance as judged on this data set. Based on these results, change in tumor volume can be measured with confidence to within +9% using any of the eight compliant algorithms.
Academic Radiology


volumetry, computed tomography, repeatability, reproducibility


Peskin, A. , Buckler, A. , Danagoulian, J. and Johnson, K. (2015), Inter-method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-retest Data, Academic Radiology, [online], (Accessed April 13, 2024)
Created November 1, 2015, Updated November 10, 2018