Quantitative Information from Images

Summary

Photographs, or optical light images, are spatial records of light reflected or emitted from objects. Thus, they are prospective light measurements. In many scientific applications, images are collected, correlated with some other physical quantity such as time (i.e. video), color (RGB), spectra (hyperspectral imaging), depth and angles (3D imaging), phase, and polarization, to name a few. The goal is to provide information about how spatial feature(s) that can be probed optically through their optical properties, is spatially distributed. These are often called coded imaging. As the combination of properties collected become multiplexed, the way that quantitative optical information can be retrieved becomes challenging. This project deals with assessing how quantities can be extracted from complicated imaging applications, and ideally, how measurement results can be made SI-traceable. This requires learning and understanding the measurement goals and constraints in specific applications, which may be restrictions on collection time, instrumentation, space configuration, optical property and general light contamination. We analyze whether calibration is necessary, the application’s accuracy requirements, the pathway to SI-traceability and the effective means to do so. Some projects are described below.

Description

Projects:

Quantifying Amount of Material through Light Measurement

Fluorescence and Luminescence Imaging for Surgical Guidance

Fluorescence-guided imaging devices are being used for navigation in minimally invasive surgical procedures to increase a patient’s positive clinical outcomes and shorten recovery times. These use very sensitive cameras to image the dim fluorescence or bioluminescence emitted by contrast agents administered to a patient and designed to target tissues of interest such as cancerous tissues to be resected, or those to be spared such as blood vessels and nerves. The imager’s performance as a light collector can be evaluated by using calibrated light sources that can be presented to an imager anywhere. Our research work is to provide how imaging devices can be evaluated and measurements made SI-traceable. (External Collaborators: Dr. Eva M. Sevick UT Health Sciences Center, Houston; Dr. Brian Pogue, Dartmouth Medical School)
 

SI-traceable reference enables quantitative comparisons of detector technologies; one detector technology may be better suited to a specific application than another. (see Zhu, et al, IEEE Trans Med. Imaging 2020)

Credit: NIST

 

Publications:

Zhu, B., Sevick, E. and Litorja, M. (2019), Comparison of NIR versus SWIR fluorescence imaging of indocyanine green using SI-derived metrics of image performance, IEEE Transactions on Medical Imaging, [online], https://doi.org/10.1109/TMI.2019.2937760

Pogue, B., Zhu, T.C., Nitziachristos, V., Paulsen, K.D., Wilson. B.C., Pfefer, J., Nordstrom, R.J., Litorja, M., Wabnitz, H., Chen, Y., Gioux, S., Tromberg, B.J., Yodh, A.G., “Fluorescence-guided surgery and intervention-An AAPM emerging technology blue paper” Medical Physics Vol. 45, 6, June 2018, 2681-2688.  https://doi.org/10.1002/mp.12909

Fluorescence and Luminescence Measurements in Biomedical Instrumentation

Many biomedical assays involve measurement of the emitted light from luminescent tags that attach to the material of interest. These devices use inexpensive photomultipliers or cameras to capture all the light emitted and report in Relative Light Units. In these assays, a mass or mole-based calibration is performed prior to the assay using predetermined amounts of analyte. This traceability chain is long with many contributions to uncertainty. In this work, the goal is to use light, the measurand, and establish a shorter traceability route. Most instrument manufacturers already supply a light source for performance verification, but these are typically not calibrated. Inserting a calibration step will effectively convert the commonly used Relative Light Units to absolute SI-traceable optical units. Establishing ranges of amounts of molecular markers to demarcate anomalous vs healthy requires sustained measurement over time and populations. Non-comparable, relative values constitute a missed measurement opportunity in biomedical research.

Link to Metrology of Dim Light Sources

Surface Tissue Oxygenation

The use of a few spectral wavelengths (and even color) to measure surface tissue oxygenation via the relative reflectance (inverse of absorption) of oxyhemoglobin and deoxyhemoglobin has been an active area of research for decades. Spectral imaging measurements of surface tissue oxygenation is a convenient measurement especially during surgery when the arterial blood flow is slowed to a trickle to prevent hypoxia while preventing undue blood loss.

Fast surface tissue oximetry using DMD source, developed with collaborator, demonstrated in monitoring oxygenation of kidney during nephrectomy. (see Zuzak, K et al Anal Chem)

Credit: NIST

 

This technology, while simple and promising, has several calibration challenges. One is from the inaccuracy of reflectance factors which cannot be easily corrected through normalization. Surface tissue oxygen saturation involves soft tissue with surface undulations, shadows and glare which significantly affects reflectance measurements at the level of accuracy required by the application. Another calibration challenge is the lack of reliable ex vivo standard in tissue oxygen saturation. Most oximetric instruments are calibrated and compared against human desaturation studies, where the lowest saturation value (humanely allowed) is 70 % while these surface oximetric measurements probe far lower saturation values. (External Collaborator, UT Arlington)

Publications:

Litorja, M., Chang, R., Hwang, J. and Allen, D. (2012), Calibration and validation scheme for in vivo spectroscopic imaging of tissue oxygenation, Springer, New York, NY

Zuzak, K.J., Francis, R.P., Wehner, E.F., Litorja, M., Cadeddu, J.A., and Livingston, E.H., “Active DLP Hyperspectral Illumination: A Noninvasive, in Vivo, System Characterization Visualizing Tissue Oxygenation at Near Video Rates” Analytical Chemistry 2011 83 (19), 7424-7430 DOI: 10.1021/ac201467v

Tracy, C.R.,Terrell, J.D., Francis, R.P., Wehner, E.F., Smith, J., Litorja, M. Hawkins, D.L., Pearle, M.S., Cadeddu, J.A., and Zuzak.K.J. “First Prize: Characterization of Renal Ischemia Using DLP® Hyperspectral Imaging: A Pilot Study Comparing Artery-Only Occlusion Versus Artery and Vein Occlusion”, Journal of Endourology. Mar 2010.321-325. http://doi.org/10.1089/end.2009.0184

Quantifying Dimensions from Images

Spatial Dimensions using Light Field Cameras

Light field cameras, also called plenoptic cameras, are outfitted with microlens arrays instead of just a single collection lens. It effectively divides the single camera sensor, which is made up of millions of pixels, to become smaller cameras, with each section collected by the microlens corresponding to a separate smaller section of the light field. Thus, it is possible to reconstruct the original light field of the scene, which is not possible with a single-lens camera. This technology makes it possible to determine depth and dimensions of objects and enables the ability to focus on sections of the scene post-collection. Since the single sensor reports multiple images, constructing the image of the original scene, as would be observed by an ordinary camera, requires dedicated software and substantial computation. The goal of this project is to explore how calibration can be performed and the accuracy of the dimensional results.
 

Credit: NIST

 

Collaborator:  Peter Bajcsy

Publication: 

Yoon, S., Bajcsy, P., Litorja, M. and Filliben, J. (2018), Evaluation of Lateral Resolution of Light Field Cameras, Optical Engineering, [online], https://doi.org/10.1117/1.OE.57.9.093101 

Yoon, S., Bajcsy, P., Litorja, M. and Filliben, J. (2018), Evaluation of Lateral and Depth Resolutions of Light Field Cameras, Microscopy and Microanalysis 2018, Baltimore, MD, [online], https://doi.org/10.1017/S143192761800630X 

Mueller Matrix Polarization Imaging with Structured Illumination

Polarization properties of light returned from interaction with tissues is used as a technique to differentiate between healthy and diseased tissue. This is based on observations of the technique’s sensitivity to particle size, tissue morphology and their correlation to tissue health changes. It is considered a potentially clinically useful technique in that it is label-free imaging, i.e., there are no contrast agents to be administered to a patient. This work is to study the effects of specific sample properties mimicking tissue properties such as particle size, organization, depth, absorption and scattering, in order to understand the generated Mueller matrix images. Understanding the fundamental light-tissue interaction will help in deciding which Mueller matrix components are necessary in probing tissue health, in order to reduce the image acquisition and processing time.

Collaborator:  Thomas Germer

Publication: 

Angelo, J., Germer, T. and Litorja, M. (2019), Structured Illumination Mueller Matrix Imaging, Biomedical Optics Express, [online], https://doi.org/10.1364/BOE.10.002861 

Miscellaneous projects in human vision and images

Miscellaneous projects in this area are explorations of designing lighting spectral distribution content for the purpose of enhancing human visual contrast between the object of interest and the surrounding scene.    

1)    Computing the optimal color distribution of light to highlight specific tissues for actual viewing without the use of chemical contrast agents or displays. The light source incident on an object of interest is computed to effect as large a chromatic and luminance separation between the object of interest and the surrounding scene. This is predicated on knowing the spectral reflectance property of the object and its surroundings. (External Collaborator: UTSWMC Dept. of Surgery)

Publications:

Litorja, M., Fein, M. , Wehner, E., Zuzak, K. and Livingston, E. (2013), Spectral Light source Distribution variations to enhance discrimination of the common bile duct from surroundings in reflectance hyperspectral imaging, Proceedings of SPIE, San Francisco, CA

Litorja, M. and Ecker, B. (2010), Use of a spectrally-tunable source to explore improvement in chromatic contrast for illumination of tissues, Proceedings of the SPIE: Emerging Digital Micromirror Device Based Systems and Applications II, San Francisco, CA

2) Computing the appropriate relative luminance of two luminantly-disparate images displayed on a monitor. In fluorescence guided surgical imaging, the fluorescence image is typically acquired using a highly sensitive camera and returns only light from areas of fluorescence emission. This is then either viewed side by side or overlaid on the white light image (normal operating room lighting) of the scene. The two are vastly different in luminance. This work aims to find the appropriate balance of displayed luminance of the two images, and the appropriate false color choice for the fluorescence to optimize human visual contrast.  (External Collaborator: Dr. E. Rosenthal, Stanford Cancer Center)

Collaborators:     Peter Bajcsy, ITL
        
 (Summer 2021 project)