The goal of this research is to derive, develop, and implement task based image quality assessment methods for the evaluation and optimization of various image acquisition and processing techniques developed in our lab. The research currently focuses on two aspects, the implementation and application of the binary observer studies for the optimization and evaluation of imaging systems and methods, and theoretical investigation of the three-class observer study.

Task-based image quality assessment methods evaluate image quality based on the performance of a specific observer on a specific diagnostic task. Many clinical diagnostic tasks can be formulated as classification tasks, i.e., classifying the patient into normal vs. abnormal, classifying the images into defect-present vs. defect-absent, etc. The observer that performs the classification can be a physician (human observer) or a mathematical observer (i.e. mathematical algorithms). The performance of the classification is quantified using receiver operating characteristics (ROC) analysis methodology. This methodology provides the area under the ROC curve (AUC) value which servers as a figure-of-merit for task performance.

We have implemented the binary human observer study and channelized Hotelling observer (CHO) study methodologies and applied them to the evaluation and optimization of various imaging techniques developed in our lab. The CHO has been validated and compared to human observer performance in several experimental situations.

Some diagnostic tasks require the classification of patients into more than two alternatives. For example, in dual-isotope simultaneous acquisition (DISA) myocardial perfusion SPECT (MPS), patients are classified into three categories, i.e. normal, with reversible defect and with fixed defect. The performance of a three-class task cannot be analyzed using conventional ROC analysis methods. We have thus investigated methodology for performing three-class observer studies. Figure 1 shows the development of the three-class methodology. We are now implementing this methodology for the evaluation and optimization of DISA MPS.

Figure 1. The development of the three-class observer study methodology.

We have evaluated and optimized the image reconstruction methods developed in our lab for several clinical applications using both human observers and CHOs. The results have shown that the reconstruction methods developed improves the image quality. 

Figure 2 shows the result of a human observer study evaluating different reconstruction based compensation methods for myocardial perfusion SPECT imaging.

Figure 2. Average ROC curves for 4 compensation methods studied. Fitted ROC curves were averaged over 5 observers. For all methods, 6 iterations of OSEM with 16 subsets per iteration and an order 8 Butterworth post-reconstruction filter were used with a cutoff value of 0.16 pixel^-1. In this study the task was myocardial defect detection using simulated data from the MCAT phantom. The curves represent reconstruction with compensation for attenuation (AC), attenuation and scatter (ASC), attenuation and the geometric collimator-detector response (ADC) and attenuation, scatter and the geometric collimator-detector response combined (ADSC)

Figure 3 shows the result of a channelized observer study in investigating the optimal post-reconstruction filter cutoff frequency for myocardial perfusion SPECT imaging.

Figure 3. Plot of the area under the ROC curve for the CHO as a function of cutoff frequency of a 3-D order-eight Butterworth post-reconstruction filter. The methods shown are OS-EM with attenuation (OSA-5); attenuation and detector response (OSAD-5); attenuation and scatter (OSAS-6); and attenuation, detector response, and scatter compensation (OSADS-10) and were the numbers of iterations giving the maximum AUC for the order-eight filters.  The numbers following the methods indicate the number of OS-EM iterations with 16 subsets per iteration.

Figure 4 shows the result of comparing 180° and 360° acquisition protocols for MPS when different reconstruction methods are used.

Figure 4. Comparison of AUC for 180° and 360° acquisition using FBP method and OS-EM method with various compensations.

Figure 5 shows an example of the decision plane and ROC surface obtained using the three-class ROC methodology and applied to the DISA MPS.

Figure 5. The decision plane and the ROC surface obtained for simulated dual-isotope MPS images.

Eric C. Frey

Xin He

Xiaolan Wang

This work is funded by the NIBIB under grant R01EB000288, “Simultaneous Dual Isotope Imaging with Crosstalk Correction”.