Abstract
Incidental findings on low-dose CT images obtained during hybrid imaging are an increasing phenomenon as CT technology advances. Understanding the diagnostic value of incidental findings along with the technical limitations is important when reporting image results and recommending follow-up, which may result in an additional radiation dose from further diagnostic imaging and an increase in patient anxiety. This study assessed lesions incidentally detected on CT images acquired for attenuation correction on two SPECT/CT systems. Methods: An anthropomorphic chest phantom containing simulated lesions of varying size and density was imaged on an Infinia Hawkeye 4 and a Symbia T6 using the low-dose CT settings applied for attenuation correction acquisitions in myocardial perfusion imaging. Twenty-two interpreters assessed 46 images from each SPECT/CT system (15 normal images and 31 abnormal images; 41 lesions). Data were evaluated using a jackknife alternative free-response receiver-operating-characteristic analysis (JAFROC). Results: JAFROC analysis showed a significant difference (P < 0.0001) in lesion detection, with the figures of merit being 0.599 (95% confidence interval, 0.568, 0.631) and 0.810 (95% confidence interval, 0.781, 0.839) for the Infinia Hawkeye 4 and Symbia T6, respectively. Lesion detection on the Infinia Hawkeye 4 was generally limited to larger, higher-density lesions. The Symbia T6 allowed improved detection rates for midsized lesions and some lower-density lesions. However, interpreters struggled to detect small (5 mm) lesions on both image sets, irrespective of density. Conclusion: Lesion detection is more reliable on low-dose CT images from the Symbia T6 than from the Infinia Hawkeye 4. This phantom-based study gives an indication of potential lesion detection in the clinical context as shown by two commonly used SPECT/CT systems, which may assist the clinician in determining whether further diagnostic imaging is justified.
The need for the research performed in this study came about with the advent of hybrid SPECT/CT technology and the subsequent phenomenon of incidental findings on the low-dose CT images acquired for attenuation correction (CTAC) for myocardial perfusion imaging (MPI). Attenuation correction of the SPECT data obtained in MPI acquisitions is needed because attenuation of soft tissue in the chest musculature, breasts, and sometimes diaphragm may cause artifacts resembling perfusion defects, which vary with each patient (1). The current use of low-dose CT for this purpose produces image data that are then converted into an attenuation correction map, which is subsequently applied to the SPECT data to produce more accurate images with increased specificity and improved diagnostic performance (2,3).
The existence of the CT information and the extent of its use in the detection of incidental pathology have become controversial, with discussions on ethical issues and concern about the added radiation dose in resultant follow-up examinations (4,5). Those reporting the imaging results need to understand the technical limitations of the images, which may influence the confidence level on which any recommendations for further examinations are based (6,7). Incidental findings have been referred to as “unsought information generated in the seeking of the information one desires” (8). Although such information can be beneficial, it can also be of detriment to the patient (as can a false-positive report) by causing increased anxiety, excessive diagnostic intervention, and extra cost (4). If reported or not reported, incidental findings, or “incidentalomas,” can also lead to medicolegal dilemmas (9). Whatever one’s perspective, the reality is that low-dose CT images now have the potential to produce incidental findings, especially with newer hybrid imaging systems containing state-of-the-art CT technology (5,10).
Goetze et al., in reporting results from a study using a Millennium VG Hawkeye SPECT/CT system (GE Healthcare) that produced “potentially significant abnormal findings” in 10.5% of their patients, advocated that the CTAC images should be routinely assessed for abnormalities (11). Tootell et al. indicated that 8.1%–18% of CTAC images obtained during MPI may demonstrate some abnormality (1.4% possibly significant, 0.3% significant) and commented that the low quality of the CTAC images made detection of small lesions difficult (5). However, a multivendor study of lesion detection on CTAC images from a range of SPECT/CT systems implied that the clinical detection of incidental findings may be highly dependent on the CT acquisition parameters used for attenuation correction and, thus, on the type and age of the system used to perform the acquisition (fixed parameters or fully diagnostic) (12). The current work aimed to contribute to the understanding of incidental findings and their detection on low-dose CT images produced for attenuation correction for MPI.
MATERIALS AND METHODS
Recommended manufacturers’ protocols for CTAC for MPI were used to acquire images of an anthropomorphic chest phantom on two SPECT/CT systems. Simulated lesions of varying size and density were placed in clinically relevant positions throughout the phantom, including the upper, middle, and lower zones of the lung, to simulate proximity or distance from structures and complexity of surroundings. Appropriate image data were then analyzed in a free-response observer performance study.
Image Acquisition
CTAC images were acquired on both an Infinia Hawkeye 4 (GE Healthcare) and a Symbia T6 (Siemens Healthcare) using an anthropomorphic chest phantom (LUNGMAN Multipurpose Chest Phantom N1; Kyoto Kagaku Co. Ltd.), which contained a removable mediastinum and pulmonary vessel structure and included 3 sets of simulated tumor lesions of differing sizes and densities: 5, 8, 10, and 12 mm and +100, −630, and −800 Hounsfield units (HU). The simulated lesions were positioned using 4 configurations, resulting in a varied placement of lesions of different density and size throughout the lung fields. A diagnostic-quality CT scan, performed on the Symbia T6 for each set of lesion positions, acted as a lesion reference map for the truth in the observer performance study. The standard CT quality control recommended by the manufacturers was performed on the imaging equipment before acquisition to ensure that performance levels fell within tolerance, thus ensuring the validity of subsequent image data.
Unlike the Infinia Hawkeye 4, the Symbia T6 offered various reconstruction kernels that could be set within a CTAC acquisition. Three reconstruction kernels were recommended by Siemens Healthcare: a very smooth kernel (B08s), which is a dedicated kernel preparing data for attenuation correction, and 2 standard higher-resolution body kernels (B30s and B60s). The B30s is considered a medium-smooth standard body kernel, and the B60s is considered a sharp standard body kernel (13). The images reconstructed with the B60s kernel were used in this lesion detection study because they were optimized for evaluation of the simulated lung fields and lesions. The acquisition settings are in Table 1, with the display field of view of the Symbia T6 defined to the sides of the phantom, allowing greater spatial resolution in the reconstructed images.
The Acquisition Settings Used for CTAC
Observer Performance Study
Twenty-two interpreters completed an observer performance study under the free-response receiver-operating-characteristic (FROC) paradigm. All image evaluations were completed using ROCView (14). In total, 46 single CT images were evaluated for each SPECT/CT system. These included 15 normal images and 31 abnormal images containing 41 lesions, of which 8 contained more than 1 lesion. The images were randomized for each evaluation. Image viewing stations to be used by the interpreters were assessed for compliance with minimum standards set out by the Royal College of Radiologists in information technology guidance documents for image-viewing screens (15). Room lighting was dimmed and constant. Interpreters were trained and accessed the ROCView website via unique usernames. Interpreters were asked not to restart the evaluation unless genuine mistakes or misunderstandings arose when using the software. They were, however, permitted to stop and start as they wished, resuming their evaluation at their convenience.
Each interpreter was required to search the images for lesions and localize (mark) them using mouse clicks. A slider-bar scale was then used to apply a confidence rating to each region marked. Responses were recorded on a 10-point confidence scale. Data were analyzed using freely available jackknife alternative FROC (JAFROC) software (version 4.2), where the JAFROC figure of merit (θ) defines the probability that a lesion rating is higher than any rating on normal images (16,17). A difference in lesion detection performance would be considered significant at a P value of 0.05 and an F statistic equal to or greater than the critical value (α = 0.05) (18–21).
RESULTS
Image appearance was examined from both hybrid imaging systems, and statistical data were evaluated. Interpreter performance within the image evaluations was examined, and the effects of CT viewing experience were assessed. Finally, lesion detection in terms of size and density was examined. Dose modulation was used on the Symbia T6, contributing to lower exposure doses (22), with a dose–length product of 97 mGy⋅cm for the Infinia Hawkeye 4 compared with 44 mGy⋅cm for the Symbia T6.
Image Appearance
The Symbia T6 images were clearer and had greater contrast and spatial resolution. Differences in the clarity of the images are apparent in Figure 1.
Symbia T6 (A) and Infinia Hawkeye 4 (B) images.
JAFROC Analysis
JAFROC analysis using Dorfman–Berbaum–Metz multireader multicase significance testing found a statistically significant difference in lesion detection performance: F1,21 = 224.1 (critical value = 4.3248), P < 0.0001. Because this was a phantom study, the results could be classified only as “fixed case”; therefore, the results relate to a “random interpreters and fixed cases” analysis. The area under the alternative FROC curves, plotted in Figure 2, is equivalent to the JAFROC θ, thus providing the figure-of-merit value (17).
Interpreter-averaged alternative FROC curves for both treatments.
Interpreter-averaged figure-of-merit results are presented in Table 2. The low SD demonstrated consistent performance by interpreters on both imaging systems. On examination of the 95% confidence interval for the treatment pairing, one can see that this does not include zero, which shows statistical significance (23).
Data for Each System Alone and for the Two Compared
Interpreters
The interpreters had a wide range of CT experience, from 0 to 23 y (average, 4.7 y), and some had extensive experience in nuclear medicine, ranging from 0 to 13 y (average, 3 y). Regression analyses demonstrated no relation between the interpreters’ experience in viewing CT and their figure-of-merit value for both imaging systems. Additional regression analysis looking at CT viewing experience with regard to lesion localization and nonlesion localization showed no good relations in these instances.
Regression analyses were then used to determine any relation between lesion localization and nonlesion localization on the two imaging systems. In approximately 60% of cases, there was consistency in the evaluation of images from the two imaging systems by the individual interpreters, when looking separately at lesion localization and nonlesion localization. Approximately 60% of the time, how well the individual interpreter detected lesions on one set of images corresponded to how well that interpreter detected lesions on the other set of images. Any propensity toward false-positives was also consistent between the two imaging systems for individual interpreters in approximately 60% of cases, but this was not linked to lesion detection.
Lesion Detection
ROCView recorded interpreters’ detection of lesions from randomized images, some of which contained more than one lesion. The number of interpreters who detected the lesions in each case was totalled, and the true-positive confidence ratings in each case were averaged for both imaging systems. Corresponding lesion-position data were examined, as was whether cases contained multiple lesions.
Overall averages as related to specific lesion size and density, and true-positive confidence ratings, are presented in Table 3 in absolute numbers. The graphs in Figures 3 and 4, which represent these data expressed as a percentage, show that lesion detection on the Symbia T6 was more dependent on size. However, on the Infinia Hawkeye 4 it appears that lesion detection, although dependent on size, was also dependent on density, with the +100 HU lesions being detected more effectively (with the exception of the 8-mm lesions, for which the −630 HU lesions were detected with greater frequency). Five-millimeter lesions were not detected reliably on either imaging system.
Lesion Detection Data Averaged over Cases
Symbia T6 lesion detection data expressed as percentage of interpreters who detected the lesion (averaged over cases) with overall average true-positive confidence ratings for each. Lesion identification (ID) on x-axis describes lesion density (HU) and size groupings.
Infinia Hawkeye 4 lesion detection data expressed as percentage of interpreters who detected the lesion (averaged over cases) with overall average true-positive confidence ratings for each. Lesion identification (ID) on x-axis describes lesion density (HU) and size groupings.
When lesion detection was examined on a case-by case basis, there were obvious differences in detection of the same lesion in some instances. These differences were more common in cases of multiple lesions but not exclusively so. To the degree to which these differences could be explained, they were due to such things as similarity between lesion and blood vessels or surrounding structures, partial-volume effect, and complexity of lesion surroundings.
DISCUSSION
It is important to remember that the original purpose of the low-dose CT scan, in this instance, is to provide attenuation correction for MPI. However, because the CT images are available, their evaluation should be considered and, some might say, required (24,25). Although controversy abounds surrounding the extent to which findings on low-dose CT acquisitions should be reported, the fact remains that incidental findings do occur. There are several factors affecting the reliability of these findings, including image quality, lesion size, and lesion density. For the purposes of this research, identifying the limitations of lesion detectability on a particular hybrid imaging system is an important objective, especially in the clinical context.
Equipment
CT rotation time is not relevant to this study, because the phantom is static and thus there are no breathing artifacts to be considered. However, the advanced technology of the Symbia T6, including ultrafast ceramic detectors enabling more effective use of x-ray exposure (26), and dose modulation, has resulted in radiation exposure doses that are half those of the Infinia Hawkeye 4. The larger matrix size used by the Symbia T6, and the adjustable display field of view, facilitates an increase in resolution. The Symbia T6 also has the advantage of allowing for multiple reconstruction kernels to be set at acquisition, enabling low-dose CT acquisition data to be easily optimized for both attenuation correction and image viewing. Although the Infinia Hawkeye 4 uses reconstruction algorithms optimized for a low-dose CT regime (27), it is unable to achieve the image quality produced by the newer technology of the Symbia T6.
JAFROC Analysis and Lesion Detection
A significant difference (P < 0.05) in lesion detection on the low-dose CT images was clearly demonstrated between the two hybrid imaging systems (P < 0.0001). More lesions were detected with more confidence on the Symbia T6, as reflected by the higher figure-of-merit value (Table 2). Although detection of lesions on the Symbia T6 appears to be more dependent on size, the effect of both size and density on lesion detection on the Infinia Hawkeye 4 is more apparent (Fig. 4). A satisfaction-of-search effect may have been seen in images with multiple lesions, because of the effect of partial-volume effect and the complexity of the surroundings, despite the use of images that showed either single or multiple lesions at their maximum visibility.
Application to Clinical Context
When breathing artifacts are factored into lesion visibility in the clinical setting, it is understandable that in our experience, some reporters may seem reluctant to report lesions seen on chest images from the Infinia Hawkeye 4. Conversely, the clarity of the images that can be produced on the Symbia T6 during low-dose CT acquisitions is readily apparent and is coupled with the fact that they have been produced with half the radiation exposure of the Infinia Hawkeye 4.
The limits demonstrated in this study for reliable lesion detection on the two hybrid imaging systems should be useful for those reporting image results. Reporters may have greater confidence about whether to attach significance to findings and whether to recommend follow-up investigations. Conversely, greater understanding of the specific technical limitations and reliability of lesion detection may decrease the number of follow-up investigations recommended and, therefore, the added radiation dose to the patient. More reliable data on this issue may ease the medical, ethical, and legal dilemmas that have arisen. Therefore, confidence levels, as used in observer performance, are clinically relevant and are important in image interpretation, for which the characteristics of the patient, imaging system, and image reporter intertwine (17).
CONCLUSION
The aim of this observer performance study was to contribute to the understanding of incidental findings and their detection on low-dose CT images obtained during MPI CT-based attenuation correction acquisitions. Lesion detection was evaluated on two commonly used SPECT/CT hybrid imaging systems. Advances in CT technology affecting image acquisition and reconstruction appear to be significant in the detection of simulated lesions.
Although only phantom-based, the results indicate potential lesion detection within the clinical context, which may assist the clinician in determining whether further diagnostic imaging is justified. Continuing research into the phenomenon of incidental findings is needed, specifically in determining limitations for the low-dose CT images and hybrid imaging systems from which these findings originate.
DISCLOSURE
This work was supported by the University of Salford. No other potential conflict of interest relevant to this article was reported.
Acknowledgments
We appreciate the kindness of Cumbria University in lending us the LUNGMAN N1 anthropomorphic chest phantom, and we thank the Central Manchester Nuclear Medicine Centre for their kind assistance. The results of this study were presented at the 2014 World Congress of the International Society of Radiographers and Radiographic Technologists.
Footnotes
Published online Jan. 22, 2015.
REFERENCES
- Received for publication August 25, 2014.
- Accepted for publication November 13, 2014.
