NMTCB Report

The first administration of the specialty exam in nuclear cardiology technology occurred at the 48th Annual Meeting of the Society of Nuclear Medicine in Toronto. Fifty-eight examinees sat for the exam, which was given in 2 separate administrations using 2 forms of the exam. The examinees were allotted 2 hours to complete the test, which consisted of 125 items. Twenty-five of the items were pre-test and were not considered in the scoring of the exam. The use of pre-test items allows for analysis of data on those items, which may then be used in future forms of the exam if they prove to be acceptable in terms of performance.

Certification as a nuclear medicine technologist is granted if an individual successfully completes a test of professional knowledge and the examinee’s score equals or exceeds a specific cutoff score. Such is the case for the specialty exam in nuclear cardiology technology as well. When a new exam is instituted, the certifying body must decide the cutoff score using one of a number of accepted methods for determining the level of performance that demonstrates competency. There are 3 general approaches to what is commonly called “standard setting.” One approach is to have a panel of experts review the exam content in its entirety and make a judgment about the overall difficulty of the exam. The second approach involves judgments made by a panel of experts looking at the content of each exam item individually in order to determine the overall difficulty. The third approach is to base the cutoff score on the observed scores of the examinees. The NMTCB chose to use the second approach, also known as the Angoff method.

A panel of 13 judges was assembled and trained on how to use the Angoff cut–score method. For the Angoff method, the judges were instructed to think of a group of minimally qualified persons and to estimate, for each item, the percentage of examinees who would answer the item correctly. In the case of the nuclear cardiology exam, a minimally qualified person would be a certified technologist with 2 years of clinical experience. Bearing that in mind, each judge went through the entire exam and looked at each item as a whole to determine the probability that a minimally qualified examinee would answer it correctly. Following that process, the exam items were reviewed by the entire panel and the judges were allowed to alter their judgment on items after discussion. The probabilities were summed over all items for each judge. The consensus of all of the judges’ ratings determined the minimum passing score for the exam based on the estimated difficulty.

Using a method such as the Angoff method allows for assessment of the level of difficulty for each item, as opposed to the holistic approach, which bases the cutoff score on the overall impression of the exam. The cutoff score was determined without knowledge of the performance of the group. One way to assess the reliability of the cut– score, however, is to compare the panel’s decision to the actual difficulty as determined from the observed scores of examinees. Item difficulty indicates the percentage of individuals who answered the item correctly and is reported in decimal form from 0.00 to 1.00. The estimation of the exam’s average item difficulty provided by the Angoff data was 0.68. The observed average item difficulty, calculated from the examinees’ actual responses, was 0.681. The high correlation indicates that the panel did an excellent job of estimating the difficulty of the exam using the Angoff method. Both the Angoff data and the observed data indicated that the cutoff score should be approximately 68%. The range of scores on the exam was 51%–89%, with a mean score of 69%.

Over the next few weeks, more analyses will be performed on the data from the exam before the official results are reported to the candidates. Congratulations to those who are among the first technologists with NCT credentials. The Board has received a great deal of feedback from the examinees, with the consensus being that the nuclear cardiology technology exam is not one that an individual can pass without properly preparing for it. At the Fall NMTCB meeting, the Board will make some decisions concerning how often and where the exam will be offered. As always, we invite your input. Call the NMTCB office at (404) 315-1739 or e-mail the Board at board{at}nmtcb.org with any questions or feedback.