Nuclear Cardiology Specialty Examination Technical Report

OVERVIEW

This report summarizes the development, administration, and scoring of the nuclear cardiology specialty examination offered by the Nuclear Medicine Technology Certification Board (NMTCB) at the Annual Meeting of the Society of Nuclear Medicine in Toronto, Canada, on June 23, 2001.

During the NMTCB meeting in the fall of 2000, the Board approved the development and administration of an advanced practice nuclear cardiology examination in response to a formal request from the Society of Nuclear Medicine−Technologist Section and the Nuclear Cardiology Committee. Many years of experience in developing and setting standards for the entry-level nuclear medicine certification examination aided the NMTCB in taking on this new responsibility. From its existing members, the NMTCB formed an ad hoc nuclear cardiology specialty examination committee composed of technologists, educators, a pharmaceutical specialist, a physician, and psychometric staff. The task of writing the examination began immediately.

PURPOSE AND REQUIREMENTS

The primary purpose of this specialty examination was to assess the knowledge beyond the entry level of an experienced nuclear cardiology technologist. Eligible candidates who took the examination were certified in entry-level nuclear medicine technology and had a minimum of 2 y full-time clinical experience in nuclear cardiology. Holding an active NMTCB, RT(N), or CAMRT certification fulfilled the entry-level certification requirement. The NMTCB office in Atlanta processed all applications, including verification of candidates’ entry-level certificate and 2-y work experience. The certification received by candidates after successfully passing the examination is valid up to 7 y from the examination date and will expire thereafter unless the individual chooses to re-certify. Re-taking and successfully passing the specialty examination could accomplish re-certification. Upon successful completion of the examination, the technologists were granted the right to use the title Nuclear Cardiology Technologist (NCT).

EXAMINATION DEVELOPMENT

A subject-matter expert (SME) group was involved in identifying critical abilities related to the practice and in defining test specifications. This SME group included 5 members of the ad hoc committee and 5 professionals practicing in the nuclear cardiology field who came from different regions of the country and from different work settings for a total of 10 members. The Guidelines for Technologist Training in Nuclear Cardiology (1), developed by the technologist committee of the American Society of Nuclear Cardiology, a leading professional organization in the field, was used to verify and refine the content domain for the examination. The SME group evaluated relevant documents and identified the examination content outlines and assigned a percentage to each piece of content by consensus. The SME group also determined the test length to be 100 multiple-choice items with 4 options and identified an appropriate testing time to be 2 h. The ad hoc committee members wrote most of the test items. The SMEs and other members of the Board completed the pilot test. After the pilot test, the group worked together to finalize the content outlines for the test, percentage of the total number of items to be used from each content area, as well as test items to be used on the test. For security reasons 2 test forms, A and B, were developed. By scrambling the item order of form A, form B was created.

The content outline with the percentage specified in parentheses from each content area is presented below. For a detailed list of the content outline, procedures, and pharmaceuticals list see the Appendix.

EXAMINATION ADMINISTRATION

A total of 59 candidates were registered to take the first nuclear cardiology examination in Toronto. Only 58 candidates completed the test, 36 in the morning (Form A) and 22 in the afternoon (Form B). Candidates were checked into the testing room by verifying their identification with the roster prepared by the Board office. The candidates were not allowed to choose their own seats but were assigned seats by the proctor. The testing was administered at the Fairmont Royal York Hotel, Toronto, Canada. The examination room provided adequate writing surface, uncrowded seating, good lighting, comfortable temperature, and freedom from distraction. The test included 100 scored and 21 pretest items. All graphics that were part of the multiple-choice items of the test were collated in a separate booklet. Candidates were given a total time of 2 h to complete the test. The proctor began the session by reading a standard set of examination instructions. Candidates were required to sign a form for maintaining the confidentiality of the test questions. They were given notification of the remaining testing time at 30, 10, and 5 min from the end of the 2-h allotment. The morning group of candidates was not dismissed after completing the test, but was escorted to a different room until all of the candidates in the afternoon session were checked in. All candidates were encouraged to submit written comments about the test on the back of the examination booklet. In each session only 5 candidates were still in the examination room when the proctor made the final, 5-min-remaining announcement of the examination time.

TEST SCORING

Before scoring, all answer sheets were verified as to their accuracy. Answer sheets from the 2 sessions were machine-scored separately. Multiple response and blank responses from candidates’ answer sheets recorded by the machine were verified. Upon verification it was found that the machine recorded multiple responses for incompletely erased bubble marks and blank responses for lightly bubbled responses. These errors were corrected before generating a preliminary item analysis report. The analysis report was conducted to detect key errors and items that were not performing properly. Members of the ad hoc committee reviewed the item analysis report and comments received from candidates. As a result of the review, 6 items were removed from scoring and 3 items were scored with a double key. All candidates were graded on 94 test items. The passing cutoff score for the test was set to be 75 on the scaled score. The reliability (KR-20) of test forms A and B were 0.86 and 0.83, respectively.

Forty-four of the total 58 (76%) candidates passed the test. Examinees who failed were provided feedback with a breakdown of their performance in the 5 content areas. Examinees were allowed to request hand scoring for a certain fee; however, these examinees were prevented from reviewing their examination booklets for the security of test items. The descriptive statistics of the scaled score were as follows: N = 58, minimum = 64.62, maximum = 93.71, mean = 79.7184, SD = 6.9551. Table 1 gives the number of items based on the range of difficulty and discrimination indexes on both test form A and B. The histogram of the examinees’ scaled scores is presented in Figure 1.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1.

Histogram of NMTCB nuclear cardiology examinees’ scaled scores.

STANDARD SETTING

A modified Angoff method was used to establish the cutoff score for the nuclear cardiology specialty examination. A panel of 13 judges set the cutoff score by rating, for each item in the test, the probability that a hypothetical candidate would answer the item correctly. In this case, the group of hypothetical candidates consisted of certified technologists with 2 y experience in nuclear cardiology. The members of the panel included 4 cardiology experts and 9 directors of the NMTCB. Some of the judges in this panel were experienced in setting cutoff scores for certification examinations. As a training process, judges were asked to share their rationale for the rating they assigned to the sample item. Everybody in the panel felt comfortable before they began rating the test items. The average cutoff percentage of the 94 scored items obtained from the standard setting session was 68.18%, and this was transferred for reporting purpose to the scaled score of 75, the scaled passing cutoff score. The coefficient α, or internal consistency among the judges, for this cutoff score was 0.82.

SUMMARY

In summary, the examination appears to exhibit appropriate statistics for a credentialing examination. The job analysis model and cutoff-score study described in this report help to substantiate the validity of the resulting examination scores.

APPENDIX