NMTCB REPORT

The NMTCB is well versed in the development and delivery of the certification exam for entry-level technologists. The Board has established guidelines for keeping the exam current and focused through the use of a practice analysis survey, which is conducted every 5 years. The results of the survey help to define the exam content as technology evolves and changes take place. This process of evaluating practice trends, revising the exam to reflect the results, and implementing the changes takes approximately 2 years to complete. While practice analysis, or content analysis, is a well-accepted method for defining exam content, there are other methods that can be used by the test developer as well.

When the NMTCB decided to develop a nuclear cardiology specialty exam, one of the first considerations was the method to be used to define the content of the exam. The Board wanted to develop and implement the exam in the most expedient manner possible. In order to avoid the lengthy content analysis method, the Board opted to utilize the expert judgment method, in which input is sought from individuals who have firsthand experience within the domain of interest. The Executive Director of the NMTCB, Dr. Bhaskar Dawadi, provided critical input in the area of exam development. His background in measurement and statistics gave the Board the necessary expertise to make decisions about the exam development process. The members of the NMTCB ad hoc committee for the nuclear cardiology specialty exam were chosen for their expertise in nuclear cardiology, education, and exam development. With added input from other experts in the field of nuclear cardiology, the Committee identified the critical abilities related to the practice. The guidelines for technologist training in nuclear cardiology, which were developed by the American Society of Nuclear Cardiology (ASNC) Technologist Committee, were used to verify and refine the content domain for the exam.

The next step in the development process was to prepare the test specifications, or the exam blueprint, which specifies the item format to be used and the proportion of items that focus on each of the identified critical abilities, or behaviors, to be tested. For the nuclear cardiology exam, the items are in multiple-choice format with 4 possible responses. The proportion of items for each subset is as follows: Instrumentation/Procedures/Processing (50%), Anatomy/Physiology/Pathology (10%), Radiopharmaceuticals/Interventional Drugs (10%), Patient Care/ECG/CPR/Patient Preparation (15%), and Non-Pharmacologic Stress Testing (15%).

The most difficult aspect of test development is constructing the initial pool of items. Each Committee member was assigned a specific area of the exam content to focus on and to create items for that would test knowledge of that content area. The items were then pooled together and the draft was sent out for the Board to review for accuracy, wording, grammar, content, and other technical flaws.

With the revisions complete, the next step was to conduct item tryouts. To accomplish this, the Board invited a group of technologists to Atlanta to take the exam. Following the exam, a debriefing was held to give the invited examinees an opportunity to comment on each item and offer suggestions for improvement. The field test of the exam was used to gather statistics on the difficulty and discrimination ability of each item. The difficulty of an item, its p-value, is the proportion of the group that responds correctly. Ideally, the p-value for an item should be between .60 and .70 (which can be interpreted as 60% - 70%) because the item’s contribution to the total score variance is maximized in that range. Discrimination is closely related to difficulty and gives an indication of how well the item differentiates between high and low achievers. If a large proportion of high achievers (those who pass the exam) correctly answer an item and a small proportion of low achievers (those who fail the exam) incorrectly answer the same item, that item has discriminated well. If more low achievers than high achievers answer a certain item correctly, that item is a negative discriminator and should be eliminated. Items that are too hard or too easy are not capable of discriminating as well as those items of moderate difficulty. With those factors in mind, poor performing items were revised or eliminated from the pool. In addition to the item analysis, the field test also allowed the cut score to be determined. The arduous task of development was complete and the final version of the exam was printed and prepared for administration.

The first administration of the nuclear cardiology specialty exam will be held in conjunction with the Annual Meeting of the Society of Nuclear Medicine in Toronto in June. A tremendous amount of work went into the exam development and the ad hoc Committee completed its task in a short period of time. The members of that Committee are to be commended for their efforts. Kathleen Murphy, MS, chaired the Committee. Her experience in nuclear cardiology and her background in education, coupled with her desire to see this project through, were critical to its success. The members of the Committee were Gary Dillehay, MD; Tony Knight, MBA; Vivian Loveless, PharmD; and Pat Wells, MAE. As always, the devoted staff at the NMTCB office in Atlanta contributed greatly to the success of this endeavor.