In recent years, ePortfolios have become increasingly popular in higher education as a powerful learning tool that encourages student reflection and engagement. When integrated intently into coursework, ePortfolio pedagogies and coursework can help develop among students a sense of academic identity and belonging, a connection to their college community, a confidence in themselves as learners, and a strong sense of academic empowerment that has been shown to lead to increased success (1). Beyond teaching and learning, ePortfolios have also been successfully used as tools for institutional assessment, student advisement, and career development.
At Bronx Community College (BCC), ePortfolios have been integrated across many academic departments and programs over the last 15 y. Most notably, the integration of ePortfolios in the first-year seminar, which a majority of incoming first-year students take in their first semester at BCC, led to increases in student reflective practice, engagement, and analytic skill development. This in turn led to markedly increased passing and retention rates when compared with non–ePortfolio-integrated sections of the same course.
Similar increases in student success have been noted in ePortfolio-integrated courses in other academic disciplines, such as business, health sciences, communications, and history. At the center of effective ePortfolio pedagogic practice has been peer engagement and faculty feedback, which encourage students to engage with ePortfolios not simply as a technologic platform (such as a learning management system, for example) but as a more complex interface that combines the academic and the personal.
In our department, we saw the potential of using ePortfolios in a new manner, combining their utility as an academic content repository with their assessment capabilities in a virtual space that engages students, encourages collective collaboration, and develops a sense of community among students that endures even after they graduate from our program.
In a post–coronavirus disease 2019 (COVID-19) academic environment, the methods of assessing program or student learning outcomes have changed. Assessment techniques have changed because how students interact with the curriculum has changed. Often, new ways of curriculum implementation are materializing as new and exciting web-based strategies. These strategies can be brand new concepts to the program or can be improvements on an already existing concept. At BCC, the use and implementation of the nuclear medicine technology (NMT) ePortfolio have evolved to the point that ePortfolio has become much more than a repository for a student’s assignments and discussion; it is now an actual primary resource for the program.
The Joint Review Committee on Educational Programs in Nuclear Medicine Technology (JRCNMT) annual report has new guidelines on program assessment. After we completed the required forms J and L, it became clear that our assessment methods within the program needed to be streamlined. Our data and resources required for this process were spread out, creating challenges in completing these forms on time. In addition, it became clear that locating the program resources was challenging for students. Previously, we were constantly uploading resources for current courses on Blackboard Learn (Anthology Inc.) each semester, which would be inaccessible to the student once the course was completed.
To address these problems, a program ePortfolio was created and has been implemented this semester. This digital platform allows us to consolidate program resources in one area while allowing us to share information with both current students and recent graduates.
CURRENT STUDENTS
For students currently enrolled in the program, we now have several program resources accessible in one organized location. It is our hope that students will become familiar with this platform, allowing them to frequently access the ePortfolio, acting as a primary resource. Some features available in the ePortfolio include advisement materials, program guidelines, clinical resources, and licensure information.
Advisement Materials
The nuclear medicine program at BCC does not currently have a certification program and is contingent on the student’s obtaining an associate of applied science degree. To ensure that students are on track with the pre- and corequisites, we posted a nuclear medicine pattern sheet identifying these degree requirements. When a student applies for the program, the program director and advisors ensure that the prerequisites have been met; however, corequisites to the degree can be taken at any time during the program. Examples include classes on such topics as history, art, and communications, which must be taken to obtain the associate of applied science degree but are not necessarily required before program entry. A pattern sheet is a helpful tool that allows students to ensure they are on track to graduate on time.
In addition, we thought that a simple link to the college course catalog would be useful for further advisement information. The catalog contains course descriptions, credits per hour, and details on the associate of applied science pathway and on program length and expectations.
Contact information for the program director, clinical coordinator, adjunct lecturers, and program advisors has also been provided in the ePortfolio, again allowing quick access to this information. We believe that having this resource readily available in one location will ensure that students can quickly get in touch with us when needed. In addition, we have also provided contact information for helpful student resources including college information technology, disability, and counseling services. With the challenges associated with the pandemic, we find that having quick contact information for these resources is more important than ever.
Program Guidelines
In addition to the advisement materials, we thought it was important to make the program handbooks easily accessible. In the past, the program director redundantly posted these handbooks on our online learning platform, Blackboard Learn, in the students’ courses. The problem with this method was that once the students completed the course, they no longer had access to these materials. With the ePortfolio, we are now able to compile all program guidelines in one location, again easily allowing students to access this material. Students within the program are given both the program handbook and the clinical handbooks. The program handbook contains program expectations, standards, and policies, whereas the clinical handbook contains regulations relating to their clinical internship. Accessing both documents is important, and the student can now do so in one simplified location.
Clinical Resources
Compiling clinical resources has always been challenging, as we have several internship sites with different hospital affiliates. After the pandemic, putting these resources together has become difficult, as several of these sites now have additional forms to complete. By using the ePortfolio, we are now able to easily upload, to one location, the documents that the individual sites require, allowing students to quickly find the information they need.
Competency forms are continuously changing, adjusting to the procedures currently being performed in the field. Our program uses the American Registry of Radiologic Technologists competency forms for nuclear medicine procedures and the Nuclear Medicine Technology Certification Board forms for CT procedures. In the past, use of these 2 different sets of forms confused some students, who often needed us to provide additional clarification when explaining how to properly complete them. Now, with these forms accessible in ePortfolio, we not only have them available to print wherever the student may be but also provide anonymized forms from the past as examples of how to fill them out.
Other forms that have previously caused some confusion include those provided by the hospital volunteer offices. Since the pandemic, there have been several new forms associated with immunizations and other health clearance procedures. Because there are many different medical centers affiliated with our program, these forms can become confusing and cumbersome. With the ePortfolio, we now have individual drop-down menus associated with each hospital group, identifying procedural steps required to obtain clearance. We hope that this will make the clearance process more concise and organized for both the students and the clinical coordinator.
Licensure Information
With the option of 2 different nuclear medicine board examinations, 2 different CT examinations, and the variations in state regulations, sharing of licensure information can be complicated. With the organizational strategy of the ePortfolio, we can now easily display material for each of the certification examinations in addition to the health department requirements. This information can be helpful for both current students and recent graduates.
Board examination information can be complex, as the American Registry of Radiologic Technologists and Nuclear Medicine Technology Certification Board have several differences in the application process, examination administration, and scoring systems. Since using the ePortfolio, we have been able to include separate menus for the American Registry of Radiologic Technologists and Nuclear Medicine Technology Certification Board, highlighting application information, the content of preparedness, test administration, scoring details, and helpful contact information. In addition, we have provided a link to the New York State Department of Health for students to easily access once they pass their licensure examinations. We believe that these enhancements should help clarify any information that might previously have been misconstrued, as well as streamline the process of obtaining licensure.
RECENT GRADUATES
Recent changes to the ePortfolio content have contributed to making the ePortfolio a more relevant resource that is enjoyed by both recent graduates and current students. Once students graduate, they can find it challenging to maintain communication. Often, students no longer access their school e-mail accounts, making it difficult for them to stay in touch. We hope that the ePortfolio will serve as a continued resource for graduates, allowing them to keep in touch with the program while obtaining access to program updates, surveys, and job postings.
NMT UPDATES
Within the NMT updates section of the ePortfolio, we offer updates on the program as well as the profession. Program updates will include announcements regarding student conferences, grant opportunities, and special achievements. Professional updates will link students to the Society of Nuclear Medicine and Molecular Imaging website, highlighting recent news about radiopharmaceuticals, industry information, and advocacy opportunities. Recent legislature, including the Facilitating Innovative Nuclear Diagnostics Act, is directing attention to our profession, discussing the need to fight for appropriate insurance reimbursement with Medicare. Legislation such as this can help grow our profession while granting patients access to vital nuclear medicine procedures. It is more important than ever that new technologists follow essential professional updates such as these while advocating for the field of nuclear medicine.
SURVEYS
Postgraduate surveys are crucial for maintaining accreditation standards and improving assessment strategies. Because these strategies must be distributed months after graduation, it has been particularly challenging to ensure that students are receiving these surveys, and it has been even more challenging to ensure that students are completing them. Online distribution methods such as Survey Monkey (Momentive) have helped streamline survey administration, but participation is still not where we want it. We plan to now offer a link to these surveys on our ePortfolio, allowing for student reminders to complete them. Because students are already familiar with using the ePortfolio as a primary resource, we believe that they will return to the ePortfolio for the resources discussed above and the added incentive of job postings.
JOB POSTINGS
The post–COVID-19 job market is something our profession has not witnessed in many years. There is a major shortage of technologists throughout the country, with many hospitals and clinics continuously looking for employees. Both the program director and the clinical coordinator are constantly receiving calls and e-mails asking if any recent graduates are looking for work. As much as we try to stay on top of these opportunities, it is difficult to maintain communication with recent graduates, and it can be impossible to know which of them may be looking to change or expand their current employment. Our ePortfolio now contains a job posting section that allows us to easily post these opportunities, allowing graduates with access to the ePortfolio an opportunity to review job postings within their area. We believe that this feature will provide an incentive to return to the ePortfolio after graduation, aiding in the job search while allowing continuous access to any of the other resources. In addition, we hope that this return to the ePortfolio will remind students to complete the surveys that will improve our assessment of the program.
ASSESSMENT
Because of the anticipated increase in student participation with the ePortfolio, this online resource can now be used as an assessment tool, focusing on, among other things, program accreditation. Furthermore, new methods of managing statistical data, program requirements, and worldwide communication have fundamentally influenced day-to-day interaction, resulting in a more fluid transfer of knowledge.
At BCC, the collaborative effort between the nuclear medicine program and the college’s assessment council has recognized the need to move assessment beyond the boundaries of the campus itself. After internal evaluation of course and program statistical data, it became clear that to properly evaluate the program’s effectiveness within the industry, assessment must also include, when possible, past students of the program, cooperating clinical partners, and the medical facilities that ultimately hire our graduates. Without this feedback, program assessment is limited to data obtained only from on-campus resources and is incomplete. To truly understand how effective the program is to the industry, postgraduate information must be gathered and analyzed.
Implementing an ePortfolio as an assessment tool and a primary program resource allows the college to reach beyond the confines of the campus and remain in contact with the industry and local partners. However, this reach comes with a potential price. Although access to such resources was never so readily available in the past, it still requires a higher level of participation and interaction to be effective. Unlike physical resources that can be presented by the instructor while engaging a class, clinical partner, or potential employer, the use of online resources is very much voluntary and therefore, in many cases, can yield a lower rate of returns.
To support this claim, Chris Efthimiou, director of the Office of Institutional Research and Testing at BCC, has found that the pre–COVID-19 collegewide teacher evaluations, known to the college as Student Evaluation of Instruction, conducted in person during the fall semester of 2019 yielded a collegewide 53% response rate. Contributing factors to this response rate include student attendance on the day of the survey and timely and proper faculty facilitation of the survey. There are also reported instances of students opting not to complete the survey. It was speculated that these students were not willing to complete the survey because they feared retaliation by the professor. Taking these factors into consideration, the college deemed the pool of respondents to be large enough and therefore to satisfy statistical relevance for the intended purpose. On the basis of a 53% rate of return, the college was able to move forward with analyzing the data, substantiating the results.
However, during the COVID-19 pandemic, online surveys of the same Student Evaluation of Instruction e-mailed to students were being returned at a rate of only 13.5%. With such a low rate of return, the data obtained from these online evaluations may not be an accurate representation of the sentiment of the student body. According to Gainsight, a leading authority on business–customer relations, reasons for a low response rate for online surveys include “lack of personality, poorly crafted subject lines, survey fatigue, poor survey timing, lack of incentive, and possibly lack of closure” (2).
To address these common pitfalls of administering online surveys, the pool of potential respondents was analyzed. At BCC, the NMT program is designed for students to progress through it as a cohort. This type of enrollment allows students to establish a level of comfort and familiarity with each other that ultimately promotes an atmosphere of success. Students tackle the rigors of the NMT courses with a collective effort and therefore consider themselves a team. This team mentality also builds loyalty to the program, the college, and ultimately the industry. When various NMT students were asked about their overall experience with the ePortfolio, the sentiment reflected was a feeling of belonging and relevance that can encourage plentiful participation. According to a senior-level NMT student, “The students embrace the ePortfolio as ‘theirs’ and not simply a resource provided by the college.”
Because of its popularity and solid engagement, the ePortfolio of the NMT program is being expanded to also act as a repository for data that will be used to satisfy JRCNMT accreditation requirements. Among other required forms and benchmark indicators, the Program Effectiveness Data Form (form L) is an assessment tool used by every accredited NMT program in the nation. This form deals largely with postgraduate student performance in the industry and on the overall health of the program at a college level. Although not every metric on form L is appropriate for students to address through ePortfolio, there are some metrics for which data can be gathered through the use of surveys embedded into the ePortfolio.
The postgraduate follow-up survey embedded into the ePortfolio is a user-friendly, nonintrusive assessment vehicle that addresses multiple metrics listed on form L. To achieve the best possible results, the survey attempts to gather postgraduate information using only 16-Likert-scale, multiple-choice, or short-answer questions. The survey has a dedicated tab on the ePortfolio home page, requiring only one click to access. Among other datasets, the questions focus on 3 required metrics listed on form L (3): the D3.1c, D3.1e, and D3.1g standards.
The D3.1c Standard
First, standard D3.1c deals with the employment of past or recent graduates. The survey has 2 questions that directly address this standard. Furthermore, there are 3 supplemental questions whose answers provide data on the quality of employment, such as data on full-time, part-time, or per-diem types of jobs. Although not a direct requirement to differentiate between types of jobs, this information is invaluable to the program director for future curriculum development, as well as to advisory board members who are active employees, managers, or clinical coordinators in the nuclear medicine field.
The D3.1e Standard
Another standard that is directly addressed in the survey is D3.1e. This standard deals with student assessment of individual didactic courses, clinical experiences, and faculty. At BCC, student assessment of individual didactic courses is satisfied with the Student Evaluation of Instruction deployed at the end of each course. Since this requirement is addressed directly with another college survey, questions about the didactic courses were not included in the postgraduate survey found in the ePortfolio. However, the postgraduate survey contains 4 questions that directly relate to the student’s clinical experience and 3 other questions that indirectly probe into clinical experiences. Again, the information gathered by these questions satisfies not only the JRCNMT requirements but also the internal program assessment conducted at the college.
The D3.1g Standard
The last standard addressed by the postgraduate survey is D3.1g. This standard explores the graduate’s assessment of program effectiveness. Again, there are 3 questions that directly relate to how adequately prepared the graduates feel to enter the workforce as a nuclear medicine technologist. These questions refer to a myriad of skills the technician should possess on graduation, listed as program-level outcomes in the college catalog. The answers to these questions are used both for college-level program assessment and for accreditation purposes for the JRCNMT.
The postgraduate survey found on the ePortfolio page of the NMT program at BCC was designed to take no longer than 5 min to complete. Because this survey is short, it reduces survey fatigue. Another characteristic promoting participation is lack of a due date. In many cases, postgraduate surveys fail because the graduate is not yet prepared to answer the questions, possibly because of lack of employment or other personal issues. When a survey is open-ended, graduates can complete it when they have more time or experience. It would not make sense to have graduates answer a question about preparedness if they are not used to reflecting on their experience.
Assessment of the NMT program at BCC is persistent, thorough, and ongoing. The assessment techniques are constantly evolving through evaluation of past strategies and implementations. Closing the loop is a philosophy deeply embraced by the assessment council at BCC. Through biannual program assessment and annual course assessment, combined with program accreditation standards, BCC hopes to be a leader in assessment techniques and strategies. Only through due diligence will we be able to produce assessment results that can change the value of the curriculum, increase the effectiveness of instruction, and maintain a program that remains relevant in an ever-evolving industry.
The integration of the ePortfolio for the NMT program at BCC has established a new way to consolidate program resources and provide more direct access to advisement material, general program guidelines, and licensure information. By having content that is readily available, up to date, and relevant for both past and current students, ePortfolio can become a primary resource for industry updates and opportunities. The BCC assessment council, in conjunction with the director of NMT and the college administration, will continue to monitor the use and effectiveness of this resource. ePortfolio is projected to act as a guide for JRCNMT annual reports, accreditation requirements, and college assessment needs.
DISCLOSURE
No potential conflict of interest relevant to this article was reported.
Footnotes
Published online Aug. 16, 2023.
- Received for publication May 1, 2023.
- Revision received July 13, 2023.
