Aiming to Make the Exam More Equitable

Each year, the ABGC Certification Exam Committee (CEC) has two primary tasks: 1) revise and approve new items written by item writers for the certification exam; and 2) review each item and exam form to be used in upcoming exam cycles.

In recent years, including 2022, the CEC has also focused on addressing concerns about diversity, equity, inclusion and justice (DEIJ) in the exam.

Diversity in Committee Representation

“Diversity and inclusivity are important so that the makeup of genetic counselors in the field is reflective of the clients we serve,” says CEC Vice Chair Kathryn Spitzer Kim, MS, CGC®. “There is good data that clients respond well to counselors who come from a similar background. And while that may not always be possible, having a wide range of individuals in the field will increase awareness and familiarity with issues among all practitioners, leading to better satisfaction for more patients.

The CEC’s job is to make sure the questions on the exam are as clear, fair and unbiased as possible. To do this, the committee’s members come from a wide range of specialties, work settings and backgrounds to reflect practicing genetic counselors as a whole.

“We have made a concerted effort over the last few years to ensure diversity of the CEC so that we can accomplish this goal,” Spitzer Kim says.

Diversity in Item Writing

In addition to having a diverse committee, another way the CEC is working to make the exam more equitable is by encouraging item writers to submit questions that reflect the wide range of clients that present for genetic counseling.

“We have tried to be very aware that candidates have been trained at many programs and work in many settings, so we screen questions to be sure we believe that the answers are widely regarded as correct and not only correct in certain circumstances. To this end, we are looking for updated references and might reject a question as biased even if an older reference supports it,” Spitzer Kim says.

“We are actively working to make the pool of item writers more diverse,” says CEC Chair Courtney Berrios, MSc, ScM. “We have begun asking item writing applicants if they identify with any groups that are under-represented in the genetic counseling field and are using this information to intentionally build diversity into the item writer pool.”

Spitzer Kim adds that the committee also takes candidate feedback seriously. “The best way they can ensure the integrity of the process is to volunteer to be an item writer and to join the CEC if they are asked,” she says.

Outside Counsel

Also as part of ABGC’s DEIJ efforts, in 2021, ABGC partnered with Nonprofit HR on a DEIJ Certification Assessment Report. The report included recommendations relevant to the CEC that the committee immediately worked to implement. In its review of 607 active exam items, for example, Nonprofit HR identified nine items as needing revisions or removal and an additional 38 for which they had questions that required consideration by subject matter experts. “The CEC immediately reviewed all nine items marked as needing revision or removal and either removed them from the exam bank or edited them to address the concerns, after which they’ll be placed in the exam bank for future pre-testing,” Berrios says. “The 38 items with questions were marked in the exam bank with a note about the question by Nonprofit HR that will be addressed before the items are potentially used on any future exam forms. In addition, the co-chairs reviewed the questions/concerns on the 38 items and summarized learning points from these items that are being used by the CEC as we review future exam items.”

Challenges

The biggest challenge Spitzer Kim sees in addressing DEIJ concerns is the “eagerness to have the items on the exam be sensitive to DEIJ concerns immediately.” It takes time to write, approve and pre-test questions, so change can’t happen overnight.

“Each new item that is written by an item writer must be reviewed and approved by the CEC, then placed on an exam as a pre-test item, and only if it performs well (as determined by a psychometrician) can it become a scored item on the exam,” Berrios says. “We are actively working to address biases in existing exam items immediately, while building in more inclusive items as new items are approved.”

Measuring Success

While the CEC is able to view statistics about the performance of individual items on the exam, the committee relies on the psychometrician at the exam development company to help it interpret those statistics and to use those to develop the exam. “In the past, demographics have not been collected for exam takers and we, therefore, are not able to determine how performance for the exam or specific items differs based on demographics,” Berrios says. “It is my understanding that the ABGC’s newly formed DEIJ Committee and BOD are considering collecting demographics in the future in ways that are sensitive and may support measuring outcomes.” In the meantime, the committee will continue to diversify the item writer pool and CEC while soliciting input from the genetic counseling community.