Why Delay Scoring on August’s Exam?

Future genetic counselors who take the certification exam during the August 2023 cycle can expect to receive their scores six to eight weeks later than usual. Typically, exams are scored instantly, but this temporary change (exam scoring will be instant again in February) is being implemented as the final step toward improving the exam based on changes to our practice analysis, which ABGC’s testing vendor, PSI, updated last year.

ABGC spoke with Dr. Frank Williams, a psychometrician at PSI, as well as ABGC board member Angie Trepanier, MS, CGC, to answer a few frequently asked questions that you may have as an educator or friend of someone taking the exam during August’s window.

What will delay scoring?

Once the exam window closes, Dr. Frank Williams will spend a week evaluating newly added exam questions to gauge if they are performing well or not. Depending on the results of this evaluation, he may choose to re-score the exam.

Once this is done, ABGC’s Standard Setting Task Force will examine how the items performed and provide input on appropriateness and difficulty. They will answer tough questions regarding each exam question, such as, “what percent of competent CGCs would answer this question correctly?”

This information will be used to set a board pass set point (a passing score) for the August exam and all exams moving forward under the current practice analysis. Once the set point is established, all future exams conducted using the current practice analysis will have instant scoring.

The Standard Setting Task Force will be composed of 12 volunteers from a variety of work settings and roles.

What is a practice analysis, and what was PSI’s process for conducting one?

A practice analysis (also called a job analysis) evaluates what knowledge professionals currently have to have and what tasks they currently perform in their professional roles. Certification exam content is developed based on the practice analysis.

A practice analysis is conducted every five years to ensure an exam will reflect current practice, including any changes in knowledge and tasks since the last practice analysis. ABGC’s previous practice analysis was conducted in 2017.

To develop the new practice analysis, ABGC first convened a Practice Analysis Task Force. Members were selected via a call for volunteers to represent diverse roles, practice areas, years of experience, and geographic locations as well as other aspects of diversity (e.g., race, ethnicity, gender identity). PSI then worked with the Practice Analysis Task Force to identify knowledge areas and tasks performed in current genetic counseling practice. This work led to the development of the practice analysis survey. The purpose of the survey was to assess how often genetic counselors use each knowledge component and how often they perform each task in their jobs. Respondents could also indicate whether there were knowledge areas and tasks they used that were not included in the survey. The survey was distributed to genetic counselors through ABGC and professional organizations, and the survey’s results were then used to develop the exam specifications.

Is the new test very different from before?

One of the changes made to the Exam Content Outline (ECO), which item writers reference when creating questions, was to make questions more knowledge-based instead of chronology-driven and task-based. Previously, the ECO was bound by the order in which test takers would accomplish tasks. PSI also changed the practice analysis so that it is less granular. All of these changes will make it easier for item writers to create exam questions.

Changes made to the exam content outline only happen after a new practice analysis has been performed. Other changes, like updating exam questions to reflect new technologies and improving questions to make them more inclusive, are made annually by the Certification Examination Committee and the work of item writers. Despite changes to the content outline, Frank explained PSI was able to continue to use around 90% of previously written items in the new versions of the exam.

What should I tell my student/ friend who is nervous about delayed scoring?

ABGC recognizes that delaying results reporting for the August exam cycle may be nerve-wracking for examinees and that it impacts the timeline to achieving full licensure. Angie certainly empathizes. However, she explained, “ABGC must go through the standard setting process in order to ensure that the new forms of the exam are a valid measure of competence. The board exam is a high-stakes exam since the results may be tied to licensure and professional practice. As such, ABGC must follow established procedures to ensure a valid examination.”