Your Questions Answered: FAQ From the ABGC Townhall

Dear ABGC Community:

We want to express our sincere appreciation to everyone who participated in the ABGC Town Hall on October 12, 2023, dedicated to addressing questions stemming from the August exam cycle. For your convenience, a link to the recorded Town Hall can be found here.

Your active engagement was evident, with over 700 questions generated during the session, showcasing the deep involvement and commitment of the ABGC community.

To promote transparency and responsiveness, the ABGC staff and Board meticulously reviewed each question, categorizing them based on topics and themes. The questions coalesced into six primary themes:

1. General questions about exam development, including:
   • How is the practice analysis conducted? 
   • How was the cut score/passing point determined?
   • What have ABGC exam pass rates been historically?
   • How is the exam reviewed for bias? 
2. General questions about exam delivery, including:
   • How are accomodations made for those who need them?
   • How is ABGC planning to improve the exam experience?
   • How is exam cost determined?
3. General questions about exam resources, including:
   • What resources are available for exam candidates?
   • Do you offer a practice exam?
4. Specific questions regarding the August exam cycle, including:
   • What action was taken for August candidates who did not pass their exam?
   • Can a test score be invalidated?
5. Need for transparency about ABGC’s affiliations and interactions with external entities, including:
   • Why is a test vendor needed?
   • How do the different genetic counseling organizations work together?
   • What is ABGC's relationship with the National Commission for Certifying Agencies (NCCA)?
6. Call for better communication and accountability
   • What is the Board's position on recent issues?
   • What are ABGC's immediate short-term plans and goals?
   • What are ABGC's long-term goals?

The sheer volume of questions and comments we received reflects the high level of engagement from the ABGC community and emphasizes the need for ABGC to enhance information dissemination. Your proactive involvement is instrumental in shaping the future of our certification process. In the document below, we have compiled comprehensive answers to address questions within the above categories. Our aim is to provide clarity and transparency while ensuring that your concerns are heard and addressed.

We also want to acknowledge that in addition to questions posed during the town hall, we received many comments through which participants expressed their anger, frustration, disappointment and lack of faith in ABGC. Although we do not address these comments in our responses to the questions, we want you to know that we hear you. We are working diligently to improve the testing experience, provide the level of service our candidates and diplomates deserve and regain your trust.

Should you have further inquiries or if there are additional questions not covered in this summary, we encourage you to reach out to ABGC via email. Your feedback is invaluable as we work to provide enhanced information and strengthen our certification process.

Sincerely,

The ABGC Board of Directors

Exam Development

How is the practice analysis conducted? 

A practice analysis (sometimes referred to as a job analysis or job task analysis,) is a scientific inquiry conducted to identify the tasks and work activities performed, the context in which those tasks and activities are carried out, and the competencies (knowledge areas, skills and abilities) required to perform a job role successfully. When completed, the practice analysis process yields exam specifications, in the form of a content outline, that accurately reflect the scope of practice of current genetic counselors, allowing for the development of fair, accurate and realistic assessments of candidates’ readiness for certification. The practice analysis is typically performed every five years. Because it serves as the primary basis for content validity evidence, as required by certification standards, the practice analysis is a primary mechanism by which a certifying body can ensure the accuracy and defensibility of an exam. It serves as the foundation of the certification exam and is critical to the success of the entire exam development process.

To conduct the practice analysis, through a call for volunteers, ABGC selected a committee of genetic counselors who represent a wide variety of work-related characteristics such as years of experience, work setting, geographic location, training program, and areas of specialty. These individuals are considered subject matter experts (SMEs). ABGC also purposefully sought to ensure that SMEs represented the diverse backgrounds of future examinees in terms of race, ethnicity and other factors. Having diverse representation is necessary to develop a scope of practice that is reflective of the various roles and responsibilities of genetic counselors and to develop an exam content outline that is relatively free from bias.

PSI Services LLC (PSI) convened a Practice Analysis Committee meeting in January 2022 with SMEs to discuss the scope of practice and develop a list of tasks and knowledge areas that reflect the practice role. PSI led the SMEs in refining task and knowledge statements. The outgoing exam content outline was used as a resource when developing the knowledge and tasks as were documents such as the ACGC practice-based competencies, the NSGC scope of practice, the ACGC standards of accreditation and other key frameworks that describe genetic counseling practice. Through two days of meetings, the committee developed 40 task statements (tasks GCs do). They also identified 76 knowledge statements over 14 content domains.

PSI, with the assistance of the practice analysis committee, then developed, administered and monitored a survey to validate the tasks and knowledge areas identified by the committee and to help determine content weighting. To this end, the survey collected respondents’ ratings of the importance and frequency for each task and knowledge area. The importance and frequency scale were used to evaluate how important each task or knowledge topic was to a person’s practice and how often they used/taught that knowledge topic/used the task. The survey was sent out to approximately 5900 ABGC certified genetic counselors. A total of 717 people took part in the survey; their demographic characteristics were similar to those described in the NSGC Professional Status Survey.

The Practice Analysis Committee convened for two more days in June 2022 to refine the list of knowledge and task statements based on the results of the practice analysis survey, went through a process of ensuring every task mapped back to a knowledge statement, and developed a final weighted content outline. Knowledge statements were organized into domains with subdomains. The final content outline includes five domains, 14 subdomains and 75 knowledge statements based on 37 tasks. The content outline serves as the basis for developing the certification exam. The content outline specifies that approximately 20-25% of the questions on the exam are recall questions, 75-80% are application/analysis questions. The most current content outline is available here. Of note, the knowledge/tasks identified in this practice analysis overlap significantly with those that formed the basis of the previous content outline.

How was the cut score/passing point determined?

The first time a new content outline is used to develop a certification exam, a new exam cut score (passing point) must be established. ABGC, through its testing partner PSI, uses a modified Angoff method to establish the cut score. The modified Angoff method is used to set cut scores for many professional licensing exams and in education to set passing scores for high stakes exams such as state-mandated tests. It is one of the more common ways to set a defensible cut score for high-stakes certification exams. Defensible means that the necessary steps have been taken to establish the minimum score required to demonstrate competence via examination.

With this approach, subject matter experts (genetic counselors with various areas of expertise and years of experience) individually go through every exam item to judge the probability that a minimally competent person could answer the question correctly. In other words, they are assessing how difficult each question is on a scale of 0 (most difficult, no one gets it right) to 100 (very easy, any test taker should get it right). Based on their combined assessments over two rounds of rankings with discussions in between, a cut score is established.

For the most recent exam, 12 subject matter experts participated in this process. Once the process was complete, PSI presented the ABGC Board with the panel-recommended cut score (mean score based on the assessments described above) and the plus and minus one and two standard error of the mean (SEM) scores. The SEM scores incorporate the variability that may occur in setting a cut score based on the subject matter experts selected. The Board reviewed the information and approved a cut score of 121 (minus two SEM). This cut score will be used for future administrations of exams developed based on the current content outline. Although the cut score will remain the same, the pass rate may vary from cycle to cycle.

What have ABGC exam pass rates been historically?

As of the current moment, the pass rate stands at 67% for the most recent exam cycle. However, we are awaiting the results from the November/December candidates to calculate the final pass rate. Below is a table that reviews the pass rate for the past seven years. In 2024, ABGC will be launching a task force to look at the pass rates and what factors may have affected the candidates’ performance on the most recent CGC exam.

How is the exam reviewed for bias? 

Some genetic counselors have voiced concerns that the ABGC Certification Exam is biased such that people who are under-represented in the profession have lower pass rates. ABGC has not historically collected demographic data. As a result, the organization has not been able to determine whether the pass rate is influenced by specific demographic variables. In the absence of having demographic information to do such analyses, ABGC has taken steps to reduce the risk of bias in the exam. Certification Examination Committee (CEC) members are now required to do training in implicit bias. In addition, two years ago, ABGC partnered with an organization, NonProfit HR, to evaluate all questions in its exam question bank for bias. Any questions with potential bias were modified or removed. NonProfit HR also examined all aspects of ABGC’s examination processes from development through administration to look for factors that could impact the exam experience for candidates. These steps did not identify any major sources of bias.

ABGC recognizes that there are structural factors that may lead to bias in any type of standardized test-taking. As such, the Board tasked the DEIJ Committee in 2023 to develop a list of demographics questions and unanimously voted to start collecting demographic information beginning in the April 2024 application cycle for the August 2024 exam. Additional information about how demographic data will be stored and used is forthcoming, as well as our plans to continually monitor and correct for any potential biases identified in the exam and/or exam processes.

Exam Delivery

How are accomodations made for those who need them?

ABGC recognizes that it is imperative that individuals who request accommodations during the certification exam be handled appropriately and smoothly. We failed in this capacity for a number of candidates in 2023, and for that we sincerely apologize. Although we were assured on several occasions by PSI that requests for accommodations would be handled, in some cases they were not and, as noted by commenters during the town hall, this resulted in adverse experiences for those candidates impacted.

We have all personally experienced and understand the stress and anxiety that is invoked by this high-stakes exam. However, the additional burden for those who experienced delays in launching of the exam, were disrespected by employees at testing centers and were not given the appropriate accommodations is unacceptable. Please know that we are all frustrated by the lapse in quality services. In fact, this was a major factor in our decision to end our contract with PSI prematurely. We accept that those experiences have contributed to the erosion of trust that candidates, particularly those who have a disability, have in ABGC and the perception that they are being “cast aside." We are taking several steps to provide the level of service our candidates deserve. We hope that the results of the actions we are taking will help us regain your trust.

How is ABGC planning to improve the exam experience?

We also acknowledge that in addition to the mishandling of accommodations, many individuals also encountered technical issues and had subpar experiences at the test centers. As a result of subpar testing experiences over the last two exam windows, effective  December 31, 2023, we are terminating our contract with PSI. We have signed a contract with a new test vendor, Prometric. Prometric was selected through a competitive process initiated by the Board earlier this year. We are optimistic that this new partnership will mitigate the problems candidates experienced  in the previous test window. One major benefit of partnering with Prometric is that we will have front line access to the accommodation requests of candidates, and we will also be the point of contact for most administrative issues with the exam. This will decrease, or hopefully eliminate, confusion regarding who should address candidates’ concerns, streamline the processes of applying for the exam and approving and providing the necessary accommodations and allow us to carefully follow these and other requests. Further, Prometric’s customer service team is located “in-house," as opposed to being subcontracted as it was with PSI, which contributed to misinformation and failed communication. With these changes, we will continue to do all we can to ensure that requested accommodations are provided and that the exam experience for these candidates is not otherwise different than their counterparts. Moving forward, if you experience a problem with requesting an accommodation or having it provided, please contact ABGC.

How is exam cost determined?

The cost of the certification exam has been a continual point of contention in the genetic counseling field.  In order to administer a certification exam that 1) meets the standards required in a health profession field, 2) is accepted by licensure boards, 3) maintains recognition with the National Commission for Certifying Agencies (NCCA) and 4) allows candidates to access the exam regardless of their geographic location, ABGC must work with a testing vendor that is able to provide exam development, exam delivery, proctored and “live” remote testing, item banking and statistical evaluation. This is too heavy a lift for a Board comprised primarily of genetic counselors and an executive office to manage. Therefore, like the vast majority of other health professional certification organizations, we have to work with a testing partner, and this comes with a cost. Compared to other health professions, genetic counseling is a relatively small field and the cost of administering the exam is spread across the limited number of candidates, leading to a higher price of the exam per person.

That being said, the current Board of Directors is considering various options to help alleviate the cost of the exam, while also exploring options to increase access in other ways, such as offering on-demand testing in place of the semi-annual testing windows. Please stay tuned as we begin integrating some changes over the next several months.

Exam Resources

What resources are available for exam candidates?

The ABGC website contains a list of some of the most common references used to draft items for the certification exam. You can access the list here.

As mentioned on the website, this is not meant to be an exhaustive list of all the references that are utilized in exam development or of all the study resources that could help prepare you for the board exam. It is meant to inform candidates of some of the materials which are often used by genetic counselors to help guide their current practice. We will continue to maintain a list of relevant study resources. We are aware that resources are continually being updated. As such, we will improve our efforts to update these resources before each testing cycle going forward.

It is also worth noting that item writers may also utilize updated guidelines, recommendations and official practice statements published by official medical organizations when drafting questions for the exam; this is imperative to ensure that the exam reflects current and updated practice in the field. While it would not be possible to release an entire list of the statements/guidelines that may be relevant to the exam, we will update the current list of references to include some of the possible sources of practice guidelines that could be used in the exam.

Do you offer a practice exam?

ABGC currently offers a 100-item practice exam online for $55 which was most recently updated in 2021. As you may know, the exam content outline was revised most recently in 2023; these revisions were based on the results of the practice analysis survey that was sent to diplomates in 2022. Once the exam content outline was revised in 2023, we decided to implement it as soon as possible in the August 2023 testing cycle. As a result, the revised exam content outline was released before a revised practice exam could be developed. Waiting for a revised practice exam would have delayed the release of the updated exam, which did not align with the goal of offering candidates a version of the exam that best reflected scope of practice in the field. 

As we had recently announced, we will be changing our testing vendor from PSI to Prometric. Due to this change, we will be prioritizing the effort to offer the certification exam to candidates with the new vendor as early as March 2024. This will unfortunately mean that we will not be able to offer a new practice exam before that time. We are hopeful that we will be able to offer a new practice exam through Prometric in time for the August testing cycle which will match the updated exam content outline.

The description of the current practice exam on ABGC’s website and PSI’s website both indicate that it is a 100-item exam. We can look into the possibility of having a practice exam with 200 items going forward. However, we would like to ensure that the practice exam be accessible to as many people as possible and a larger practice exam would take considerably more effort to develop and test, which may affect the cost of the practice exam. 

August Exam Cycle 

What action was taken for August candidates who did not pass their exam?

All candidates who did not pass their exam were offered a free retest in one of the two upcoming exam cycles: Retest/test during a test window that will be open from November 27 – December 23, 2023. Testing will be done through PSI test centers or remote proctoring. Or, test during our next exam window which will be open from March 1 – March 31, 2024 with our new test vendor. Please note that because we are transitioning to a new test vendor, the test window will open one month later than usual.

Both test windows will have instant scoring. As such, all candidates will receive their scores immediately after taking the exam. Three separate emails were delivered to the following individuals to communicate the Board’s decision about the retest window on November 6:

• • Candidates who did not pass and have not submitted an application to retest. They were given instructions on how to proceed with each option listed above.
  • Candidates who did not pass and already paid for their exam in February. They were given instructions on how to proceed with each option above and information on when they can expect an exam refund.
  • Candidates scheduled to sit for the exam for the first time in February. They were informed that they can choose to test during the November/December test window or take their exam in March. Instructions on how to apply for the March window are forthcoming from the ABGC Headquarters Staff. These individuals have to pay for their exam.

A similar update was also provided to the program directors through GCEA on November 7. As stated above, ABGC has contracted with a new testing partner, Prometric, starting in 2024. Because of this transition, the winter exam window, which will be offered by Prometric, will start on March 1, 2024 instead of in February. Prometric offers testing through Prometric-owned test centers and through their remote proctoring system, ProProctor. Through Prometric, CGC candidates will now be able to search for preferred testing locations prior to submitting their CGC exam applications, view videos on what to expect on their testing day and review test center procedures as well as robust FAQ documents. 

ABGC understands that moving the exam window from February to March is not ideal for those who have already scheduled for the February exam. ABGC headquarters staff will work with those candidates to reschedule with Prometric; we will provide more information in mid-December. Our goal is to provide the exam experience that our candidates expect. Additional communications will be sent to all ABGC Diplomates along with updates to the ABGC website shortly.

ABGC views the November/December test window as an opportunity to retest given the unprecedented issues experienced by many during the August exam cycle. We created this test window given the difficulties in administering the August exam and because our next test window will open a month later than usual, as we transition to our new vendor. It is our understanding that in at least one state, genetic counselors who fail the exam once cannot practice within their full scope until they pass. The November–December test window gives them an opportunity to test much earlier. Our intent is not to force candidates to retake the exam before they are ready but to give them the opportunity to do so if it is the best decision for them given their individual circumstances. 

ABGC cannot comment on how licensure boards, employers and even ACGC will view this retest opportunity. The Board encourages candidates and programs to reach out to those other entities with specific questions. If a letter documenting the reason ABGC is providing this retest opportunity could be beneficial, please reach out to our team and we will be happy to provide such documentation.

Ideally, we would like eligible candidates to use their one free retake opportunity in either the November/December or March exam windows. If there are extenuating circumstances that necessitates a candidate waiting until the August 2024 exam window, they can reach out to ABGC directly to explain and the Board will consider these requests on a case-by-case basis.

Can a test score be invalidated?

We recognize that not passing the ABGC exam can have a profound downstream impact on licensure, job retention, promotion and/or salaries. As a result, the question of whether it is possible to invalidate a score in the event of a candidate's failure has been raised.

The Board has thoughtfully considered the numerous requests to invalidate the exam scores for those candidates who did not pass in August. While it was brought to our attention that at least one previous ABGC Board invalidated scores, given our current accreditation by the National Commission for Certifying Agencies (NCCA), there are only a few situations where NCCA would support invalidating exam scores, for example if the wrong form of an exam was launched. Additionally, we have been advised that in order to consider invalidating the scores of those candidates that did not pass the exam, we would need to consider invalidating all scores, including those who did pass the exam. As you may imagine, the Board does not feel that this is an option we can consider. We feel that a free retest opportunity is the very best option that we can provide and will not be invalidating the August exam scores.

ABGC Relationships

Why is a test vendor needed?

As mentioned in this FAQ, developing a high-stakes exam like our certification exam requires that we use best practice in exam development. This necessitates partnering with a test vendor that employs psychometricians, experts in test development and assessment. For more on this topic, see the National Commission for Certifying Agencies.

How do the different genetic counseling organizations work together?

Please refer to the ABGC Genetic Counseling Professional Organizations Brochure .

What is ABGC's relationship with the National Commission for Certifying Agencies (NCCA)?

ABGC’s CGC credential is accredited by the National Commission for Certifying Agencies (NCCA). NCCA’s Standards for the Accreditation of Certification Programs were the first standards developed by the credentialing industry for professional certification programs. The NCCA Standards were developed to help ensure the health, welfare, and safety of the public. They highlight the essential elements of a high-quality examination program. The 2021 NCCA Standards are implemented for the accreditation of certification programs.

The NCCA standards are consistent with The Standards for Educational and Psychological Testing  (AERA, APA, & NCME, 1999) and are applicable to all professions and industries. NCCA accredited programs certify individuals in a wide range of professions and occupations including nurses, automotive professionals, respiratory therapists, counselors, emergency technicians, crane operators and more. To date, NCCA has accredited more than 315 programs from more than 130 organizations.

Accreditation for professional or personnel certification programs provides impartial, third-party validation that your program has met recognized national and international credentialing industry standards for development, implementation, and maintenance of certification programs.

To maintain the standards of the CGC program accreditation, ABGC depends on the partnership with our testing vendor not only to deliver the exam to our candidates, but to provide the psychometric services to ensure that our exam meets all standards. The relation with our vendor is extremely important and one that the ABGC Board does not take lightly.

Communication and Accountability

What is the Board's position on recent issues?

First and foremost, we want to sincerely apologize for the recent lapse in communication regarding the issues that arose in the August exam cycle. We recognize that our response was not as prompt and clear as it should have been, leading to a regrettable erosion of trust. We want to express our sincere appreciation for those of you who brought specific concerns to our attention. We understand the frustration caused by the inconsistent email notifications, delayed posting of the new content outlines and the lack of information on our website. Additionally, we acknowledge the frustration with poor communication from our vendor and their customer service team. Your concerns, particularly regarding accountability and transparency, are valid and have not gone unnoticed. We take full responsibility for this and understand the impact it has had on our diplomates.

What are ABGC's immediate short-term plans and goals?

In response to these specific concerns, we are implementing immediate short-term measures to address communication gaps. We are taking the following concrete steps in the short term:

• We are committed to providing clearer communication about the development, delivery and scoring of our exams. This encompasses detailed information on the test development process, measures in place to identify and address biases and a comprehensive understanding of how the test is delivered and scored. We understand the importance of keeping our diplomates well-informed, and we are dedicated to ensuring that information is disseminated in a timely and accessible manner. This Q&A document serves as an initial step in our ongoing efforts to enhance communication about the exam.
• We are committed to providing our candidates with an exceptional exam experience, and to ensure this, we have made the decision to part ways with our current vendor, PSI. We have entered into a partnership with a new vendor, Prometric, known not only for their excellence in exam development and delivery practices but also their exceptional reputation in customer service. Our goal is to enhance your overall experience and instill confidence in the certification process we provide.
• We strongly believe in the principles of transparency and accessibility, recognizing the importance of providing information to everyone. To address concerns raised about the difficulty in finding essential documents on our website, we are committed to a comprehensive update of the website within the next six months. This overhaul will ensure that critical documents, including ABGC policies and procedures, exam development process documents, exam resources, recertification details and more, are readily available and easily accessible to all. Our aim is to create a user-friendly platform that empowers our community with the information they need for a clearer understanding of our processes and policies.
• Moreover, we want to address the accountability aspect raised in your questions. We take full responsibility for any shortcomings in our communication and sincerely apologize for the impact this has had on our diplomates. Your feedback has prompted a thorough review of our processes, and we are actively working to rectify these issues.

What are ABGC's long-term goals?

Looking ahead, ABGC has identified communication as a strategic pillar for the current year. Beyond the immediate measures outlined above, we are committed to implementing additional initiatives to foster transparency and bidirectional communication. This includes establishing channels for feedback and actively seeking input from our diplomates to shape our communication strategies. We recognize that open communication is crucial for building and maintaining trust, and we are committed to fostering an environment where diplomates feel heard, valued and informed. We want to assure you that the concerns raised regarding the impact on the livelihoods of professionals and the accuracy of the test in assessing competence are taken seriously. We acknowledge the potential disparities and are committed to addressing them through ongoing evaluations and improvements to our processes.

We recognize the critical importance of effective communication, and we are dedicated to learning from past shortcomings to implement lasting improvements. ABGC remains fully committed to ensuring that our diplomates are well-informed and confident in our commitment to transparency. We appreciate your candor and commitment to holding us accountable. ABGC is committed to learning from our mistakes and working collaboratively with our community to build a stronger and more transparent certification process. Thank you for your understanding, and we welcome continued feedback as we strive to improve and rebuild trust.