The Asian Pacific Society of Cardiology (APSC) comprises 23 cardiology societies in the Asia-Pacific region.1 Currently, cardiology subspecialty training is not uniform, and there is no standardised examination to assess the core cardiology knowledge of trainees.2 Instead, each member of the cardiac society of APSC certifies their own trainees independently. In response to the need for harmonisation of cardiology training in the region, the APSC approached the European Society of Cardiology (ESC) in 2020 to allow its members to take the European Examination in Core Cardiology (EECC), a broad, balanced and up-to-date test of core cardiology knowledge aligned with the 2020 ESC Core Curriculum for the Cardiologist.2,3 The EECC is currently being used by multiple cardiac societies in Europe, as well as an increasing number of non-European cardiac societies, to test evidence-based knowledge among its members, and facilitate progressive improvement and harmonisation of cardiology training and clinical practice. Similarly, the EECC has been used as the APSC Exit Examination for examinees from APSC member countries since 2020, with the same EECC questions administered to trainees from APSC member nations as their counterparts in the rest of the world. While none of the APSC member cardiac societies include the APSC Exit Examinations as a requirement for cardiology training, the cardiac societies are encouraged to nominate their trainees to take the examination to serve as an assessment tool to improve their respective training programmes.
This article reports the examination results of the APSC examinees over the first 4 years of implementation, and compares their results with examinees from the rest of the world (non-APSC). This analysis is intended to benchmark the core knowledge of cardiology trainees in the Asian-Pacific region, aid in the evaluation of the current quality of their training, and guide plans for improvement and harmonisation of cardiology education.
Methods
Study Setting and Design
The EECC is delivered using an online platform (CYIM) with remote proctoring (ProctorU) and is accessible anywhere in the world.4 It consists of 120 questions in standard English and is taken on a computer over 3 hours.2,3 Each question comprises a short clinical scenario, a single question and five possible options shown in alphabetical order, with a single best answer. Topics are taken from the ESC Core Curriculum for the Cardiologist, and since 2022, questions are divided equally between the following sections: imaging and valvular heart disease; rhythm disorders; coronary artery disease, acute cardiovascular care, prevention, rehabilitation and sports; and heart failure and cardiac patients in other settings.2,4 Until 2021, the blueprint of the examination used other category denominators, as these reflected the previous curriculum. Hence, until 2021, the sections were as follows: valvular and myocardial disease; ischaemic heart disease; rhythm disorders; adult congenital heart disease and non-invasive investigation; and general.
Questions are written and submitted online by a team of cardiologists from all participating countries/regions before being edited in small groups at physical or virtual meetings. Once questions have been selected for an examination, the standard-setting group estimates their difficulty using the modified Angoff method, and their collated scores inform the pass mark.5 The determination of the pass mark has been previously described.2 The performance of each question in the examination is analysed by an independent psychometrician. Any poorly performing questions are reviewed and may be excluded. The pass mark is then determined by the EECC board using the Hofstee method.6
Participating cardiac societies are informed about the performance of each of their candidates within 2 weeks of the examination, and candidates are given an online breakdown of their marks for each section of the EECC. Certificates are then issued by the national cardiac societies. Further details of the EECC examination development and delivery have been described previously.2,4
This is a retrospective analysis of the scores of examinees in the EECC from 2020 to 2023 using the scores of EECC non-APSC examinees of the corresponding years as a benchmark. APSC countries that participated in the EECC from 2020 to 2023 are shown in Supplementary Table 1.
Outcomes, Data Collection and Analysis
The de-identified examination results of APSC candidates were summarised into the following categories per country and region: total number of candidates, number of candidates who passed or failed, mean score (%) per section, mean total score (%) and pass rate (%). The pass rate was calculated as the number of candidates who passed divided by the total number of candidates who took the examination.
The participating countries/regions were further classified into developed versus developing based on whether or not they are considered high-income according to the United Nations World Economic Situation and Prospects 2022 Country Classification (Supplementary Table 2).7 The pass rates of developed and developing countries/regions were then compared to assess any effect of economic development.
However, the scores of participating countries from non-ESC non-APSC cardiac societies, such as the Cardiac Society of Australia and New Zealand (CSANZ) and the Gulf Heart Association, were aggregated under each society and not the examinee’s specific country of origin. The overall pass rate and the pass rate of developed and developing countries/regions were also reported for non-APSC examinees.
Results
A total of 287 examinees from the Asia-Pacific region and 2,492 non-APSC examinees took the EECC from 2020 to 2023 (Table 1). The number of examinees from APSC member countries, as well as non-APSC member countries, showed a steady annual increase.
The overall pass rate of APSC member countries was 72.8%, while the overall pass rate for non-APSC countries/regions was 83.9%. A gradual, but steadily increasing, trend in the APSC pass rate can be observed from 2020 to 2023 (from 60.53% in 2020 to 83.00% in 2023; Figure 1). The overall non-APSC pass rate started at 83.43% in 2020, but decreased to 77.74% in 2021, then increased in 2022 and 2023.
Figure 2 shows the annual per-section percent scores of 2020 and 2021 examinees, and Figure 3 shows those of 2022 and 2023 examinees. The scores of non-APSC examinees were numerically slightly higher than APSC examinees, except in adult congenital heart disease and non-invasive investigations in 2021.
When classified according to economic development, examinees from developed countries in APSC and the rest of the world had numerically higher pass rates than developing countries in 2020 and 2021. However, the gap between the pass rate of developed and developing countries was reversed in 2022 for non-APSC candidates and reversed in 2023 for APSC candidates (Figure 4).
Discussion
The EECC is a well-established test of core cardiology knowledge, and the results of this analysis showed that it has been adopted successfully across a broad range of developed and developing countries in the Asia-Pacific region as the APSC Exit Examination.
Our analysis shows that the total number of participants has increased substantially year-on-year from both APSC member countries and non-APSC member countries (Table 1). While this can be attributed largely to the increase in the number of examinees participating per country, there has also been an increase in the number of participating nations, especially from non-APSC countries/regions (Supplementary Table 1). Among APSC member countries, the increase in the number of participants per country is due to active efforts by the APSC to encourage its affiliate cardiac societies to register examinees for the examination. The APSC has also been actively conducting awareness-raising activities during its educational events and through its partnership with the Journal of the Asian Pacific Society of Cardiology. The present article is the third in a series describing the experience of APSC member countries since APSC partnered with EECC in 2020.
The overall performance of the examinees from APSC member countries has improved year-on-year since the establishment of the joint examination, and was not affected by the revision of the examination sections in 2022 to align with the ESC Core curriculum for the Cardiologist.8 It should also be noted that at the time of this analysis, the APSC had been participating for only 4 years, and unpublished data from the EECC show a trend of improving performance among the examinees of countries in the first few years after initiating participation in the EECC (Plummer CJ; email correspondence, January 2024). As internal factors, such as test preparedness, are the most commonly perceived determinants of exit examination performance, the trend to improved performance over time may be attributed to improved examinee preparation (e.g. improved familiarity of examinees with the examination scope and format, and experience-sharing between past and future examinees, leading to better preparation).9
Our analysis shows a small numerical difference between the pass rate of APSC and non-APSC examinees. The higher proportion of non-APSC examinees passing the EECC is hypothesised to be driven, at least in part, by the high proportion of candidates in this subgroup, primarily those from Europe, who require the EECC for successful completion of cardiology training, which may motivate examinees to prepare more intensively. This motivator is not present among APSC member countries, as the examination is not yet mandatory for the completion of cardiology training in these countries. Future analysis to confirm the impact of this variable on test scores is recommended.
A review of performance per examination section shows that ischaemic heart disease produced the lowest scores for both APSC and non-APSC examinees in 2020–2021. It is not possible to be certain whether this persisted in 2022 and 2023 because the section on coronary artery disease was then combined with acute cardiovascular care and prevention, rehabilitation, and sports. There are many possible reasons for lower performance in a section of an examination. While exploration of these possibilities is beyond the scope of this paper, the authors strongly encourage the EECC and the APSC and its affiliate cardiac societies to investigate this observation, given the high global burden of ischaemic heart disease.
While this analysis found that examinees from developed countries/regions had a numerically higher pass rate than those from developing countries/regions in 2020 and 2021, this was not consistent, and the opposite relationship was seen in 2022 non-APSC examinees and in 2023 within APSC examinees. Hence, this analysis does not provide any evidence that the economic development of the examinees’ country is a determinant of the pass rate. Instead, other factors, such as the mix of countries participating in each round of examination, and the increasing familiarity and improved preparedness of examinees, are more likely to be important. The impact of the COVID-19 pandemic on the performance of examinees is unclear, but it could also play a role in the lower pass rates in 2020 and 2021 in developing countries/regions. Studies of the conduct and results of the pandemic on medical exit examination delivery and performance are conflicting, with some reports showing a dip in examinee performance, while others noting minimal impact when using online platforms to test theoretical and practical questions.10–12
The authors highlight several limitations of this analysis. Comparisons between sections of the examination were limited by the change in the number and content of sections in 2022 to reflect changes in the ESC Core Curriculum for the Cardiologist.8 Statistical analyses were limited by the small numbers of participants in some countries/regions and categories, and the aggregation of data from some countries into multinational associations. The authors also acknowledge the presence of other factors that may affect the scores of examinees, such as differences in the use of English as a language of instruction, the length of preparation, and the examinees’ length of experience and previous academic background and performance. These factors may be investigated in future studies. Finally, to avoid the identification of individual candidates from smaller countries, their demographic data were not analysed, making it impossible to draw any conclusions about age or sex and performance.
Conclusion
While the pass rate of APSC examinees was lower than non-APSC examinees during the early rounds of EECC participation, the gap between pass rates diminished in the later years of implementation, consistent with observations of other countries/regions joining the EECC. We advocate the use of the EECC as the APSC Exit Examination across the Asia-Pacific region, as it is a well-validated, high-quality test of core cardiology knowledge. As the APSC Exit Examination, EECC may be considered as part of the assessment process for cardiology trainees, and may contribute to the harmonisation of training and benchmarking of educational standards across the world.
Clinical Perspective
- This retrospective analysis showed that the pass rate of Asian Pacific Society of Cardiology examinees was lower than non-Asian Pacific Society of Cardiology examinees during the early rounds of the European Examination in Core Cardiology, but that the gap has reduced over time.
- This suggests that the calibre of cardiologists participating in the European Examination in Core Cardiology is approaching those of the international community.
- The Asian Pacific Society of Cardiology may continue to use the European Examination in Core Cardiology as its exit examination as part of harmonisation of cardiology standards across the world.