Methods: Using data from previously developed surveys and expert panels, a web-delivered survey was distributed to 3,443 residency program directors. Results: Among 347 respondents, program directors reported that all advanced communication and some procedural tasks are important to assess. Discussion: Results from this study support that assessing procedural skills such as cardiac resuscitation, sterile technique, and phlebotomy would be amenable to assessment at the end of medical school, but most procedural and advanced communications skills would be amenable to assessment at the end of the first year of residency training or later. Conclusions: Gathering data from residency program directors provides support for developing new assessment tools in high-stakes licensing examinations. To access the supplementary material to this article please see Supplementary Files under Article Tools online.
Over the last decade, the Accreditation Council on Graduate Medical Education (ACGME) and American Osteopathic Association (AOA) competencies have been integrated into graduate medical education (1, 2).
Preparing medical students for residency training has been the focus of medical schools in the United States (11, 12). Numerous studies have identified educational gaps between graduating medical students and residents, with regard to clinical skills such as advanced communication and procedural skills (14–21). While many studies have attempted to identify the clinical skills required of trainees, most have been limited to a particular institution or discipline. Recommendations from NBOME's Clinical Skills Testing Advisory Committee (8 members) and strategic planning committee (16 members) were incorporated into the survey. After receiving input from these expert panels, items were reviewed, and further enhancements were made by NBOME's Research Advisory Committee (12 members) composed of experts in assessment, education, and research. The web-delivered survey was distributed to 3,443 ACGME and AOA-accredited residency program directors with valid email addresses contained in NBOME's residency program director database.
Institutional Review Board approval was granted by the Center for the Advancement of Healthcare Education and Delivery (C-AHEAD) to collect, analyze and report these data for this study.
A total of 347 program directors completed the survey, representing a response rate of 10.1%. Figure 1 presents program directors’ opinions about the importance of assessing 28 procedural skills.
With regard to the importance of assessing procedures, agreement varied among program directors of different specialties.
The majority of program directors reported residency program faculty to be the most appropriate for assessing procedural skills Table 2. As for the most appropriate time to assess the procedural skills (Table 3), program directors reported that assessment of most procedures should be completed at the end of the first year of residency or later.
Figure 3 displays program directors’ opinions about the importance of assessing 18 advanced communication skills. Importance of advanced communication skills assessment by program directors’ responses. Regarding the 18 communication skills, program directors overwhelmingly reported that integrative evaluations using both summative and formative assessment should be utilized (Fig.
Type of assessment for advanced communication skills by program directors’ responses. Program directors reported that all advanced communication tasks and some procedural tasks are important to assess during medical training. For both advanced communication and procedural skills, program directors reported that assessments should include a combination of formative and summative evaluation.


Program directors reported that most clinical skills should be assessed at the end of the first year of residency (or later) and not before graduation from medical school. Therefore, identifying which skills should be assessed at each decision point is critical for informing examination development, and gathering input from residency program directors is important. For each of the 28 procedural and 18 advanced communication skills, program directors were asked which clinical skills should be assessed, by whom, when, and how. As a consequence, greater attention has been given to defining competence and identifying specific skills required of residents (3).
As part of the Medical School Objectives Project (MSOP), the Association of American Medical Colleges (AAMC) has identified specific skills required of medical students prior to graduation (11), such as patient care, communication and procedural skills.
For instance, Wagner and Lypson described objective standardized clinical examinations (OSCEs) administered to residents at the start of their training (14). Identifying these clinical skills is particularly important for developing and enhancing assessments for licensing examinations.
Members from both committees are considered experts in medical education and assessment; members include representatives from undergraduate medical education (deans, associate deans, faculty), graduate medical education (ACGME and AOA-accredited residency program directors, directors of medical education), clinicians, medical educators, and psychometricians. The final instrument (found in Supplemental content) addressed 28 procedural skills and 18 advanced communication skills, was pretested by physician staff, and distributed using Survey Monkey.
Program directors were randomly divided into two groups; each group received one of two versions of the survey (one with procedural skills presented first and another with advanced communication skills presented first). Program directors from a wide range of disciplines responded to the survey, and specialty distributions were reflective of national data (Table 1) (36, 37). ICC is an index representing proportion of the total variance explained by group effects with higher ICC values indicating larger group variation (lower agreement between specialty groups).
2 are program directors’ opinions regarding how the 28 procedural skills should be assessed. Only for the phlebotomy skill, medical school faculty were regarded more appropriate than residency program faculty.
Although their responses were consistent across disciplines when considering communication tasks, there was variability among groups when asked about procedures. This was particularly true for advanced communication tasks, demonstrated by a small number of program directors advocating for exclusive summative assessment. Exceptions to this include demonstration of respectfulness, sterile technique, cardiac resuscitation, and phlebotomy; these were considered important to assess before the start of residency.
Descriptive statistics were collected, and Intraclass Correlations (ICC) were conducted to determine consistency across different specialties. As an example, the ACGME and American Board of Pediatrics (ABP) have established the Pediatric Milestones Working Group to define specific competencies required of residents at different levels of training (4, 5). Specific examples include ability to communicate effectively with patients, families and colleagues and ability to perform routine procedures such as venipuncture, lumbar puncture, laceration repair, and thoracentesis (11). New resident performance in communication assessments were consistently high, but patient care scores varied widely with particularly low scores in the areas of hand hygiene and aseptic technique. Two high-stakes clinical skills examinations are used in the United States to assess clinical skills performance of medical students: USMLE Step 2 Clinical Skills (USMLE Step 2-CS) and COMLEX-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) (7, 8). Intraclass Correlations (ICC), which describe the degree of group agreement, were calculated to examine the disparity in responses of program directors of different specialties. For instance, 45 surgery and surgical subspecialty program directors were included in the sample (13% of the sample), compared to 946 in the national sample (19.9% of all residencies).


The skills are sorted in the descending order of importance rated by all program directors (n=293).
Program directors overwhelmingly reported that each of the procedures should be assessed in a formative fashion, followed by a combination of both formative and summative assessment. Exclusive summative assessment was not considered a suitable format by most program directors. High levels of agreement between program directors of different specialties were seen for nasogastric tube placement, obtaining a blood culture, cardiac resuscitation, phlebotomy, sterile technique and injections. Compared to the other procedural skills, cardiac resuscitation, phlebotomy, sterile technique, injection and intravenous placement were considered amenable to summative assessment. Not only are program directors attempting to clarify which skills should be assessed at which level of training, but national licensing boards are also working toward identifying clear assessment objectives (6).
In a recent study by the National Board of Medical Examiners (NBME), residents reported performing a number of procedures and communication tasks during their first few months of training (13).
In another study of new residents, Lypson reported lowest OSCE scores in the areas of informed consent and identification of critical values (15).
However, neither clinical skills examination currently assesses advanced communication (triadic encounters, death and dying, etc), procedural, or clinical skills which may be unique to a particular specialty.
The majority of the program directors reported residency program faculty to be the most appropriate to assess advanced communication skills (Table 4). Strong agreement is likely explained by the fact that these procedures are common to all physicians, not just those of a particular discipline.
The authors conclude that perhaps these clinical skills should be taught and assessed prior to completion of medical school training. Both studies not only identified variability among entering residents, but also exposed gaps between residency program faculty expectations and actual performance reflective of undergraduate medical education training. The ICC coefficients examining group agreement between program directors of different specialties ranged from near 0 to 0.13. Identifying consistency among program directors of different disciplines is important, given the recent growth of specialization in graduate medical education (38–40).
Therefore, identifying which skills should be assessed at each of these decision points becomes critical.
Identifying disparate expectations of medical students and residents has challenged medical educators to identify which specific clinical skills should be taught and assessed.
In particular, the objective is to survey residency program directors and identify which clinical skills are important to assess, by whom, when, and how (formative versus summative). Some procedural skills, such as sterile technique and cardiac resuscitation (BLS), displayed both low group disparity and high importance ratings. However, of these procedures with high levels of agreement, only cardiac resuscitation, sterile technique and injection were considered important to assess.
Previous investigators report conflicting opinions regarding which clinical skills should be taught or assessed during medical school (17–24) and residency training (18, 25, 26). Among those who completed the survey, 293 respondents completed the section on procedural skills and 284 completed the section on advanced communication skills.



Making video using photoshop
Miracles from heaven review


Comments

  1. 27.11.2015 at 23:55:22


    You are naturally a manifesting i locate it exciting that individuals.

    Author: lakidon
  2. 27.11.2015 at 21:40:20


    Nutrition college in the country, defines a well being.

    Author: NaRKo_BiZnES
  3. 27.11.2015 at 10:42:50


    Consuming, physical activity, sleep concerns, monetary management.

    Author: 860423904