Guide to Evaluation Forms for Resident Clinical Performance Assessment
Susan J. Lieff M.D., M.Ed., FRCPC Evaluation forms can be used for either summative or formative
purposes. The goal of formative
assessments is to give the trainee specific feedback that facilitates
reflection on performance, identification
of areas in need of development
and decision-making on strategies to improve. They are used strictly
to facilitate learning and are
not tied to passing or failing.
On the other hand, summative evaluations are used for documenting
successful completion or passing such as
completion of a residency core
training requirement, graduation, or certification. The forms provided
here can be used for either
formative or summative purposes.
It is important to separate the feedback/teaching function from
the assessment/decision-making function.
Encourage immediate feedback for
formative purposes. Residents should never be asked to participate
in any summative (formal evaluation)
process without ample notification.
Members
of the American Association for
Geriatric Psychiatry Resident Evaluation
Task Force were asked to create resident performance
assessment form templates that
incorporate the Accreditation Council
for Graduate Medical Education (ACGME)
core competency constructs and
categories. It became apparent
to task force members, that the
creation of a single, ideal form, applicable
to the clinical supervisory experience,
and useful for both formative and
summative purposes, that also meets
the requirements of ACGME was unrealistic. Different competency
constructs require different methods.
Additionally, the ACGME-required
midterm and final year summative evaluations are not consistent
with providing the immediate and
detailed formative feedback necessary
to facilitate learning in the clinical supervisory context.
Hence, a number of global assessment
forms were created that can be
used as a starting point for this new evaluative process. It is
anticipated
that as programs become increasingly
familiar with the core competencies,
ACGME will require
more objective methods than a global
assessment alone, as it is the
least reliable and valid method
of assessment. Nonetheless, currently, there is demand for
such forms. To enhance the validity
and reliability of the information
gathered with these forms a number of augmentation strategies are
offered. 360-degree evaluations
and patient surveys have also been
included to cover some of the newer
competency categories such as interpersonal
and communication skills and professionalism.
At
the start
At the outset of the rotation,
trainees need to be informed of
the criteria, methods and frequency
of evaluations. They also need
to know who will be doing evaluations
for both formative and summative
purposes. They should be provided with copies of all forms, and
the minimum required to achieve
competence or pass the rotation
should be
clearly articulated. When Likeart
scales are used it is important
to define the cutoff for achievement
of competence. For example,
is
a 3 average for their level of
training or is a 5 required to
pass?
Evaluation Templates Included:
- The Geriatric Fellow Competency
Assessment Form
- Interview Evaluation Form
- Evaluation of Geriatric Psychiatry
Fellow Performance Form
- 360-degree Evaluation Form
- Patient and Family Member’s Evaluation of Geriatric Psychiatry
Fellow
The Geriatric Fellow Competency Assessment Form
The Geriatric Psychiatry
Fellow Competency Assessment
Form uses a type of global
assessment with a bottom up approach,
looking at the clinical and
teaching setting and integrating
the core competencies into that
framework. It does not comprehensively
cover all aspects of the
core competencies but certainly
most. It should be supplemented
by documenting "mini-evaluations" that
have taken place over the
course of a rotation. "Mini-evaluations" such
as chart stimulated recalls,
oral exams, record reviews, etc. provide part of the "evidence" for
how a trainee’s scores
were determined. These structured
approaches to the evaluation of clinical performance can be documented
at the time of performance
and measured relative to
a standard.
Interview Evaluation Form
The Geriatric Psychiatry
Fellow Competency Assessment
Form assessment does not
provide the detailed feedback
necessary for the evaluation of interviewing
skills. Since interviewing
skills often require their
own assessment template a
sample of an interviewing
skills evaluation is provided for consideration and adaptation.
Evaluation
of Geriatric Psychiatry Fellow Performance Form
The Evaluation of Geriatric
Psychiatry Fellow Performance
Form, another type of global
assessment, is a top down
approach that has core competencies
spelled out with anchor points.
This evaluation should also
be supplemented with multiple
data points such as observed
interviews, record reviews, chart stimulated
recall, etc. Faculty may
prefer to use one scale for
the quarterly and another for
the end of year or, alternatively,
one for formative and the
other for summative assessments.
360-Degree Evaluation
To provide reliability and
validity, the 360-degree
evaluations need to be completed
by a minimum of 7 to 10 allied
health professionals, peers,
or colleagues. An even greater
number of evaluations must
be completed if patients
and faculty members are the
reviewers. It is recommended that a structured process
that does not directly involve
the trainee should be used
for distribution and collection
of these surveys.
Patient
and Family Member’s Evaluation of Geriatric Psychiatry Fellow
The form included here can
be distributed to both patients
and family members and used
in conjunction with the 360-degree
evaluation.
Enhance Reliability
and Validity
- Use multiple observers and employ
multiple observations
in different clinical situations to enhance reproducibility. Clinical
competence
is highly case and situation
specific and is therefore
difficult to generalize.
- Implement a strategy for systematic
sampling of cases, situations,
diagnoses and tasks and supplement this with observation
of performance in standardized clinical cases
to provide control over
the case mix.
- Minimize delays between observation
and documentation, avoiding
the typical midterm and endpoint evaluations by encouraging
raters to make performance notes at the time of observation
using notebooks, instant
event report cards, oral exams, chart stimulated recall,
etc.
The templates
included here can serve as a starting
point to assist program
directors and faculty in the development of their own competency evaluation
forms
for trainees. As the forms
are used, consider two
questions for future revisions. Are the items as a group truly representative
of
and sufficient to capture the
constructs within the categories
of core competencies? Do these forms accurately and reproducibly
measure resident performance within
the core domains? References
- ACGME Toolbox of Assessment
Methods. ACGME Outcome
project. http://www.acgme.org/outcome/assess/toolbox.aspacgme
- Williams RG. Klamen DA.
McGaghie WC. Cognitive,
Social and Environmental Sources
of Bias in Clinical Performance
Ratings. Teaching & Learning in Medicine 2003. 15(4):270-92
- Yudkowsky R. Lieff SJ.
Master Educator Workshop
I: Assessment. Association
for Academic Psychiatry
Annual Meeting. 0ct 3, 2003. Philadelphia,
Pennsylvania
|
