1. science
2. medicine
3. oncology

VOLUME
28
䡠
NUMBER
22
䡠
AUGUST
1
2010
JOURNAL OF CLINICAL ONCOLOGY
S P E C I A L
A R T I C L E
How to Organize a Fellowship Program: Lessons Learned
and How to Include Accreditation Council for Graduate
Medical Education Competencies in the Curriculum
Frances A. Collichio, Michael P. Kosty, Timothy J. Moynihan, Thomas H. Davis, and James A. Stewart
From the University of North Carolina
Chapel Hill, Chapel Hill, NC; Mayo
Clinic, Rochester, MI; Scripps Clinic, La
Jolla, CA; Dartmouth Hitchcock Medical
Center, Lebanon, NH; and Baystate
Medical Center, Springfield, MA.
Submitted January 29, 2010; accepted
May 4, 2010; published online ahead of
print at www.jco.org on July 6, 2010.
Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this
article.
Corresponding author: Frances Collichio,
MD, Division of Hematology Oncology,
University of North Carolina, 170
Manning Dr, Box 7305, Chapel Hill, NC
27599; e-mail: [email protected].
© 2010 by American Society of Clinical
Oncology
0732-183X/10/2822-3659/$20.00
DOI: 10.1200/JCO.2010.28.1964
INTRODUCTION
Over the past 15 years, the complexity in fellowship
training in hematology/oncology has increased.
This is because of the vast growth of knowledge in
these fields and a change in the structure and accreditation requirements of graduate medical education.
The latter was put into place by the Accreditation
Council for Graduate Medical Education (ACGME)
in the mid 1990s.1 The ACGME responded to a
sense that it was difficult to determine whether physicians were properly trained. Medical students, residents, and fellows work in complex environments
and, unlike in formal classrooms in high school and
college where teaching and testing are uniform, physicians in training can be exposed to a range of
unique experiences, and assessments of progress can
be arbitrary. The ACGME review called into question the entire process and prompted a structure
that is known as the “Six Competencies of Graduate
Medical Education.” Now, fellowship programs in
hematology/oncology must expose their trainees to
the knowledge in the field while also complying with
assessments of these six competencies and must
show by a variety of measures that their trainees are
achieving accepted standards. This article is intended to show programs, program administrators,
trainees, and the oncology community in general,
steps that can lead to a successful hematology/
oncology training program with the ultimate goal
of having the finest trained specialists in the field
of hematology and oncology.
STEPS TO RUNNING A GRADUATE MEDICAL
EDUCATION PROGRAM
Step 1: Program Structure
Each program must determine the overall
structure of its fellowship. It is important to have a
vision of how the fellowship appears to candidates,
reviewers, and faculty. What is the flow of rotations
and are they appropriately graded in skill level, challenge, and responsibility as the fellow progresses
through the program? Is there a good balance of
time between clinical experience with time spent in
journal clubs, lectures, and other educational activities? Are all activities assessed for educational value?
Oversight of all hematology/oncology fellowships is provided locally by the home institution’s
Department of Medicine education office and GME
committee and nationally by the ACGME through
its Residency Review Committees (RRCs). The
ACGME provides oversight of the program with
evaluation and accreditation of all residencies.2 The
American Board of Internal Medicine (ABIM) is an
independent evaluation organization that certifies
individual internists and internal medicine subspecialists.3 Both the ACGME and ABIM have useful
Web sites that are important resources for fellowship directors and fellows. The ACGME site provides detailed descriptions of minimum program
requirements and offers a variety of tools to help in
program organization and improvement. The
ABIM site has a blueprint of the content of the examinations for the hematology or oncology certifying board examinations. It is important for program
directors and administrators to become familiar
with these Web sites because information changes.4
The ACGME describes the minimum elements
needed for a successful program and specifies details
of the time needed for clinical and overall training
and the required procedures that should demonstrate competence in such procedures as bone marrow biopsies and intrathecal administration of
chemotherapy. To be eligible for board certification
in either hematology or oncology, the trainee must
complete 2 years in an accredited program of which
at least 12 months must be clinical training. By July
2011, 50% of this clinical experience will need to be
in the outpatient setting. To be eligible for board
certification in both hematology and oncology,
there must be at least 18 months of clinical work
(50% in the outpatient setting), and 6 months will be
devoted to non-neoplastic hematology (NNH). A
trainee is required to have continuity clinic experience and a scholarly project that demonstrates
systems-based practice and practice-based learning
and improvement.
With these guidelines in mind, a program can
become accredited by the ACGME to offer training
© 2010 by American Society of Clinical Oncology
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
3659
Collichio et al
in hematology, oncology, or both. The program can choose to mix
NNH rotations with oncology rotations, or it may choose to separate
them. The ACGME does not specify how much time a program
should devote to research or scholarly activities. The ACGME recognizes that some programs will make research a large part of their
identity whereas other programs will not. A program could choose to
devote the 18 months that remain after accounting for the required
clinical activity to research or to a scholarly activity, or the program
could choose to commit a smaller portion of this time. The research
and scholarly options can range from small projects, to clinical trials,
to laboratory research, to any combination thereof.
The ABIM and ACGME also allow for flexibility. For example, a
6-year research track exists for residents who determine early on that
they wish to have an academic, research-based career. The trainee can
enter an accredited fellowship program after the second year of their
internal medicine residency.2
Step 2: Funding
Funding hematology and oncology fellowship programs can be
complicated. Fellows earn a salary and benefits for their postgraduate
year of training that meet the standard, which averages about $75,000
per fellow per year.5 Fellowship programs must also provide for administrative support. The ACGME requires that a portion of the
program director’s time be funded. With increasing demands on
program directors related to fellowship administration and program
development, it is recommended that program directors have a serious discussion with their division chief regarding just how much time
is allocated for fellowship activities. Some programs receive support
from the parent institution, and some use divisional or departmental
funding mechanisms. Most fellowship programs have a fellowship
coordinator, and many have an associate program director, depending on the size of the program. Adequate dedicated time and effort are
needed for those working on fellowship activities. Programs must also
account for space, supplies, and other support needed for educational
programs. Considering all of this, the precise dollar amount needed is
difficult to estimate but it could be as high as $100,000 per fellow
per year.
In the 2006-2007 academic year, the primary source of funding
for private hospitals was patient revenue.5 Funding sources include
faculty practice plans, patient care revenue, endowments, the Department of Veterans Affairs, the Department of Defense, Medicaid, and
Medicare. The money for trainee salaries usually goes to the parent
institution with distribution from the GME office rather than to the
individual divisions and departments.6 As a consequence, most fellows are employees of the hospital and not the medical school in
most fellowships.
Another source of funding is from research grants. Program
directors must be careful in program use of research funds to ensure
the fellow is in fact doing research during the time the fellow is funded
by research support. The National Institutes of Health (NIH) has
National Research Service Award (NRSA) Institutional Research
Training Granings (T32) that require protected time away from clinical responsibility, so careful attention to scheduling is important for
these trainees.7 The NIH has Institutional Research and Academic
Career Development Award (K12) grants for more experienced research fellows. The mixture of NIH research dollars and clinical training can be complex; thus, the program director should work closely
3660
© 2010 by American Society of Clinical Oncology
with the GME department and administrators who are knowledgeable
about fellowship and research funding.
Once these sources of money have been identified by the program, the program must decide how clinical work will be balanced
with nonfunded educational endeavors. In the United States, there has
been a shift toward having residents and subspecialty residents (fellows) spend more time in educational activities and less time in clinical
service. Programs must have didactic conferences, examples of which
include journal clubs, board reviews, clinical and basic science lectures, and in-training examinations (ITEs) as well as having clinical
and research mentoring programs. All of these activities take time
away from clinical work, but they are critical to promoting highquality training and adherence to ACGME standards.
Clinical work in the hospitals and clinics requires direct supervision of fellows. They are expected to develop expertise and clinical
efficiency as they progress through the program, thus requiring differing levels of supervision and faculty time as they mature. Supervising
staff must be readily available to the trainees and not have an excessive
number of patients or other duties that interfere with their ability to
supervise the care of the fellows’ patients. This usually requires the
program to free up the faculty to be completely involved. Training
programs must account for faculty supervision when they set up their
services. The overall amount of clinical time that residents and fellows
can provide has decreased. The ACGME work hour restrictions have
led to more costs related to clinical care. Many programs have increased the numbers of hospitalists and midlevel providers to offset
the reduction in trainee clinical time resulting from work hour restrictions.8 Moreover, to be fair and compliant with the mission of the NIH
training grants, fellows who earn a spot in these programs need to have
protected time away from clinical responsibility, sometimes as much
as 80%. In the past, these programs were used to fund training, but
now they are more restrictive and also quite competitive.7 All of this
can lead to outstanding training but limits the amount of clinical
service (and indirectly, clinical revenue) the trainees can provide.
Step 3: Program Size
The number of trainees in a given program is a complex issue
determined by multiple factors. The minimum number of trainees
specified by the ACGME is one fellow per year of program length.
Most importantly, the institution must have faculty educators who are
willing and able to share clinical expertise. In addition to the program
director, the ACGME requires key clinical faculty who dedicate, on
average, 10 hours per week throughout the year to the training program. For programs with more than five fellows, a ratio of key clinical
faculty to fellows of at least 1:1.5 must be maintained. The ACGME
does not specify the number of clinical faculty required when there are
fewer than five fellows. Second, the institution should provide adequate patient volume and mix for robust training. Third, adequate
funding must exist (see Step 2). Finally, the ACGME has to approve
the number of trainees in any given program on the basis of information it receives from the program administration and the ability of that
program to adequately supervise and educate those trainees.
Sometimes the number of trainees can change. A program may
be granted 15 trainees in total but, because of a personal event for a
trainee, the ACGME may grant a temporary change in the number. If,
for example, a trainee leaves for maternity reasons and then returns to
complete the training several months later, the program may have
more than the 15 trainees when the trainee returns from leave. A
JOURNAL OF CLINICAL ONCOLOGY
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
Organizing a Fellowship Program and ACGME Competencies
Table 1. Example of How Rotations Could Be Counted Toward Oncology
Board and Hematology Board Eligibility
Rotation
Type
Hematology (NNH)
Board
Required
Consult rotation
Outpatient hematology
Hematopathology
Elective
Coagulation laboratory
Private practice
Transfusion medicine
Oncology Board
Inpatient oncology
Leukemia service
Bone marrow transplantation
Outpatient oncology
Genetics
Gynecologic oncology
Radiation oncology
Palliative care
Private practice
Abbreviation: NNH, non-neoplastic hematology.
catastrophe that affects an entire program, such as Hurricane Katrina, may require trainees to transfer to other programs. In these
economically turbulent times, it is possible that hospitals with
ACGME-approved programs may close, which would require a transfer of trainees to other sites. The ACGME has policies to facilitate
this process.9
NAME
Step 4: Clinical Experiences
Combined programs can determine which of their rotations will
meet the eligibility requirements for oncology and which will meet the
eligibility requirements for hematology. For the combined certification in hematology and oncology, 18 clinical months are required, 6
months of which must be in NNH. Guidelines for making these
distinctions can come from the American Society of Clinical Oncology
(ASCO) core curriculum.10,11 The topics that are listed for certification examination by the ABIM can also help programs decide how to
divide clinical experience time.3 Table 1 provides an example of rotations that could be categorized as non-NNH and those that could be
categorized as oncology. Each year, the ABIM requires that the program enter the number of months that the trainee spent in NNH,
hematologic malignancies, clinical oncology, research, and other. For
a trainee who is seeking eligibility for both boards, keeping an accurate
count of this information can be particularly important. Figure 1 is a
form that shows how a program could keep track of its rotations; it can
be modified by each program.
A continuity clinic experience throughout the fellowship
training is required. General oncology experiences might be available at Department of Veterans Affairs hospitals, private offices,
regional clinics, and in other urban venues. In many university-based
cancer programs, there are disease-oriented clinics with significant
Date
Counting Rotations Toward ABIM Boards
For Heme Board Certification
Nonneoplastic Hematology (NNH) Totals
Coagulation lab
Hematopathology
Transfusion medicine
Consult service
Heme outpatient
Total for NNH Equals 6
Fellowship
Year 1
Fellowship
Year 2
Fellowship
Year 3
Totals
Equals 6
For Oncology Board Certification
Heme malignancies
Heme malignancy inpatient
Bone marrow transplant
Fig 1. A form for keeping track of rotations
that count toward American Board of Internal Medicine (ABIM) board certification.
Heme, hematology; GYN, gynecology.
Total
Clinical Oncology
Inpatient
Outpatient oncology
Genetics
GYN oncology
Radiation oncology
Palliative care
Private practice
Total
Total Clinical for Oncology Boards
Equals
12
Total Clinical for NNH and Oncology
Total Research or Other Years 2, 3
Total Months Per Year
www.jco.org
© 2010 by American Society of Clinical Oncology
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
3661
Collichio et al
faculty subspecialization. This could mean that a trainee might get an
intense experience in one disease site, such as breast cancer, if the
trainee stayed with that clinic throughout the training program, but
miss a continuity experience in populations with other diseases. To
avoid a limited clinical experience, a fellow can switch from one
continuity clinic to another, but each clinic should be of least 6 months
duration to achieve a meaningful longitudinal experience. The
ACGME requires that the continuity clinic have four to eight patients
during each half-day session and that there should be at least 25% of
patients of each sex. Each program director and the program administration should keep track of the continuity clinic requirements, because the ACGME guidelines may change.4 For each continuity
experience, the fellow should have a population of patients in which
the fellow is viewed as the primary oncologist and in which there is
sufficient time to experience important natural history elements of the
disease type.
Step 5: A Competency-Based Training Manual
A training manual or handbook can help the program be successful. This handbook could be a source document that the ACGME uses
to review the overall structure of the program, and it can be updated
on an annual basis as features of the program change. The handbook
could read like a book, with a table of contents, followed by key
telephone numbers, people to contact, and Web sites. An overview
that states the goals of the program and the available tracks such as
clinical, clinical with clinical research, clinical with laboratory research, or other tracks is helpful. Sections should be included on
mandatory and elective clinical rotations, the program’s scholarly
structure and research opportunities, how evaluation of fellows and
key faculty takes place, and how residency work hours are accounted
for. Other key documents such as due process guidelines, program
benefits, policies on personal health, and on-call responsibilities could
also be included. Wherever possible, the competencies as drivers of
programmatic goals and fellow evaluations should be emphasized.
The ACGME checks to see that rotations are described within the
framework of the six competencies and that the goals and objectives
for each rotation are appropriate for each year of training and include
progressive levels of responsibility. ASCO has guidelines for core curriculum, and ASCO is considering developing competency-based curricula for specific cancers and cancers in general.10,11 It is unlikely that
this project could meet the needs of all of the programs, and it could be
too specific for some. It is essential that programs adapt some of the
preexisting tools and develop their own multisource assessments.4
The handbook could provide an overview that is tailored to the program. Table 2 lists each of the six competencies and shows what
content could be considered in each one with respect to hematology
and oncology. Table 2 also shows how the six competencies can be
applied using head and neck cancer as an example.
Step 6: Interview Process
Fellowship programs in hematology and oncology joined the
National Residency Matching Program (NRMP) in 2007. The NRMP
uses the Electronic Residency Applications System (ERAS) to distribute the applicants’ documents to programs and to collect the programs’ ranking lists. To participate, 75% of the programs in the
specialty and 75% of the available spots in each program must be listed
in the match. These rules exist to keep the system fair and open to the
applicants. There can be special circumstances that allow programs to
3662
© 2010 by American Society of Clinical Oncology
allocate some of their slots outside the match. There may be, for
example, a couple that wishes to match into one program. Or there
may be an educational reason to take an applicant outside of the match
so as to accommodate a trainee who wishes to pursue preparation in
both geriatrics and oncology at the same program. Each program
should decide how many applicants to interview. An Internet chat on
ASCO’s Listserve among program directors suggested that a good
ratio is to interview eight applicants for every available position. Each
program, however, has to learn by experience what it takes to fill its
slots, so the ratio may vary by program. Competitive programs can
have more than 300 applicants and it can be time-consuming to
review the information on ERAS. Funding restrictions at some institutions cannot accept international applicants with certain kinds of
visas. ERAS has a visa filter as well as other helpful filters to sort
applications by United States Medical Licensing Examination
(USMLE) scores, medical schools, and other details that may be important to the individual program. Having more than one person
screen the applicants, such as the program director and the associate
program director(s), can streamline the process, and the screeners can
look for a list of optimal candidates to invite for an interview.
The program should decide what the interview process should be
like. The program, for example, may wish to have a more personalized
setting by interviewing one or two applicants at a time. Other programs offer interviews only on a few days and therefore bring in several
applicants at a time. No matter what the approach, a written or oral
presentation by the program director or by the program director and
several faculty members is useful. It is also important to train interviewers about the types of questions that are out of bounds according
to federal equal opportunity employment law. Each institution’s human resource department can provide guidance.
Finally, a system to determine how the applicants are ranked is
helpful. If each applicant is interviewed by two or more people and
given a score, the average score can be used as an early ranking list. Of
course, over time the program director will recognize who among the
interviewers are the hard graders and who are the easy graders and
adjust the rankings accordingly. After the interview process is complete, a match meeting is useful to review each resident interviewed
and devise a rank list that the program director will submit to the
NRMP. There should be documentation that this meeting took place
to have a record showing that the selection process is fair and reasonable. This is an important defense against the rare employment discrimination lawsuit. The program director must keep in mind the
ERAS deadlines for submission of the number of spots available and
the final ranking list.
Step 7: Fellowship Work Hours
The ACGME has a zero-tolerance policy toward work hour violations. Studies have shown that sleep deprivation is correlated with
poor performance and can lead to poor quality of care including
dangerous medical errors.12,13 Work hour mandates began in 2004.
They apply to the clinical work that resident and subspecialty residents
(fellows) do. At this time, they include a maximum of 80 hours per
week averaged over a 4-week period, no more than 24 consecutive
hours with 6 additional hours for transfer of care, continuity clinic,
didactic lessons, and maintenance of continuity of care, a 10-hour
break between shifts, and an average of 1 day off in 7 days, averaged
over 4 weeks.4 Web-based tools have been devised and are being used
to help the program keep track of work hours.
JOURNAL OF CLINICAL ONCOLOGY
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
Organizing a Fellowship Program and ACGME Competencies
Table 2. Guide for the Six ACGME Competencies in Hematology and Oncology Using Head and Neck Cancer as an Example
Competencies
Content
Possible Evaluation Methods
Patient care—Residentsⴱ
must be able to
provide patient care
that is compassionate,
appropriate, and
effective for the
treatment of health
problems and the
promotion of health.
Records comprehensive
history and performs
physical examinations
with attention to issues
related to treatment
toxicity. Plans
interdisciplinary care,
conducts family
meetings, evaluates
patient’s health literacy
and understanding of
disease, treatment
options and prognosis,
and evaluation and
treatment of symptoms.
Direct observation of a mini
clinical examination
Standardized patient
Structured clinical examination
Simulated procedure
Medical
knowledge—Residents
must demonstrate
knowledge about
established and
evolving biomedical,
clinical, and cognate
(eg, epidemiologic and
social-behavioral)
sciences and the
application of this
knowledge to patient
care.
Demonstrates knowledge of
evidence-based, diseasespecific information
regarding diagnosis,
staging, treatment
options and prognosis,
and symptom
management.
Demonstrates knowledge
of available clinical trials
and overall knowledge of
clinical trials, including
protection of human
subjects.
Board examinations
In-training examinations
Test modules
Training in ethical conduct of
research
Practice-based learning and Discusses and debriefs
improvement—Residents outcomes of treatment
must be able to
for each patient.
investigate and
Discusses chemotherapyevaluate their patient
related toxicity and
care practices, appraise
untoward effects.
and assimilate
Debriefs after each
scientific evidence, and
patient death with
improve their patient
interdisciplinary team.
care practices.
QOPI
Other reflection tools
Content for Head and Neck
Cancer
Records comprehensive
history and performs
physical examination for all
new consults and provides
disease-appropriate history
and focused examination for
patients presenting in
follow-up.
Histories include
1. Tobacco use in
pack-years
2. Evaluation of systemic
symptoms (eg, loss of
appetite, weight loss,
bone pain)
3. Evaluation of neurologic
symptoms (eg, headache)
4. Estimation of functional
status.
Examinations include
1. All palpable
lymphadenopathy. Care
incorporates patients’
health literacy, including
their understanding of
disease, treatment
options, and prognosis. It
involves teaching family
and friends and involving
this network in the
patient’s care.
Understands the staging
system for head and neck
cancer. Learns the role of
appropriate surgical staging.
Identifies patient stages that
can be treated with curative
intent. Learns and
understands the value of
adjuvant and neoadjuvant
chemotherapy, the role and
goals of palliative chemotherapy, and the role of clinical
trials in patient care. Learns
to apply symptom
management and recognize
and treat chemotherapy
adverse effects when
appropriate.
Discusses cases in tumor
boards. Includes in the
consult note a thoughtful
discussion of the reasons
for the treatment
recommendation and, when
appropriate, a reference.
Includes staging, personally
reviews computed
tomography scan, and
reviews laboratory results
for systemic disease and
treatment options. For the
treatment plan, debriefs
with an attending physician
for patients on chemotherapy who are having
significant adverse effects,
especially those that require
dose adjustment.
Evaluation for Head and Neck
Cancer
Direct observation
Presents a case to the
division. The case is one
that stimulated interest on
the fellow’s part and it
should be a formal
presentation that includes
radiographs and pathology
slides.
Proficiency is expected to
improve from years 1 to 3.
In-training examination scores
are expected to improve
from years 1 to 3
Self-reflection tool
QOPI
(continued on following page)
www.jco.org
© 2010 by American Society of Clinical Oncology
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
3663
Collichio et al
Table 2. Guide for the Six ACGME Competencies in Hematology/Oncology Using Head and Neck Cancer as an Example (continued)
Competencies
Content
Possible Evaluation Methods
Interpersonal and
communications
skills—Residents must
demonstrate
interpersonal and
communication skills
that result in effective
information exchange
and teaming with
patients, patients’
families, and
professional
associates.
Effectively discusses with
patient and teaches
patient and family about
the patient’s condition,
results of tests, and
choices and goals of
treatment. Effectively,
honestly, and
compassionately tells bad
news and helps patient
and family clarify goals.
Facilitates family
meetings, explains
advance directives, and
establishes code status.
Provides courteous,
professional, and timely
consultation and works
effectively with
consultants.
Communicates
effectively, courteously,
and professionally with
nursing and clinic staff
and peers.
360-degree observation-based
assessment
Mini clinical examination
Professionalism—Residents
must demonstrate a
commitment to
carrying out
professional
responsibilities,
adherence to ethical
principles, and
sensitivity to a diverse
patient population.
Provides timely, courteous,
and thorough patient
care. Communicates with
consulting team after
developing assessment
and plan. Provides legible,
comprehensive, and
timely medical records.
360-degree patient surveys
Objective data (chart
completion)
Systems-based practice—
Residents must
demonstrate an
awareness of and
responsiveness to the
larger context and
system of health care
and the ability to
effectively call on
system resources to
provide care that is of
optimal value.
Uses multidisciplinary
diagnostic resources,
particularly electronic, but
also other consultants,
including national experts.
Uses expertise of
members of
interdisciplinary team.
Works effectively with
members of the
interdisciplinary team,
including social workers,
physical therapists,
pharmacists, nurses, and
clinical research team
members, to provide
comprehensive patient
care.
Quality assessment tool
Systems audit
Cost-effective care
Safety attitude questionnaire
Content for Head and Neck
Cancer
Evaluation for Head and Neck
Cancer
Effectively participates in
discussions with patients
and their families about
their cancer. As the rotation
progresses or as the year
unfolds, the trainee will lead
these discussions. Provides
results of tests, choices,
and goals of treatment.
Integrates work with the
nurse extenders. Relies on
the nurse, when
appropriate, for patient callbacks for updating normal
results that were not
available at clinic,
instructions on medicines,
clarification of scheduled
tests, and instructions prior
to tests such as “NPO after
midnight.” Effectively
communicates with the
nurses who administer the
chemotherapy, including
accurate chemotherapy
orders that comply with
hospital policy, call-backs to
questions that are made
through the paging system,
and visits to the patient and
nurse in the chemotherapy
room when requested.
Contacts an attending physician
from the group and
discusses how the patients
will be distributed.
Completes clinic note within
24 hours of the encounter.
The note should be signed
(key board templates are
automatically signed) within
48 hours unless waiting for
key laboratory data to
support the document.
Attends all required
fellowship conferences.
Maintains continuity clinics.
Participates in the on-call
duties.
360-degree direct observation
Communication tools should
show improvement from
year 1 to 3
Appreciates the multidisciplinary
approach to the diagnosis
and treatment of head and
neck cancer. Integrates
thoracic surgeons, radiation
oncologists, social workers,
and patient counselors in
patient care. Integrates work
with support from
pharmacists and social
workers.
Quality assessment tool
Thorough notes are completed
on time.
Patient surveys should show
improvement from year 1 to
3.
Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; QOPI, Quality Oncology Practice Initiative; NPO, nil per os 关nothing by mouth兴.
ⴱ
The terms “residents,” “fellows,” and “trainees” are used interchangeably.
3664
© 2010 by American Society of Clinical Oncology
JOURNAL OF CLINICAL ONCOLOGY
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
Organizing a Fellowship Program and ACGME Competencies
Step 8: Evaluation
The ACGME requires formal assessment of each trainee, the
trainers, and the program. Evaluation and feedback are essential for
ongoing professional development and to provide check points for
positive change in the trainee, the trainers, and the program. If assessments are made with multiple sources and at multiple time points, the
process should be fair, acceptable, and more reliable than using minimal time points and evaluators.14
Evaluations of trainees must be structured to include components of all six core competencies. Medical knowledge is the one
competency that is easiest to evaluate and measure. ASCO and the
American Society of Hematology (ASH) ITEs are useful tools for this
purpose.15 The ASCO Medical Oncology ITE also attempts to include
some competencies other than medical knowledge.15 Most of the
accredited programs—138 programs in 2008 and 145 programs in
2009 —participated in the ASCO ITE. Most programs had their first-,
second-, and third-year fellows take the examination. There was no
ASH examination in 2008. In 2009, ASH had 1,018 test takers. Each
examination has a modest fee per trainee. Programs should budget for
the fee or at least let the trainees know early on about these opportunities and the required fees. Ideally, the program, rather than the
individual fellow, pays for the examination.
Some programs have adopted a 360-degree assessment to measure patient care, professionalism, and communication competencies.
The 360-degree assessment, or multisource evaluation, describes evaluations by multiple caregivers from different categories such as nurses,
pharmacists, social workers, and other allied health professionals, as
well as colleagues such as other fellows. Implementation is challenging
because multiple evaluators are needed, and the data collection and
collation from multiple evaluators at multiple time points can be
time-consuming and difficult to complete.16
Standardized patients are used in internal medicine for patient
care and communication and are beginning to be used in oncology
training programs. Practice-based learning can be done by selfreflection and with other tools such as ASCO’s Quality Oncology
Practice Initiative (QOPI).17 Systems-based practice is often considered the most challenging competency to assess. It can be measured by
quality assessment tools and systems audits and also by faculty assessment of the fellows’ ability to work within the system for the betterment of their patients. This competency involves awareness of the
health system where the fellow works, considerations of cost-benefit
and risk-benefit, optimal negotiation within interprofessional teams,
and ability to identify systems errors. This competency is often evaluated by direct observation of oncology fellows. Professionalism also
presents difficulties in assessment. Studies show that professionalism
aligns with academic ability.18 Patient surveys and data gathered from
360-degree assessments can be helpful. Professionalism includes the
ability to follow rules and complete tasks. In this regard, professionalism can be assessed by looking at objective data, such as physicians
completing their charts on time, going to mandatory meetings,
treating colleagues with respect, and keeping up with educational conferences.
The proposed 2011 revision in the Program Requirements states
that competency in procedures must be based on proficiency rather
than the number of procedures completed. Forms and processes are
currently being developed. When tools for documenting proficiency
are completed, they could be added to the handbook. A tool to evaluwww.jco.org
ate communication of bad news, for example, is shown in Appendix
Figure A1A and A1B (online only).
It is important to train the evaluators in the process and content
of evaluation tools. The faculty may not be familiar with the six
competencies, especially if the faculty were trained before the competencies were introduced in the late 1990s. Feedback to faculty on their
performance as evaluators can improve the quality of the evaluation
and help fulfill the requirement for faculty development.19 The program can keep track of its evaluations by any systematic method.
There are a number of electronic systems that programs or their GME
parent offices can purchase to assist in the process of evaluation.
Step 9: Research
Some programs may choose to emphasize research. These programs must maintain the 18 months of clinical requirements for
combined training in the 3-year program plus the continuity clinic
experience or the 12 required clinical months for the single-specialty
training plus the continuity clinic experience. These programs may
choose to describe research in terms of tracks such as clinical research,
basic laboratory research, or any other relevant hematology/oncology
endeavor. No matter what is decided, the ACGME requires supervised
scholarly activity for any trainee enrolled in a program of 24 months
duration. The activity should be in blocks of protected time and
should include teaching the trainee the elements of ethical conduct of
research.4 Research can be funded by a training grant, by industry, or
by other sources such as the ASCO Young Investigator Award.8,20 The
ACGME requires that the majority of fellows demonstrate evidence of
research productivity through publications or by presentations of
abstracts at national specialty meetings. Training programs can enhance their trainees’ opportunity for success by formally assessing the
trainees’ scholarly progress. A form to document progress is shown in
Appendix Figure A2 (online only). Trainees who receive funding
through NIH training grants must limit their clinical time during the
period of the grant (see Step 2).
Step 10: Didactic Lectures and/or Seminar Series
A series of lectures or seminars must be part of any program.4
ASCO’s core curriculum can serve as an overall template for topics.10,11 Topics could also come from the blueprints of the ABIM
certifying examinations in hematology and oncology.3 The topic list
should cover the basic knowledge required for the specialty and could
have guest faculty from the division, department, or other areas of the
medical center. Fellows are encouraged to participate as lecturers.
Having fellows give some of the program’s lectures will help them
develop their proficiency as teachers and expert speakers; however,
faculty attendance at such presentations is still expected by ACGME to
ensure that accurate information is disseminated. A formal mechanism for providing feedback to the fellows giving lectures is helpful. A
list of topics is shown in Appendix Table A1 (online only). The example shows how some topics can be appropriate to the learner in hematology and oncology whereas some topics are specific to either
hematology or oncology. The example includes topics that are part of
systems-based practice (Coding and Compliance Training) and communication (eg, The Psychiatric Issues Cancer Patients Face). The
program can be creative as well and include topics such as “Medical
Burn-Out” or “How to Give a Presentation.” A board review session
can be useful and, in many programs, is run by the fellows.
© 2010 by American Society of Clinical Oncology
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
3665
Collichio et al
Step 11: Monitoring and Adhering to the ACGME and
ABIM Requirements Including the Program
Information Forms and Surveys
The ACGME reviews accredited programs every 2 to 5 years. This
review is an assessment of whether the program complies with the
ACGME guidelines by offering appropriate educational programs,
adequate and varied patient volume, feedback mechanisms that are
fair and timely, opportunity for research or other scholarly activities, a
compassionate work environment with sufficient space, and adherence to work hour standards so that the trainee is able to learn effectively and provide quality care. A program that is constantly aware of
the ACGME guidelines and works in a well-defined structure that is
documented in an organized handbook (see Step 5) will find the
review process to be relatively easy and will make the overall
program more enjoyable for trainees, teachers, and support staff.
The information that is required in the Program Information Form
(PIF) can be documented in the handbook. The ACGME annual
survey captures compliance with the regulations. It is important to
review the survey with the trainees so that everyone is aware of what is
offered in the program.
Steps 1 through 10 allow the program to have an organized,
smooth performance but attention to detail and to the dynamics of
change are essential for efficient transitions from year to year. It is
important to recognize that while the ACGME policies and procedures are designed to promote excellent education, the details of these
policies and procedures may not be intuitively obvious to clinical
faculty, including novice program directors. Program directors as well
as program administrators should review the ACGME Web site on a
periodic basis and consider attending the annual ACGME meeting.4
The ACGME requires that a program have key faculty and that
they are qualified, dedicated, and scholarly. Board certification is the
sine qua non of qualification for key faculty. Dedication is defined
broadly as the faculty’s ability to provide an environment conducive to
education, including faculty presence at case conferences, rounds, and
journal club in addition to traditional patient care supervision and
lecturing. Key faculty are defined more narrowly as those who dedicate, on average, 10 hours per week throughout the year to the training
program. For programs with more than five fellows, a ratio of key
clinical faculty to fellows of at least 1:1.5 must be maintained. It can be
challenging to account for the exact teaching time of faculty whose
involvement with fellows may change from day to day and month to
month. Appendix Figure A3 (online only) shows a method for accounting for teaching time.
The ACGME also requires that a majority of the faculty engage in
scholarship, such as peer review funding or publication of original
research, review articles, editorials, case reports, or chapters in textbooks.19 The regulations are quite specific about scholarly expectations, and the program director should periodically monitor the
faculty’s productivity.
Step 12: Personnel Issues
A program director’s major challenge—and one for which there
is usually inadequate training—is that of dealing with human beings.
Fellows are people and thus are at risk for generating human problems. Often the program director is expected to fix things. Unexpected
illness can disrupt a well-planned clinical schedule. A fellow’s minor
run-in with a nurse or faculty member can take significant time to
resolve. Despite the best intentions of the program and the program
3666
© 2010 by American Society of Clinical Oncology
director, there can be instances of the problem fellow who is not able to
keep up with the requirements of the program or is disruptive to the
program in some other way. A detailed guide on how to deal with this
type of problem is beyond the scope of this article, but it is important
to include it in one of our steps, because every program director
should be aware of the institution’s programs to help residents with
the stress of training. Program administration guidelines should include formal grievance procedures and the appropriate personnel
management process. Unfortunately, most program directors receive
little, if any, training in this important area.21-23
The importance of documentation cannot be overstated. For
example, if a nurse calls in to question the ability of a fellow one time,
then there may not be a case to question the fellow. But, if there are
several instances questioning academic or clinical ability and several
different people are making these queries, then a case can be made to
discuss and document the situation with the trainee. If the program
director has to have a stressful face-to-face encounter with the trainee,
consideration should be given to having a witness for both parties. In
the clinical environment, patient safety comes first. If the behavior in
question could adversely affect patient outcome, the hospital or clinical administration, in addition to the program itself, may get involved.
Step 13: Keep Track of Trainees After They Leave
A system for keeping up with trainees after they leave can not only
bring good will when it is time to include them in the alumni events
and seminars, but it can also be helpful when it is time to provide data
on board certification status and the jobs or careers that fellows pursue
after completing their fellowship. Some training grants require this
information on prior trainees as part of the grant application process.
The ACGME requires a summary by the program director of the
trainee’s performance during the final period of education and verification that the trainee has sufficient competence to enter practice
without direct supervision.4
In conclusion, training fellows in hematology and oncology is a
complex process, not only because of the enormous amount of information and skills that must be mastered in these fields but also because
of the complex environment of medical practice that requires proficiency and expertise in systems, communication, and professionalism.
The Six Competencies of Graduate Medical Education developed by
the ACGME help programs keep track of these areas in their training.
Programs are required to adhere to these competencies and to document the processes by which they do so. The steps and forms in this
article are intended to help each program understand and adhere to
the requirements of fellowship training. Our hope is that organized,
systematic approaches, as described here, will reduce programmatic
and administrative tension and allow faculty and fellows to enjoy the
fellowship experience. There is likely no greater gift we can give to our
patients than a well prepared next generation of oncologists.
AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS
OF INTEREST
The author(s) indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Frances A. Collichio, Timothy J. Moynihan,
James A. Stewart
JOURNAL OF CLINICAL ONCOLOGY
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
Organizing a Fellowship Program and ACGME Competencies
Administrative support: Frances A. Collichio
Collection and assembly of data: Frances A. Collichio, Michael P. Kosty,
James A. Stewart
Data analysis and interpretation: Frances A. Collichio, James A. Stewart
REFERENCES
1. Batalden P, Leach D, Swing S, et al: General
competencies and accreditation in graduate medical
education. Health Aff (Millwood) 21:103-111, 2002
2. Accreditation Council for Graduate Medical
Education: Resident Services, 2009. http://www
.acgme.org/acWebsite/home/home.asp
3. The American Board of Internal Medicine:
Take the exam—Your complete guide, 2009. http://
www.abim.org/exam/exams.aspx
4. Accreditation Council for Graduate Medical
Education: Program Director Guide to the Common
Program Requirements, Version 2.2, 2009. http://
www.acgme.org/acWebsite/navPages/commonpr_
documents/CompleteGuide_v2%20.pdf
5. Association of American Medical Colleges
(AAMC), AAMC Division of Health Care Affairs:
AAMC Survey of Housestaff Stipends, Benefits and
Funding: Autumn 2007 Report. http://www.aamc
.org/data/stipend/2007_hssreport.pdf
6. Covey AS, Friedlaender GE: Financing graduate medical education: Sorting out the confusion.
J Bone Joint Surg Am 85-A:1594-1604, 2003
7. US Department of Health and Human Services: Office of Extramural Research, 2009. http://
grants1.nih.gov/grants/oer.htm
8. Nuckols TK, Bhattacharya J, Wolman DM, et
al: Cost implications of reduced work hours and
Manuscript writing: Frances A. Collichio, Michael P. Kosty, Timothy J.
Moynihan, Thomas H. Davis, James A. Stewart
Final approval of manuscript: Frances A. Collichio, Michael P. Kosty,
Timothy J. Moynihan, Thomas H. Davis, James A. Stewart
workloads for resident physicians. N Engl J Med
360:2202-2215, 2009
9. Accreditation Council for Graduate Medical Education: Policies and Procedures, 2009. http://www
.acgme.org/acWebsite/about/ab_ACGMEPolicies
Procedures.pdf
10. American Society of Clinical Oncology: Curriculum for Medical Oncology, 2009. http://www.asco
.org/ASCOv2/Education⫹%26⫹Training/ASCO⫹
Program⫹Guidelines
11. Muss HB, Von Roenn J, Damon LE, et al:
ACCO: ASCO core curriculum outline. J Clin Oncol
23:2049-2077, 2005
12. Van Dongen HP, Maislin G, Mullington JM, et
al: The cumulative cost of additional wakefulness:
Dose-response effects on neurobehavioral functions and sleep physiology from chronic sleep restriction and total sleep deprivation. Sleep 26:
117-126, 2003
13. Dawson D, Reid K: Fatigue, alcohol and performance impairment. Nature 388:235, 1997
14. Lockyer J: Multisource feedback in the assessment of physician competencies. J Contin Educ
Health Prof 23:4-12, 2003
15. Collichio FA, Kayoumi KM, Hande KR, et al:
Developing an in-training examination for fellows:
The experience of the American Society of Clinical
Oncology. J Clin Oncol 27:1706-1711, 2009
16. Accreditation Council for Graduate Medical
Education, American Board of Medical Specialties:
Toolbox of Assessment Methods: A Product of the
Joint Initiative—ACGME Outcomes Project, Version
1.1, 2000. http://www.acgme.org/outcome/assess/
toolbox.pdf
17. American Society of Clinical Oncology: The
Quality Oncology Practice Initiative, 2009. http://
qopi.asco.org/
18. Veloski JJ, Fields SK, Boex JR, et al: Measuring professionalism: A review of studies with instruments reported in the literature between 1982 and
2002. Acad Med 80:366-370, 2005
19. Accreditation Council for Graduate Medical Education: Internal Medicine Program Requirements,
2009. http://www.acgme.org/acwebsite/rrc_140/140_
prindex.asp
20. ASCO Cancer Foundation, 2009. http://www
.ascocancerfoundation.org/TACF/Grants/Grant⫹
Opportunities/Young⫹Investigator⫹Award
21. Minicucci RF, Lewis BF: Trouble in academia:
Ten years of litigation in medical education. Acad
Med 78:S13–S15, 2003
22. Helms LB, Helms CM: Forty years of litigation involving residents and their training: I. General programmatic issues. Acad Med 66:649-655,
1991
23. Papadakis MA, Teherani A, Banach MA, et al:
Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med 353:26732682, 2005
■ ■ ■
www.jco.org
© 2010 by American Society of Clinical Oncology
Downloaded from jco.ascopubs.org on September 16, 2014. For personal use only. No other uses without permission.
Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
3667