Published Ahead of Print on July 6, 2010 as 10.1200/JCO.2010.28.1964 The latest version is at http://jco.ascopubs.org/cgi/doi/10.1200/JCO.2010.28.1964 JOURNAL OF CLINICAL ONCOLOGY S P E C I A L A R T I C L E How to Organize a Fellowship Program: Lessons Learned and How to Include Accreditation Council for Graduate Medical Education Competencies in the Curriculum Frances A. Collichio, Michael P. Kosty, Timothy J. Moynihan, Thomas H. Davis, and James A. Stewart From the University of North Carolina Chapel Hill, Chapel Hill, NC; Mayo Clinic, Rochester, MI; Scripps Clinic, La Jolla, CA; Dartmouth Hitchcock Medical Center, Lebanon, NH; and Baystate Medical Center, Springfield, MA. Submitted January 29, 2010; accepted May 4, 2010; published online ahead of print at www.jco.org on July 6, 2010. Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article. Corresponding author: Frances Collichio, MD, Division of Hematology Oncology, University of North Carolina, 170 Manning Dr, Box 7305, Chapel Hill, NC 27599; e-mail: [email protected]. © 2010 by American Society of Clinical Oncology 0732-183X/10/2899-1/$20.00 DOI: 10.1200/JCO.2010.28.1964 INTRODUCTION Over the past 15 years, the complexity in fellowship training in hematology/oncology has increased. This is because of the vast growth of knowledge in these fields and a change in the structure and accreditation requirements of graduate medical education. The latter was put into place by the Accreditation Council for Graduate Medical Education (ACGME) in the mid 1990s.1 The ACGME responded to a sense that it was difficult to determine whether physicians were properly trained. Medical students, residents, and fellows work in complex environments and, unlike in formal classrooms in high school and college where teaching and testing are uniform, physicians in training can be exposed to a range of unique experiences, and assessments of progress can be arbitrary. The ACGME review called into question the entire process and prompted a structure that is known as the “Six Competencies of Graduate Medical Education.” Now, fellowship programs in hematology/oncology must expose their trainees to the knowledge in the field while also complying with assessments of these six competencies and must show by a variety of measures that their trainees are achieving accepted standards. This article is intended to show programs, program administrators, trainees, and the oncology community in general, steps that can lead to a successful hematology/ oncology training program with the ultimate goal of having the finest trained specialists in the field of hematology and oncology. STEPS TO RUNNING A GRADUATE MEDICAL EDUCATION PROGRAM Step 1: Program Structure Each program must determine the overall structure of its fellowship. It is important to have a vision of how the fellowship appears to candidates, reviewers, and faculty. What is the flow of rotations and are they appropriately graded in skill level, challenge, and responsibility as the fellow progresses through the program? Is there a good balance of time between clinical experience with time spent in journal clubs, lectures, and other educational activities? Are all activities assessed for educational value? Oversight of all hematology/oncology fellowships is provided locally by the home institution’s Department of Medicine education office and GME committee and nationally by the ACGME through its Residency Review Committees (RRCs). The ACGME provides oversight of the program with evaluation and accreditation of all residencies.2 The American Board of Internal Medicine (ABIM) is an independent evaluation organization that certifies individual internists and internal medicine subspecialists.3 Both the ACGME and ABIM have useful Web sites that are important resources for fellowship directors and fellows. The ACGME site provides detailed descriptions of minimum program requirements and offers a variety of tools to help in program organization and improvement. The ABIM site has a blueprint of the content of the examinations for the hematology or oncology certifying board examinations. It is important for program directors and administrators to become familiar with these Web sites because information changes.4 The ACGME describes the minimum elements needed for a successful program and specifies details of the time needed for clinical and overall training and the required procedures that should demonstrate competence in such procedures as bone marrow biopsies and intrathecal administration of chemotherapy. To be eligible for board certification in either hematology or oncology, the trainee must complete 2 years in an accredited program of which at least 12 months must be clinical training. By July 2011, 50% of this clinical experience will need to be in the outpatient setting. To be eligible for board certification in both hematology and oncology, there must be at least 18 months of clinical work (50% in the outpatient setting), and 6 months will be devoted to non-neoplastic hematology (NNH). A trainee is required to have continuity clinic experience and a scholarly project that demonstrates systems-based practice and practice-based learning and improvement. With these guidelines in mind, a program can become accredited by the ACGME to offer training © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Copyright 2010 by American Society of Clinical Oncology 1 Collichio et al in hematology, oncology, or both. The program can choose to mix NNH rotations with oncology rotations, or it may choose to separate them. The ACGME does not specify how much time a program should devote to research or scholarly activities. The ACGME recognizes that some programs will make research a large part of their identity whereas other programs will not. A program could choose to devote the 18 months that remain after accounting for the required clinical activity to research or to a scholarly activity, or the program could choose to commit a smaller portion of this time. The research and scholarly options can range from small projects, to clinical trials, to laboratory research, to any combination thereof. The ABIM and ACGME also allow for flexibility. For example, a 6-year research track exists for residents who determine early on that they wish to have an academic, research-based career. The trainee can enter an accredited fellowship program after the second year of their internal medicine residency.2 Step 2: Funding Funding hematology and oncology fellowship programs can be complicated. Fellows earn a salary and benefits for their postgraduate year of training that meet the standard, which averages about $75,000 per fellow per year.5 Fellowship programs must also provide for administrative support. The ACGME requires that a portion of the program director’s time be funded. With increasing demands on program directors related to fellowship administration and program development, it is recommended that program directors have a serious discussion with their division chief regarding just how much time is allocated for fellowship activities. Some programs receive support from the parent institution, and some use divisional or departmental funding mechanisms. Most fellowship programs have a fellowship coordinator, and many have an associate program director, depending on the size of the program. Adequate dedicated time and effort are needed for those working on fellowship activities. Programs must also account for space, supplies, and other support needed for educational programs. Considering all of this, the precise dollar amount needed is difficult to estimate but it could be as high as $100,000 per fellow per year. In the 2006-2007 academic year, the primary source of funding for private hospitals was patient revenue.5 Funding sources include faculty practice plans, patient care revenue, endowments, the Department of Veterans Affairs, the Department of Defense, Medicaid, and Medicare. The money for trainee salaries usually goes to the parent institution with distribution from the GME office rather than to the individual divisions and departments.6 As a consequence, most fellows are employees of the hospital and not the medical school in most fellowships. Another source of funding is from research grants. Program directors must be careful in program use of research funds to ensure the fellow is in fact doing research during the time the fellow is funded by research support. The National Institutes of Health (NIH) has National Research Service Award (NRSA) Institutional Research Training Granings (T32) that require protected time away from clinical responsibility, so careful attention to scheduling is important for these trainees.7 The NIH has Institutional Research and Academic Career Development Award (K12) grants for more experienced research fellows. The mixture of NIH research dollars and clinical training can be complex; thus, the program director should work closely 2 © 2010 by American Society of Clinical Oncology with the GME department and administrators who are knowledgeable about fellowship and research funding. Once these sources of money have been identified by the program, the program must decide how clinical work will be balanced with nonfunded educational endeavors. In the United States, there has been a shift toward having residents and subspecialty residents (fellows) spend more time in educational activities and less time in clinical service. Programs must have didactic conferences, examples of which include journal clubs, board reviews, clinical and basic science lectures, and in-training examinations (ITEs) as well as having clinical and research mentoring programs. All of these activities take time away from clinical work, but they are critical to promoting highquality training and adherence to ACGME standards. Clinical work in the hospitals and clinics requires direct supervision of fellows. They are expected to develop expertise and clinical efficiency as they progress through the program, thus requiring differing levels of supervision and faculty time as they mature. Supervising staff must be readily available to the trainees and not have an excessive number of patients or other duties that interfere with their ability to supervise the care of the fellows’ patients. This usually requires the program to free up the faculty to be completely involved. Training programs must account for faculty supervision when they set up their services. The overall amount of clinical time that residents and fellows can provide has decreased. The ACGME work hour restrictions have led to more costs related to clinical care. Many programs have increased the numbers of hospitalists and midlevel providers to offset the reduction in trainee clinical time resulting from work hour restrictions.8 Moreover, to be fair and compliant with the mission of the NIH training grants, fellows who earn a spot in these programs need to have protected time away from clinical responsibility, sometimes as much as 80%. In the past, these programs were used to fund training, but now they are more restrictive and also quite competitive.7 All of this can lead to outstanding training but limits the amount of clinical service (and indirectly, clinical revenue) the trainees can provide. Step 3: Program Size The number of trainees in a given program is a complex issue determined by multiple factors. The minimum number of trainees specified by the ACGME is one fellow per year of program length. Most importantly, the institution must have faculty educators who are willing and able to share clinical expertise. In addition to the program director, the ACGME requires key clinical faculty who dedicate, on average, 10 hours per week throughout the year to the training program. For programs with more than five fellows, a ratio of key clinical faculty to fellows of at least 1:1.5 must be maintained. The ACGME does not specify the number of clinical faculty required when there are fewer than five fellows. Second, the institution should provide adequate patient volume and mix for robust training. Third, adequate funding must exist (see Step 2). Finally, the ACGME has to approve the number of trainees in any given program on the basis of information it receives from the program administration and the ability of that program to adequately supervise and educate those trainees. Sometimes the number of trainees can change. A program may be granted 15 trainees in total but, because of a personal event for a trainee, the ACGME may grant a temporary change in the number. If, for example, a trainee leaves for maternity reasons and then returns to complete the training several months later, the program may have more than the 15 trainees when the trainee returns from leave. A JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Table 1. Example of How Rotations Could Be Counted Toward Oncology Board and Hematology Board Eligibility Rotation Type Hematology (NNH) Board Required Consult rotation Outpatient hematology Hematopathology Elective Coagulation laboratory Private practice Transfusion medicine Oncology Board Inpatient oncology Leukemia service Bone marrow transplantation Outpatient oncology Genetics Gynecologic oncology Radiation oncology Palliative care Private practice Abbreviation: NNH, non-neoplastic hematology. catastrophe that affects an entire program, such as Hurricane Katrina, may require trainees to transfer to other programs. In these economically turbulent times, it is possible that hospitals with ACGME-approved programs may close, which would require a transfer of trainees to other sites. The ACGME has policies to facilitate this process.9 NAME Step 4: Clinical Experiences Combined programs can determine which of their rotations will meet the eligibility requirements for oncology and which will meet the eligibility requirements for hematology. For the combined certification in hematology and oncology, 18 clinical months are required, 6 months of which must be in NNH. Guidelines for making these distinctions can come from the American Society of Clinical Oncology (ASCO) core curriculum.10,11 The topics that are listed for certification examination by the ABIM can also help programs decide how to divide clinical experience time.3 Table 1 provides an example of rotations that could be categorized as non-NNH and those that could be categorized as oncology. Each year, the ABIM requires that the program enter the number of months that the trainee spent in NNH, hematologic malignancies, clinical oncology, research, and other. For a trainee who is seeking eligibility for both boards, keeping an accurate count of this information can be particularly important. Figure 1 is a form that shows how a program could keep track of its rotations; it can be modified by each program. A continuity clinic experience throughout the fellowship training is required. General oncology experiences might be available at Department of Veterans Affairs hospitals, private offices, regional clinics, and in other urban venues. In many university-based cancer programs, there are disease-oriented clinics with significant Date Counting Rotations Toward ABIM Boards For Heme Board Certification Nonneoplastic Hematology (NNH) Totals Coagulation lab Hematopathology Transfusion medicine Consult service Heme outpatient Total for NNH Equals 6 Fellowship Year 1 Fellowship Year 2 Fellowship Year 3 Totals Equals 6 For Oncology Board Certification Heme malignancies Heme malignancy inpatient Bone marrow transplant Fig 1. A form for keeping track of rotations that count toward American Board of Internal Medicine (ABIM) board certification. Heme, hematology; GYN, gynecology. Total Clinical Oncology Inpatient Outpatient oncology Genetics GYN oncology Radiation oncology Palliative care Private practice Total Total Clinical for Oncology Boards Equals 12 Total Clinical for NNH and Oncology Total Research or Other Years 2, 3 Total Months Per Year www.jco.org © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 3 Collichio et al faculty subspecialization. This could mean that a trainee might get an intense experience in one disease site, such as breast cancer, if the trainee stayed with that clinic throughout the training program, but miss a continuity experience in populations with other diseases. To avoid a limited clinical experience, a fellow can switch from one continuity clinic to another, but each clinic should be of least 6 months duration to achieve a meaningful longitudinal experience. The ACGME requires that the continuity clinic have four to eight patients during each half-day session and that there should be at least 25% of patients of each sex. Each program director and the program administration should keep track of the continuity clinic requirements, because the ACGME guidelines may change.4 For each continuity experience, the fellow should have a population of patients in which the fellow is viewed as the primary oncologist and in which there is sufficient time to experience important natural history elements of the disease type. Step 5: A Competency-Based Training Manual A training manual or handbook can help the program be successful. This handbook could be a source document that the ACGME uses to review the overall structure of the program, and it can be updated on an annual basis as features of the program change. The handbook could read like a book, with a table of contents, followed by key telephone numbers, people to contact, and Web sites. An overview that states the goals of the program and the available tracks such as clinical, clinical with clinical research, clinical with laboratory research, or other tracks is helpful. Sections should be included on mandatory and elective clinical rotations, the program’s scholarly structure and research opportunities, how evaluation of fellows and key faculty takes place, and how residency work hours are accounted for. Other key documents such as due process guidelines, program benefits, policies on personal health, and on-call responsibilities could also be included. Wherever possible, the competencies as drivers of programmatic goals and fellow evaluations should be emphasized. The ACGME checks to see that rotations are described within the framework of the six competencies and that the goals and objectives for each rotation are appropriate for each year of training and include progressive levels of responsibility. ASCO has guidelines for core curriculum, and ASCO is considering developing competency-based curricula for specific cancers and cancers in general.10,11 It is unlikely that this project could meet the needs of all of the programs, and it could be too specific for some. It is essential that programs adapt some of the preexisting tools and develop their own multisource assessments.4 The handbook could provide an overview that is tailored to the program. Table 2 lists each of the six competencies and shows what content could be considered in each one with respect to hematology and oncology. Table 2 also shows how the six competencies can be applied using head and neck cancer as an example. Step 6: Interview Process Fellowship programs in hematology and oncology joined the National Residency Matching Program (NRMP) in 2007. The NRMP uses the Electronic Residency Applications System (ERAS) to distribute the applicants’ documents to programs and to collect the programs’ ranking lists. To participate, 75% of the programs in the specialty and 75% of the available spots in each program must be listed in the match. These rules exist to keep the system fair and open to the applicants. There can be special circumstances that allow programs to 4 © 2010 by American Society of Clinical Oncology allocate some of their slots outside the match. There may be, for example, a couple that wishes to match into one program. Or there may be an educational reason to take an applicant outside of the match so as to accommodate a trainee who wishes to pursue preparation in both geriatrics and oncology at the same program. Each program should decide how many applicants to interview. An Internet chat on ASCO’s Listserve among program directors suggested that a good ratio is to interview eight applicants for every available position. Each program, however, has to learn by experience what it takes to fill its slots, so the ratio may vary by program. Competitive programs can have more than 300 applicants and it can be time-consuming to review the information on ERAS. Funding restrictions at some institutions cannot accept international applicants with certain kinds of visas. ERAS has a visa filter as well as other helpful filters to sort applications by United States Medical Licensing Examination (USMLE) scores, medical schools, and other details that may be important to the individual program. Having more than one person screen the applicants, such as the program director and the associate program director(s), can streamline the process, and the screeners can look for a list of optimal candidates to invite for an interview. The program should decide what the interview process should be like. The program, for example, may wish to have a more personalized setting by interviewing one or two applicants at a time. Other programs offer interviews only on a few days and therefore bring in several applicants at a time. No matter what the approach, a written or oral presentation by the program director or by the program director and several faculty members is useful. It is also important to train interviewers about the types of questions that are out of bounds according to federal equal opportunity employment law. Each institution’s human resource department can provide guidance. Finally, a system to determine how the applicants are ranked is helpful. If each applicant is interviewed by two or more people and given a score, the average score can be used as an early ranking list. Of course, over time the program director will recognize who among the interviewers are the hard graders and who are the easy graders and adjust the rankings accordingly. After the interview process is complete, a match meeting is useful to review each resident interviewed and devise a rank list that the program director will submit to the NRMP. There should be documentation that this meeting took place to have a record showing that the selection process is fair and reasonable. This is an important defense against the rare employment discrimination lawsuit. The program director must keep in mind the ERAS deadlines for submission of the number of spots available and the final ranking list. Step 7: Fellowship Work Hours The ACGME has a zero-tolerance policy toward work hour violations. Studies have shown that sleep deprivation is correlated with poor performance and can lead to poor quality of care including dangerous medical errors.12,13 Work hour mandates began in 2004. They apply to the clinical work that resident and subspecialty residents (fellows) do. At this time, they include a maximum of 80 hours per week averaged over a 4-week period, no more than 24 consecutive hours with 6 additional hours for transfer of care, continuity clinic, didactic lessons, and maintenance of continuity of care, a 10-hour break between shifts, and an average of 1 day off in 7 days, averaged over 4 weeks.4 Web-based tools have been devised and are being used to help the program keep track of work hours. JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Table 2. Guide for the Six ACGME Competencies in Hematology and Oncology Using Head and Neck Cancer as an Example Competencies Content Possible Evaluation Methods Patient care—Residentsⴱ must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. Records comprehensive history and performs physical examinations with attention to issues related to treatment toxicity. Plans interdisciplinary care, conducts family meetings, evaluates patient’s health literacy and understanding of disease, treatment options and prognosis, and evaluation and treatment of symptoms. Direct observation of a mini clinical examination Standardized patient Structured clinical examination Simulated procedure Medical knowledge—Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate (eg, epidemiologic and social-behavioral) sciences and the application of this knowledge to patient care. Demonstrates knowledge of evidence-based, diseasespecific information regarding diagnosis, staging, treatment options and prognosis, and symptom management. Demonstrates knowledge of available clinical trials and overall knowledge of clinical trials, including protection of human subjects. Board examinations In-training examinations Test modules Training in ethical conduct of research Practice-based learning and Discusses and debriefs improvement—Residents outcomes of treatment must be able to for each patient. investigate and Discusses chemotherapyevaluate their patient related toxicity and care practices, appraise untoward effects. and assimilate Debriefs after each scientific evidence, and patient death with improve their patient interdisciplinary team. care practices. QOPI Other reflection tools Content for Head and Neck Cancer Records comprehensive history and performs physical examination for all new consults and provides disease-appropriate history and focused examination for patients presenting in follow-up. Histories include 1. Tobacco use in pack-years 2. Evaluation of systemic symptoms (eg, loss of appetite, weight loss, bone pain) 3. Evaluation of neurologic symptoms (eg, headache) 4. Estimation of functional status. Examinations include 1. All palpable lymphadenopathy. Care incorporates patients’ health literacy, including their understanding of disease, treatment options, and prognosis. It involves teaching family and friends and involving this network in the patient’s care. Understands the staging system for head and neck cancer. Learns the role of appropriate surgical staging. Identifies patient stages that can be treated with curative intent. Learns and understands the value of adjuvant and neoadjuvant chemotherapy, the role and goals of palliative chemotherapy, and the role of clinical trials in patient care. Learns to apply symptom management and recognize and treat chemotherapy adverse effects when appropriate. Discusses cases in tumor boards. Includes in the consult note a thoughtful discussion of the reasons for the treatment recommendation and, when appropriate, a reference. Includes staging, personally reviews computed tomography scan, and reviews laboratory results for systemic disease and treatment options. For the treatment plan, debriefs with an attending physician for patients on chemotherapy who are having significant adverse effects, especially those that require dose adjustment. Evaluation for Head and Neck Cancer Direct observation Presents a case to the division. The case is one that stimulated interest on the fellow’s part and it should be a formal presentation that includes radiographs and pathology slides. Proficiency is expected to improve from years 1 to 3. In-training examination scores are expected to improve from years 1 to 3 Self-reflection tool QOPI (continued on following page) www.jco.org © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 5 Collichio et al Table 2. Guide for the Six ACGME Competencies in Hematology/Oncology Using Head and Neck Cancer as an Example (continued) Competencies Content Possible Evaluation Methods Interpersonal and communications skills—Residents must demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patients, patients’ families, and professional associates. Effectively discusses with patient and teaches patient and family about the patient’s condition, results of tests, and choices and goals of treatment. Effectively, honestly, and compassionately tells bad news and helps patient and family clarify goals. Facilitates family meetings, explains advance directives, and establishes code status. Provides courteous, professional, and timely consultation and works effectively with consultants. Communicates effectively, courteously, and professionally with nursing and clinic staff and peers. 360-degree observation-based assessment Mini clinical examination Professionalism—Residents must demonstrate a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population. Provides timely, courteous, and thorough patient care. Communicates with consulting team after developing assessment and plan. Provides legible, comprehensive, and timely medical records. 360-degree patient surveys Objective data (chart completion) Systems-based practice— Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value. Uses multidisciplinary diagnostic resources, particularly electronic, but also other consultants, including national experts. Uses expertise of members of interdisciplinary team. Works effectively with members of the interdisciplinary team, including social workers, physical therapists, pharmacists, nurses, and clinical research team members, to provide comprehensive patient care. Quality assessment tool Systems audit Cost-effective care Safety attitude questionnaire Content for Head and Neck Cancer Evaluation for Head and Neck Cancer Effectively participates in discussions with patients and their families about their cancer. As the rotation progresses or as the year unfolds, the trainee will lead these discussions. Provides results of tests, choices, and goals of treatment. Integrates work with the nurse extenders. Relies on the nurse, when appropriate, for patient callbacks for updating normal results that were not available at clinic, instructions on medicines, clarification of scheduled tests, and instructions prior to tests such as “NPO after midnight.” Effectively communicates with the nurses who administer the chemotherapy, including accurate chemotherapy orders that comply with hospital policy, call-backs to questions that are made through the paging system, and visits to the patient and nurse in the chemotherapy room when requested. Contacts an attending physician from the group and discusses how the patients will be distributed. Completes clinic note within 24 hours of the encounter. The note should be signed (key board templates are automatically signed) within 48 hours unless waiting for key laboratory data to support the document. Attends all required fellowship conferences. Maintains continuity clinics. Participates in the on-call duties. 360-degree direct observation Communication tools should show improvement from year 1 to 3 Appreciates the multidisciplinary approach to the diagnosis and treatment of head and neck cancer. Integrates thoracic surgeons, radiation oncologists, social workers, and patient counselors in patient care. Integrates work with support from pharmacists and social workers. Quality assessment tool Thorough notes are completed on time. Patient surveys should show improvement from year 1 to 3. Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; QOPI, Quality Oncology Practice Initiative; NPO, nil per os 关nothing by mouth兴. ⴱ The terms “residents,” “fellows,” and “trainees” are used interchangeably. 6 © 2010 by American Society of Clinical Oncology JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Step 8: Evaluation The ACGME requires formal assessment of each trainee, the trainers, and the program. Evaluation and feedback are essential for ongoing professional development and to provide check points for positive change in the trainee, the trainers, and the program. If assessments are made with multiple sources and at multiple time points, the process should be fair, acceptable, and more reliable than using minimal time points and evaluators.14 Evaluations of trainees must be structured to include components of all six core competencies. Medical knowledge is the one competency that is easiest to evaluate and measure. ASCO and the American Society of Hematology (ASH) ITEs are useful tools for this purpose.15 The ASCO Medical Oncology ITE also attempts to include some competencies other than medical knowledge.15 Most of the accredited programs—138 programs in 2008 and 145 programs in 2009 —participated in the ASCO ITE. Most programs had their first-, second-, and third-year fellows take the examination. There was no ASH examination in 2008. In 2009, ASH had 1,018 test takers. Each examination has a modest fee per trainee. Programs should budget for the fee or at least let the trainees know early on about these opportunities and the required fees. Ideally, the program, rather than the individual fellow, pays for the examination. Some programs have adopted a 360-degree assessment to measure patient care, professionalism, and communication competencies. The 360-degree assessment, or multisource evaluation, describes evaluations by multiple caregivers from different categories such as nurses, pharmacists, social workers, and other allied health professionals, as well as colleagues such as other fellows. Implementation is challenging because multiple evaluators are needed, and the data collection and collation from multiple evaluators at multiple time points can be time-consuming and difficult to complete.16 Standardized patients are used in internal medicine for patient care and communication and are beginning to be used in oncology training programs. Practice-based learning can be done by selfreflection and with other tools such as ASCO’s Quality Oncology Practice Initiative (QOPI).17 Systems-based practice is often considered the most challenging competency to assess. It can be measured by quality assessment tools and systems audits and also by faculty assessment of the fellows’ ability to work within the system for the betterment of their patients. This competency involves awareness of the health system where the fellow works, considerations of cost-benefit and risk-benefit, optimal negotiation within interprofessional teams, and ability to identify systems errors. This competency is often evaluated by direct observation of oncology fellows. Professionalism also presents difficulties in assessment. Studies show that professionalism aligns with academic ability.18 Patient surveys and data gathered from 360-degree assessments can be helpful. Professionalism includes the ability to follow rules and complete tasks. In this regard, professionalism can be assessed by looking at objective data, such as physicians completing their charts on time, going to mandatory meetings, treating colleagues with respect, and keeping up with educational conferences. The proposed 2011 revision in the Program Requirements states that competency in procedures must be based on proficiency rather than the number of procedures completed. Forms and processes are currently being developed. When tools for documenting proficiency are completed, they could be added to the handbook. A tool to evaluwww.jco.org ate communication of bad news, for example, is shown in Appendix Figure A1A and A1B (online only). It is important to train the evaluators in the process and content of evaluation tools. The faculty may not be familiar with the six competencies, especially if the faculty were trained before the competencies were introduced in the late 1990s. Feedback to faculty on their performance as evaluators can improve the quality of the evaluation and help fulfill the requirement for faculty development.19 The program can keep track of its evaluations by any systematic method. There are a number of electronic systems that programs or their GME parent offices can purchase to assist in the process of evaluation. Step 9: Research Some programs may choose to emphasize research. These programs must maintain the 18 months of clinical requirements for combined training in the 3-year program plus the continuity clinic experience or the 12 required clinical months for the single-specialty training plus the continuity clinic experience. These programs may choose to describe research in terms of tracks such as clinical research, basic laboratory research, or any other relevant hematology/oncology endeavor. No matter what is decided, the ACGME requires supervised scholarly activity for any trainee enrolled in a program of 24 months duration. The activity should be in blocks of protected time and should include teaching the trainee the elements of ethical conduct of research.4 Research can be funded by a training grant, by industry, or by other sources such as the ASCO Young Investigator Award.8,20 The ACGME requires that the majority of fellows demonstrate evidence of research productivity through publications or by presentations of abstracts at national specialty meetings. Training programs can enhance their trainees’ opportunity for success by formally assessing the trainees’ scholarly progress. A form to document progress is shown in Appendix Figure A2 (online only). Trainees who receive funding through NIH training grants must limit their clinical time during the period of the grant (see Step 2). Step 10: Didactic Lectures and/or Seminar Series A series of lectures or seminars must be part of any program.4 ASCO’s core curriculum can serve as an overall template for topics.10,11 Topics could also come from the blueprints of the ABIM certifying examinations in hematology and oncology.3 The topic list should cover the basic knowledge required for the specialty and could have guest faculty from the division, department, or other areas of the medical center. Fellows are encouraged to participate as lecturers. Having fellows give some of the program’s lectures will help them develop their proficiency as teachers and expert speakers; however, faculty attendance at such presentations is still expected by ACGME to ensure that accurate information is disseminated. A formal mechanism for providing feedback to the fellows giving lectures is helpful. A list of topics is shown in Appendix Table A1 (online only). The example shows how some topics can be appropriate to the learner in hematology and oncology whereas some topics are specific to either hematology or oncology. The example includes topics that are part of systems-based practice (Coding and Compliance Training) and communication (eg, The Psychiatric Issues Cancer Patients Face). The program can be creative as well and include topics such as “Medical Burn-Out” or “How to Give a Presentation.” A board review session can be useful and, in many programs, is run by the fellows. © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 7 Collichio et al Step 11: Monitoring and Adhering to the ACGME and ABIM Requirements Including the Program Information Forms and Surveys The ACGME reviews accredited programs every 2 to 5 years. This review is an assessment of whether the program complies with the ACGME guidelines by offering appropriate educational programs, adequate and varied patient volume, feedback mechanisms that are fair and timely, opportunity for research or other scholarly activities, a compassionate work environment with sufficient space, and adherence to work hour standards so that the trainee is able to learn effectively and provide quality care. A program that is constantly aware of the ACGME guidelines and works in a well-defined structure that is documented in an organized handbook (see Step 5) will find the review process to be relatively easy and will make the overall program more enjoyable for trainees, teachers, and support staff. The information that is required in the Program Information Form (PIF) can be documented in the handbook. The ACGME annual survey captures compliance with the regulations. It is important to review the survey with the trainees so that everyone is aware of what is offered in the program. Steps 1 through 10 allow the program to have an organized, smooth performance but attention to detail and to the dynamics of change are essential for efficient transitions from year to year. It is important to recognize that while the ACGME policies and procedures are designed to promote excellent education, the details of these policies and procedures may not be intuitively obvious to clinical faculty, including novice program directors. Program directors as well as program administrators should review the ACGME Web site on a periodic basis and consider attending the annual ACGME meeting.4 The ACGME requires that a program have key faculty and that they are qualified, dedicated, and scholarly. Board certification is the sine qua non of qualification for key faculty. Dedication is defined broadly as the faculty’s ability to provide an environment conducive to education, including faculty presence at case conferences, rounds, and journal club in addition to traditional patient care supervision and lecturing. Key faculty are defined more narrowly as those who dedicate, on average, 10 hours per week throughout the year to the training program. For programs with more than five fellows, a ratio of key clinical faculty to fellows of at least 1:1.5 must be maintained. It can be challenging to account for the exact teaching time of faculty whose involvement with fellows may change from day to day and month to month. Appendix Figure A3 (online only) shows a method for accounting for teaching time. The ACGME also requires that a majority of the faculty engage in scholarship, such as peer review funding or publication of original research, review articles, editorials, case reports, or chapters in textbooks.19 The regulations are quite specific about scholarly expectations, and the program director should periodically monitor the faculty’s productivity. Step 12: Personnel Issues A program director’s major challenge—and one for which there is usually inadequate training—is that of dealing with human beings. Fellows are people and thus are at risk for generating human problems. Often the program director is expected to fix things. Unexpected illness can disrupt a well-planned clinical schedule. A fellow’s minor run-in with a nurse or faculty member can take significant time to resolve. Despite the best intentions of the program and the program 8 © 2010 by American Society of Clinical Oncology director, there can be instances of the problem fellow who is not able to keep up with the requirements of the program or is disruptive to the program in some other way. A detailed guide on how to deal with this type of problem is beyond the scope of this article, but it is important to include it in one of our steps, because every program director should be aware of the institution’s programs to help residents with the stress of training. Program administration guidelines should include formal grievance procedures and the appropriate personnel management process. Unfortunately, most program directors receive little, if any, training in this important area.21-23 The importance of documentation cannot be overstated. For example, if a nurse calls in to question the ability of a fellow one time, then there may not be a case to question the fellow. But, if there are several instances questioning academic or clinical ability and several different people are making these queries, then a case can be made to discuss and document the situation with the trainee. If the program director has to have a stressful face-to-face encounter with the trainee, consideration should be given to having a witness for both parties. In the clinical environment, patient safety comes first. If the behavior in question could adversely affect patient outcome, the hospital or clinical administration, in addition to the program itself, may get involved. Step 13: Keep Track of Trainees After They Leave A system for keeping up with trainees after they leave can not only bring good will when it is time to include them in the alumni events and seminars, but it can also be helpful when it is time to provide data on board certification status and the jobs or careers that fellows pursue after completing their fellowship. Some training grants require this information on prior trainees as part of the grant application process. The ACGME requires a summary by the program director of the trainee’s performance during the final period of education and verification that the trainee has sufficient competence to enter practice without direct supervision.4 In conclusion, training fellows in hematology and oncology is a complex process, not only because of the enormous amount of information and skills that must be mastered in these fields but also because of the complex environment of medical practice that requires proficiency and expertise in systems, communication, and professionalism. The Six Competencies of Graduate Medical Education developed by the ACGME help programs keep track of these areas in their training. Programs are required to adhere to these competencies and to document the processes by which they do so. The steps and forms in this article are intended to help each program understand and adhere to the requirements of fellowship training. Our hope is that organized, systematic approaches, as described here, will reduce programmatic and administrative tension and allow faculty and fellows to enjoy the fellowship experience. There is likely no greater gift we can give to our patients than a well prepared next generation of oncologists. AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST The author(s) indicated no potential conflicts of interest. AUTHOR CONTRIBUTIONS Conception and design: Frances A. Collichio, Timothy J. Moynihan, James A. Stewart JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Administrative support: Frances A. Collichio Collection and assembly of data: Frances A. Collichio, Michael P. Kosty, James A. Stewart Data analysis and interpretation: Frances A. Collichio, James A. Stewart REFERENCES 1. Batalden P, Leach D, Swing S, et al: General competencies and accreditation in graduate medical education. Health Aff (Millwood) 21:103-111, 2002 2. Accreditation Council for Graduate Medical Education: Resident Services, 2009. http://www .acgme.org/acWebsite/home/home.asp 3. The American Board of Internal Medicine: Take the exam—Your complete guide, 2009. http:// www.abim.org/exam/exams.aspx 4. Accreditation Council for Graduate Medical Education: Program Director Guide to the Common Program Requirements, Version 2.2, 2009. http:// www.acgme.org/acWebsite/navPages/commonpr_ documents/CompleteGuide_v2%20.pdf 5. Association of American Medical Colleges (AAMC), AAMC Division of Health Care Affairs: AAMC Survey of Housestaff Stipends, Benefits and Funding: Autumn 2007 Report. http://www.aamc .org/data/stipend/2007_hssreport.pdf 6. Covey AS, Friedlaender GE: Financing graduate medical education: Sorting out the confusion. J Bone Joint Surg Am 85-A:1594-1604, 2003 7. US Department of Health and Human Services: Office of Extramural Research, 2009. http:// grants1.nih.gov/grants/oer.htm 8. Nuckols TK, Bhattacharya J, Wolman DM, et al: Cost implications of reduced work hours and Manuscript writing: Frances A. Collichio, Michael P. Kosty, Timothy J. Moynihan, Thomas H. Davis, James A. Stewart Final approval of manuscript: Frances A. Collichio, Michael P. Kosty, Timothy J. Moynihan, Thomas H. Davis, James A. Stewart workloads for resident physicians. N Engl J Med 360:2202-2215, 2009 9. Accreditation Council for Graduate Medical Education: Policies and Procedures, 2009. http://www .acgme.org/acWebsite/about/ab_ACGMEPolicies Procedures.pdf 10. American Society of Clinical Oncology: Curriculum for Medical Oncology, 2009. http://www.asco .org/ASCOv2/Education⫹%26⫹Training/ASCO⫹ Program⫹Guidelines 11. Muss HB, Von Roenn J, Damon LE, et al: ACCO: ASCO core curriculum outline. J Clin Oncol 23:2049-2077, 2005 12. Van Dongen HP, Maislin G, Mullington JM, et al: The cumulative cost of additional wakefulness: Dose-response effects on neurobehavioral functions and sleep physiology from chronic sleep restriction and total sleep deprivation. Sleep 26: 117-126, 2003 13. Dawson D, Reid K: Fatigue, alcohol and performance impairment. Nature 388:235, 1997 14. Lockyer J: Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof 23:4-12, 2003 15. Collichio FA, Kayoumi KM, Hande KR, et al: Developing an in-training examination for fellows: The experience of the American Society of Clinical Oncology. J Clin Oncol 27:1706-1711, 2009 16. Accreditation Council for Graduate Medical Education, American Board of Medical Specialties: Toolbox of Assessment Methods: A Product of the Joint Initiative—ACGME Outcomes Project, Version 1.1, 2000. http://www.acgme.org/outcome/assess/ toolbox.pdf 17. American Society of Clinical Oncology: The Quality Oncology Practice Initiative, 2009. http:// qopi.asco.org/ 18. Veloski JJ, Fields SK, Boex JR, et al: Measuring professionalism: A review of studies with instruments reported in the literature between 1982 and 2002. Acad Med 80:366-370, 2005 19. Accreditation Council for Graduate Medical Education: Internal Medicine Program Requirements, 2009. http://www.acgme.org/acwebsite/rrc_140/140_ prindex.asp 20. ASCO Cancer Foundation, 2009. http://www .ascocancerfoundation.org/TACF/Grants/Grant⫹ Opportunities/Young⫹Investigator⫹Award 21. Minicucci RF, Lewis BF: Trouble in academia: Ten years of litigation in medical education. Acad Med 78:S13–S15, 2003 22. Helms LB, Helms CM: Forty years of litigation involving residents and their training: I. General programmatic issues. Acad Med 66:649-655, 1991 23. Papadakis MA, Teherani A, Banach MA, et al: Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med 353:26732682, 2005 ■ ■ ■ Acknowledgment The following people contributed to the tables and figures for this article: P. Godley, N. Hayes (University of North Carolina), S. Gitlin (University of Michigan), L. Nabell (University of Alabama), and L. Lyckholm (Virginia Commonwealth University). T. Stinchcombe provided editorial support and Terri King provided administrative support (both from the University of North Carolina). www.jco.org © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 9 Collichio et al Appendix Table A1. Didactic Lectures Hematology/oncology Febrile neutropenia Acute leukemia How to do bone marrow biopsies Overview of pharmacology Symptom management How to communicate with your patients Chronic leukemias Bone marrow transplantation overview Multiple myeloma Intermediate- and high-grade non-Hodgkin’s lymphoma Hodgkin’s lymphoma Radiation oncology overview Coding and compliance training Computed tomography scanning Biology of normal cells and basic carcinogenics Statistics overview Oncology Breast cancer overview Breast cancer, metastatic disease Central nervous system overview Head and neck cancer overview The psychiatric issues cancer patients face Anal, esophageal, and gastric cancer Colorectal cancer Biliary, gallbladder, hepatocellular, and pancreatic cancer Melanoma overview Non–small-cell lung cancer overview Small-cell lung cancer/soft tissue sarcoma Oncology emergencies Bladder cancer Germ cell cancer Prostate cancer Renal cell cancer Genetics overview Gynecologic oncology Geriatric oncology Principles of surgical oncology Hematology Thrombocytopenia Hemophilia Myelodysplasia Hematologic pathology Sickle cell anemia overview Myeloproliferative disorders Thrombosis/thrombophilia Thallasemia and iron overload Hemochromatosis Clotting factor deficiencies and Von Willibrand disease Hemolytic anemia 10 © 2010 by American Society of Clinical Oncology JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Formative Evaluation of Giving Bad News Doctor_____________ Evaluator ______________ Date____________ Breaking bad news patient conversation check list Setting Eliminate pager / interruptions ___________________________________________________________________________ Not done Well done Appropriate introduction ___________________________________________________________________________ Not done Reviewed Appropriate interaction (Sit down, face patient, ask if others should be present etc) ___________________________________________________________________________ Rude Good interaction Appropriate language for patient ___________________________________________________________________________ Spoke in terms not understood by patient Adjusted to patient need Perception Elicits what patient knows ___________________________________________________________________________ Not done Well done Invitation Offers patient invitation ___________________________________________________________________________ Not done Offered Appropriate response if patient declines ___________________________________________________________________________ No Well done Fig A1. Evaluating delivery of bad news. (A) Page 1. (B) Page 2. www.jco.org © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 11 Collichio et al Knowledge Give news in short, easy to understand language ___________________________________________________________________________ Too technical Easy to understand Use of silence ___________________________________________________________________________ Kept talking Good use of silence Answer questions appropriately ___________________________________________________________________________ Not done Clear answers Emotions Identify emotions ___________________________________________________________________________ Not done Recognized Empathize ___________________________________________________________________________ Not done Empathetic Summary Formulate plan ___________________________________________________________________________ Not done Reviewed Summarize conversation ___________________________________________________________________________ Not done Reviewed Fig A1. continued 12 © 2010 by American Society of Clinical Oncology JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. Organizing a Fellowship Program and ACGME Competencies Hematology/OncologyFellowship Program Research Mentor’s Evaluation Form FacuIty Mentor’s Name: Evaluation Period: Trainee’s Name: This evaluation form attempts to document your impression of this trainee’s accomplishments and progress of acquiring knowledge, experience and skills to become an independent investigator. Poor Outstanding Quality of trainee’s project(s) Pace of progress of trainee’s project(s) Trainee’s acquisition of knowledge and skills related to conducting research project(s) Trainee’s progress to becoming able to be an investigator Quality of trainee’s presentation skills Quality of trainee’s writing skills Level of trainee’s desire to seek new knowledge and acquire new skills Ability of trainee to critically evaluate and assimilate data and to apply it to their research Ability of trainee to self-direct research project Trainee’s overall work ethic Overall assessment of trainee’s research skills development Average 1 1 2 2 3 3 4 4 5 5 1 2 3 4 5 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 1 2 3 4 5 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 1 2 3 4 5 CIRCLE ALL THAT APPLY Has trainee written any grant applications? Yes Submitted Has trainee been awarded any grant funding? Yes Details: Has trainee written any manuscripts related to research project? Yes Has trainee published any manuscripts related to research project(s)? In press Details: Yes No In Progress Planned No No In Progress Submitted No Has trainee submitted or presented any abstracts at extramural meetings/conferences? Submitted Poster Presentation Details: Platform Presentation Comments: Signature: Date: Fig A2. Hematology and oncology fellowship program research mentor’s evaluation. www.jco.org © 2010 by American Society of Clinical Oncology Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved. 13 Collichio et al Dr. A 2 20 20 4 8 6 6 2 6 0 3 Dr. D Average hrs/week hours Total Fellowship-Related # hrs/year Administrative Conferences (@ 1 hr ea) # hours Didactics + Mentoring (@20 hr/mo) # months Research Supervision 8 Dr. B Dr. C # hours/month Consult Supervision (@ 6 hr/wk) # weeks Continuity Clinic hr/wk) # weeks Wards (@40 32 hr/mo) # months Outpt Clinic (@ Name In this example the number of hours in each endeavor is multiplied by the number of weeks that it is done. Then, the total is divided by 52 weeks per year. 98 2 974 19 495 10 0 0 507 10 1241 24 Dr. E 3 9 0 4 0 3 Dr. Prog Dir 4 6 26 6 2 5 Dr. F 4 3 2 2 0 4 Dr. G 4 7 0 2 6 2 Dr. H 4 0 0 4 1 2 198 4 Dr. I 7 7 0 2 3 2 590 11 Dr. J 1 13 0 2 9 3 759 15 Dr. Asst Prog Dir 7 2 0 1 11 2 638 12 Dr. K 0 4 0 2 1 1 205 4 Dr. L 3 5 0 1 1 2 330 6 Dr. M 4 4 0 6 6 2 6 488 9 Dr. N 4 5 0 4 0 2 4 382 7 10 600 4 100 288 6 558 11 Dr. O 3 4 24 6 1 2 504 10 Dr. P 5 7 0 6 0 2 514 10 Dr. Q 4 0 0 4 0 3 179 3 Fig A3. Sample form for accounting for teaching time. 14 © 2010 by American Society of Clinical Oncology JOURNAL OF CLINICAL ONCOLOGY Downloaded from jco.ascopubs.org on September 19, 2014. For personal use only. No other uses without permission. Copyright © 2010 American Society of Clinical Oncology. All rights reserved.
© Copyright 2024