Internal Review Policy
Purpose
The Graduate Medical Education Committee (GMEC) is responsible for the development, implementation, and oversight of an internal review process for accredited Mary Hitchcock Memorial Hospital (MHMH) graduate medical education programs.
Subcommittee Charge
In fulfillment of the ACGME institutional accreditation requirement mandating oversight of the internal review process, the GMEC establishes an Internal Review Subcommittee to develop and implement an internal review policy; evaluate and approve program internal reviews; and monitor internal review action item improvement work plans. The chair will make periodic and timely reports of subcommittee deliberations and actions to the full body of the GMEC. The subcommittee will be comprised of a balanced mix of residency program directors, and residents, and may also include non-physician administrators. Members of the subcommittee are exempt from service on internal review committees during subcommittee tenure.
Process
A standard internal review is conducted by a committee designated by the GMEC to review each ACGME-accredited program in order to judge whether the program is in substantial compliance with the RRC Common Program Requirements; specialty-specific RRC Program Requirements; andthe ACGME Institutional Requirements
Each standard internal review committeemust include faculty and residents, and may include non-physicians. The review must follow the written protocol approved by the GMEC. The internal review closure date is the GMEC-identified mid-point. Closure of the internal review should occur within 60 days of the actual ACGME-identified mid-point date. The start and closure dates of program internal reviews must be documented in GMEC minutes.
When a program has no residents enrolled at the mid-point of the review cycle the GMEC will continue oversight through a modified internal review to ensure the program maintains adequate faculty and staff resources, clinical volume, and other necessary curricular elements required to remain in substantial compliance with the institutional, common and specialty-specific program requirements prior to the program enrolling a resident. After enrolling a resident, a standard internal review must be completed within the second six-month period of the resident's first year in the program.
While assessing the residency program's compliance with each of the ACGME RRC's program standards, both standard and modified reviews must appraise:
- Educational objectives of the program
- Effectiveness of the program in meeting its objectives
- Adequacy of available educational and financial resources to support the program
- Effectiveness of the program in addressing areas of noncompliance and concerns in previous ACGME accreditation letters and previous internal reviews
- Effectiveness of the program in defining the specific knowledge, skills, attitudes, and educational experiences required for the residents to achieve competence in the following: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice
- Effectiveness of the program in using evaluation tools developed to assess a resident's level of competence in each of the six general areas listed above
- Effectiveness of the program in using dependable outcome measures developed for each of the six general competencies listed above
- Effectiveness of the program in implementing a process that links educational outcomes with program improvement. This will include a review of the program's own annual review of aggregate resident performance, faculty development, graduate performance, and program quality.
- Effectiveness of the program's duty hour monitoring system.
- Written guidelines for and effectiveness of supervision of residents according to specialty requirements.
- Effectiveness of administrative systems in support of the educational program.
Materials and data to be used in the review process must include program documents as specified in the internal review protocol. In addition to reviewing program documentation the internal review committee is expected to review GME Office surveys of all current residents, faculty, and recent graduates. The committee is expected to interview the program director, a representative sample of faculty and peer-selected residents from each level of training in the program. Other staff within the clinical setting and other individuals from outside the program may also be deemed appropriate for interview by the committee.
Protocol
Staffing
The GMEC designates the Office of Graduate Medical Education (GME) to coordinate the conduct of regular internal reviews of MHMH residency programs to assess compliance with ACGME institutional requirements and RRC program requirements, to evaluate how effectively the programs are fulfilling their educational missions, and to report to the GMEC findings of the internal review.
Internal reviews will be conducted under the supervision of the Associate Dean for Graduate Medical Education (DIO). Whenever possible the DIO will chair internal review committees for short-cycled programs; other review committees will be chaired by program directors or faculty members from outside the department wherein the program under review resides. The DIO will appoint GME Office staff to support the work of each review committee.
Review Committee Membership
Committee members must be drawn from outside the program under review, and from outside the department wherein the program under review resides. Minimum committee membership is three people, including at least one program director or former program director and one resident. The committee may include non-physician administrators as deemed appropriate. External reviewers from outside MHMH and Dartmouth-Hitchcock Medical Center (DHMC) may also be included on the review committeeas determined by the DIO and GMEC.
Review Committee Size
Recognizing that a primary activity for committee members is to interview a program's faculty and current residents, the GMEC recommends an internal review committee be sized to adequately engage the program. An appropriate balance of faculty, residents, and any administrators must be maintained.
Review Committee Responsibilities
Specific duties for committee members include participating in a meeting to kick-off the process; reviewing program materials and data; interviewing faculty and residents; preparation of a written summary of the interviews for inclusion in the final internal review report; and providing feedback on the draft report.
Additional responsibilities for review committee chairsincludes providing leadership; approval of final wording in the draft report; and participation in the closure meeting. Review committee chairs are also invited to participate in the presentation of the review report to the GMEC subcommittee.
Program Information
As part of the internal review process the program and GME Office will collaboratively prepare a set of materials to document various aspects and elements of the training program. Materials will be electronically archived by the GME Office and made available to committee members for their review. Documentation will include:
- Internal Review Committee Roster
- Past Internal Review Reports and Work Plans
- Internal Review Surveys - Residents, Recent Graduates, and Faculty
- Interview Guidelines
- ACGME Resident Survey Results
- Resident, Recent Graduates, Faculty Rosters
- Program Description
- ACGME – RRC Accreditation Requirements
- RRC Accreditation Correspondence
- Program Policies
- Program Curriculum
- Program Schedules
- Evaluation Tools, Schedules, Outcomes
- Resident Logs
- In-Service Training Examination Results
- Board Completion and Pass Rates
- Annual Program Evaluation Minutes
- Resident Duty Hours
- Key Faculty CVs
- Affiliation Agreements
- GME Policies and Procedures
From time to time the review committee may require other program documents to aid in the evaluation process.
File Audit
As part of the review process the GME Office will conduct an audit of program files to ensure they meet standards. Findings will be included in the final internal review report.
Internal Review Report
For both standard and modified internal reviews there must be a written report that contains, at a minimum, the following:
- Name of the program reviewed, the date of the ACGME-assigned midpoint; the date of the GMEC-assigned midpoint; the dates of the review; closure date; and date of review and approval of the report by the GMEC Internal Review Subcommittee
- Names and titles of the internal review committee members including identification of residents and indication of PGY level
- Brief description of the internal review process used including who was interviewed (specific names will not be included in the final report to protect confidentiality, but will be maintained in the GME office for verification purposes) and the documents reviewed
- List of the areas of noncompliance or any concerns or comments from the previous ACGME accreditation letter and last internal review with a summary of how the program and/or institution addressed each one
- Sufficient documentation or discussion to demonstrate that a comprehensive review was conducted and was based on the GMEC's internal review protocol
- Identification of any areas of non-compliance or concerns identified as action items for internal follow-up and review by the GMEC
Closure
GME Office staff and the chair of the internal review committee will meet with the program director to share findings of the draft report and discuss next steps including presentation to the entire GMEC and any action item follow-up that may be indicated. The final report will be sent to the program director by the GME Office within 48 hours of the closure meeting.
Continuous Quality Improvement
A work plan addressing corrective measures to any action items identified in the internal review report must be submitted by the program director to the GME Office within 30 days of closure. Interim progress reports may be identified by the program or required by the GMEC. A final progress report will be made to the DIO approximately 6 months prior to the next anticipated RRC site visit. The Internal Review Subcommittee chair or their designee will provide work plan completion status reports to the GMEC.
Presentation to the GMEC
The GMEC's Internal Review Subcommittee will meet bi-monthly to review and approve all internal review reports. Program directors and the chair of their internal reviews are invited to participate in subcommittee discussions when reports are presented.
The subcommittee will report regularly to the entire GMEC on the approved reports; areas of significant concern and recommendations requiring immediate action; and examples of exemplary practices. Minutes of GMEC must reflect action taken on each internal review report.
Documentation for RRC
Upon GMEC acceptance of a program's report, the GME Office will provide the program with a letter confirming completion of the internal review process for verification by site visitors. A copy of this policy will be attached. The confirmation letter will not contain information from or conclusions drawn in the report other than the names and credentials of committee members.
Confidentiality
The internal review process is a quality assurance program evaluation that is protected pursuant to NH RSA 151 13a and RSA 329:29a. The GMEC supports confidentiality and accepts responsibility to keep secure and confidential the information collected about a program during and after the internal review process. A confidential cover page must always accompany reports related to the internal review process, indicating NH RSA 151 13a and RSA 329:29a quality assurance protection.
Internal review reports are confidential and must not be shared with RRC site visitors. To confirm compliance with this protocol and institutional requirements, internal review reports are included in the institutional review document. They are reviewed by the Institutional Review Committee at the time of institutional accreditation review to verify that we are following our internal review policy and protocol.
Sharing Results of Internal Review Report with Faculty and Residents
In order to complete the review process, the director should share the results of the review with all residents and faculty in the program. Discussion of the report and the action items should take place as part of the program's quality improvement process and should be integrated with faculty and resident evaluation of the program.
