Evaluator

Job Summary

The Evaluation Team within the School of Medicine's Office of Assessment, Evaluation, and Outcomes (AEO) is responsible for the design, implementation, analysis, and reporting of all course, faculty, and program evaluation for Undergraduate Medical Education (UME). The Evaluator applies both project management and data-focused processes to evaluation activities, including longitudinal reporting and continuous quality improvement. This position assists with the design and distribution of survey instruments, data collection and analysis, and results reporting to support curricular improvement, LCME accreditation, faculty promotion, and faculty research, all while maintaining the confidentiality of sensitive data.

Key Responsibilities

Data Collection - 35%

• Communicates with faculty, staff, and IT professionals to answer questions about the evaluation process and resolve issues that arise during data collection.
• Designs and implements surveys to adhere to best practices for data collection including random sampling to reduce survey burden.
• Distributes evaluations to students and tracks progress of submissions to ensure sample sufficiency and representativeness.
• Implements timelines for releasing, collecting, and managing course and faculty evaluations.
• Reviews all open-ended survey feedback and determines whether any comments indicate potential issues to escalate to the AEO Manager.

Quantitative Data Analysis and Reporting - 25%

• Analyzes, summarizes, and reports evaluation data to match the data type and stakeholder needs, ensuring accurate results and the confidentiality of survey respondents.
• Designs, builds, and maintains Tableau Dashboards to report evaluation results for course and program leadership, both longitudinally within and across courses.
• Create and distribute summary reports for program and course directors, distribute individual reports to faculty, residents, and other educators, and create individual reports for upload to PRiSM accounts of faculty educators.

Qualitative Data Analysis and Reporting - 35%

• Collects, interprets, and synthesizes course and curricular content area feedback from various sources (e.g., students, staff, faculty, and historical documents) to create thematic analyses used to prompt and facilitate conversation during continuous quality improvement meetings for LCME accreditation.
• Conducts thematic analyses of open-ended feedback from course evaluations.
• Facilitates and summarizes results from focus groups.

Ad-hoc Data Requests - 5%

• Responds to requests for ad-hoc reports, assessing the feasibility from both logistical and student confidentiality standpoints, and providing alternate reporting possibilities when necessary.
• Uses Microsoft SQL to generate reports based on stakeholder needs.