The UBC community has raised concerns about bias in Student Experience of Instruction (SEI) results over the years. Several reports about bias have been completed at UBC as below. Recently it has become possible to combine SEI data with data from the UBC Employment Equity Survey, once the response rate to the latter was high enough to make meaningful inferences from the analysis. The UBC Planning and Institutional Research (PAIR) office is preparing a report on such an analysis, to be presented to Senate committees and Senates when completed. This report will provide information about an aggregate analysis of data across UBC, and like any such analysis, it will not be able to fully reflect individual experiences.
In order to review this information a working group will be established by the Senate Teaching and Learning Committee at UBCV and the Senate Learning and Research Committee at UBCO. Its broad remit will be to consider analyses of SEI data focusing on demographic variables such as disability, gender, race, Indigenous identity, and their intersection with other known variables that explain variance in SEI data, such as course size, class level, and discipline. The group will also recommend steps to address results of such analyses. The group will also discuss how to surface and respond to individual faculty experiences with SEI survey results that can’t be captured in aggregate data analyses. Membership in the working group will include perspectives from faculty, students, and staff in relevant units, across both campuses. Membership will be diverse across a range of identities and disciplines, and also include some members with expertise in data analysis.
Objectives
- Discuss the upcoming demographic and additional contextual analysis of UBC SEI data and recommend next steps
- Advise on how to gather qualitative data on individual experiences with SEI results, such as through interviews, surveys, etc. Review results of such analyses and recommend next steps
- Discuss possibility of doing aggregate analyses of text comments on the surveys, including feasibility based on availability of tools to do so
- Advise on procedures for addressing discriminatory or otherwise harmful comments on SEI surveys, as well as comments that raise other concerns, such as student safety or disclosures of harms they have experienced themselves.
Previous Analyses of UBC SEI Data
- An investigation into the effects of instructor gender, field of study and student-respondent gender on UMI Scores in the 2008-2009 SEOT administration (2009) -- LINK TO BE ADDED SHORTLY -- considered binary sex only (Male or Female), given available data at the time
- The SEOT Working Group’s report to Senates in May of 2020 -- includes a summary of results of later studies on bias in UBC data related to binary sex on p. 48, with copies of two studies from UBCO on pp. 49-54.
Previous literature reviews
- Review of the variables that influence student evaluations of teaching (2013)
- The SEOT Working Group’s report to Senates in May of 2020 -- includes a literature review of peer-reviewed studies from 2013-2019 about bias in SEI results, on pp. 9-40.