ARBITRATION CASE- THE RYERSON DECISION

Many professors dread anonymous Student Evaluations of Teaching (SETs) or Faculty Course Surveys (FCS) as less than stellar assessments could mean the end of their employment or a serious roadblock on their career path. Tenure and promotion applications are most often concerned with two main criteria: scholarship and teaching effectiveness. SETS or FCS are often considered as part of the assessment of an academic's teaching.

This then begs the question of how 'teaching effectiveness' can be accurately assessed, in particular, by students' opinions.

In 2018, the use of SETs was challenged in an arbitration between Ryerson university and its Faculty Association, over which Arbitrator William Kaplan presided.

The Faculty Association and Ryerson Administrators had been at odds in bargaining over the issue since 2003. Facing an impasse, the two sides agreed to the creation of a joint committee and an ongoing pilot project to address concerns about the surveys. Unable to resolve the matter, the Faculty Association filed a first grievance in 2009, then a second in 2015. The matter went to an unsuccessful mediation and proceeded to an arbitration hearing in Toronto in April, 2018.. RYERSON ADMINISTRATORS' POSITION Ryerson argued that although student surveys were not solely determinative of the teaching effectiveness of a faculty member, the questionnaires did allow common issues and concerns to be identified alongside the other methods of evaluation. In addition, Ryerson felt that changes to evaluative tools should be gradual and left to the internal workings of the University to figure out.

FACULTY POSITION Faculty members had been expressing concerns about the use of student survey data for at least 15 years prior to Kaplan issuing his decision. The Faculty's position was that the use of scoring averages was ineffective and inaccurate because student surveys failed to provide reliable data. They alleged a significant bias in many of the surveys and even possible violations of the Human Rights Code. Ultimately, they believed that student evaluations had no place in the evaluation of teaching effectiveness.

THE AWARD That award/decision meted out by Kaplan, resulted in Ryerson losing the ability to use student evaluations as evidence of a professor's effectiveness (or lack thereof) in the classroom. The arbitration award rendered in June 2018, generated much interest both in Canada and internationally.

Kaplan weighed the strengths and weaknesses of Ryerson's student survey system in arriving at his conclusion. In doing so, he relied heavily on the expert evidence of Professors Philip Stark and Richard Freishtat of UC Berkeley. Stark and Freishtat's evidence was that student surveys were biased based on an array of immutable personal characteristics including race, gender, accent, age and even a professor's attractiveness. This evidence led Kaplan to conclude that Ryerson's student surveys were "imperfect at best and downright biased and unreliable at worst."

In his decision, Kaplan noted that student opinion surveys may still have value since they are the main source of information from students about their educational experience. SET results have a role to play in providing data about many things such as the instructors ability to clearly communicate, missed classes made up, assignments promptly returned, the students enjoyment and experience of the class, and its difficulty or ease, not to mention overall engagement. But he cautioned that the data should be carefully contextualized and that the strengths and weaknesses of the SET need to be fully understood.

IMPACTS University administrations are starting to pay attention. In May, the University of Southern California announced it would stop using student evaluations of teaching in promotion decisions and use the peer-review model instead. The university will still use student evaluations to provide feedback about students learning experiences and to give context, but not as a primary measure of teaching effectiveness during faculty review processes given their vulnerability to implicit bias and lack of validity as a teaching measure, wrote the vice-provost for academic and faculty affairs in a memorandum to the academic senate and faculty council chairs.

The University of Oregon is also planning to stop using numerical ratings from student course surveys in tenure and promotion reviews. The institution is working on a peer review framework. The goal is to increase equity and transparency in teaching evaluation for merit, contract renewal, promotion and tenure, while simultaneously providing tools for continual course improvement, according to the universitys website.

INSTRUCTIONS

a paper, minimum of 4 pages, single-space, outlining their position on the case.

Read the case above and use the Resources to gather more background information. Ensure your argument is rational and include supporting facts.

State your position - decide whether to Argue FOR or AGAINST Arbitrator Kaplan's decision.

Supporting Points - was the arbitrator right, how right was he within the parameters of the the Labour Law, was the University right in their argument, was the Faculty Association right, what has been YOUR personal experience with surveys about your courses, can students be unbiased in their feedback, if you were a faculty member what would you suggest to improve or include students' voices.

Conclusion must include what would you recommend and why?

sub headings - Introduction/your position/supporting points/Conclusion