View Bill 16-17-24

Senate Bill 16-17-24

Bill ID: 16-17-24
Name: Student Course Evaluation Improvement Project
Proposed: 01/19/2017
Sponsor: Benjamin Bederson & Alice Donlan
Proposal: Over ten years ago, the University of Maryland instituted student course evaluations on campus based on work from the May 2004 report to the Senate from the Task Force on Course Evaluations and Teaching (Senate document #02-03-39) which preceded Senate bill 10-11-06 . There were 4 primary purposes of these evaluations articulated in the 2005 Final Report :

a. Formative evaluation: to provide diagnostic feedback to faculty for the improvement of teaching

b. Summative evaluation: to provide one measure of teaching effectiveness for use in the APT and post tenure review processes and in annual productivity reviews

c. Informative evaluation: to provide information to students for their use in the selection of courses and instructors

d. Outcome evaluation: for the purposes of documenting student learning.

The Task Force outlined several recommendations to aid in the pursuit of these four purposes, including a recommendation that the University have a university-wide requirement for student evaluations in all undergraduate and graduate courses.
Then, in 2010, the SEC received a proposal requesting a review of the current processes for course evaluations and the APAS Committee was tasked with reviewing the course evaluation system and considering whether it was consistent with the intent of the earlier Senate actions. The resulting Senate bill #10-11-06 recommended a few changes to the course evaluations system, including continued oversight of the CourseEvalUM system by a shared governance body, the development of unit-specific questions, and renewed consideration of a few specific issues, including how to better meet student needs through the course evaluations, how to educate students on the importance of civility in responses, and what efforts need to be made to ensure that APT dossiers include diverse documentation of teaching effectiveness.

While the first instantiation of course evaluations made considerable progress, future efforts can build off of these recommendations to incorporate them into practice. We believe more can be done to improve the content and process of course evaluations to make the process more useful to campus stakeholders.

Three concerns make this proposal particularly timely. First, the current system asks a parallel set of questions for student viewing, and personnel decisions, doubling the length of the survey instead of using questions for multiple-purposes. Second, principal components analysis of current evaluation data has shown that the current questions measure one overarching factor of course satisfaction, as opposed to measuring multiple, theoretically-grounded education constructs as it was originally designed to do. Third, recent research has identified significant bias in most student course evaluations that disadvantage female, ethnic minority, and other groups of instructors.

We propose a process to evaluate and revise the current questions and procedures for course evaluations. In particular, we recommend designing the course evaluation to measure four pillars of effective education that comes from the education scholarly literature: classroom climate, course content, teaching practices, and assessment.

? Classroom Climate: Is the classroom environment constructed by the instructor inclusive and supportive of
learning?
? Course Content: Is the content up-to-date, appropriate for the level of the course, and relevant for learners?
? Teaching Practices: Does the instructor include evidence-based teaching practices, such as providing timely
feedback, scaffolding new information on to prior knowledge, and incorporating active learning assignments?
? Assessment: Are the assessments of learning (e.g., tests, quizzes, graded assignments) valid metrics of learning
outcomes?

Structuring the evaluation around these constructs will more effectively address the four stated purposes of course evaluations.

We also anticipate that asking students about concrete classroom activities and practices instead of ambiguous questions about course satisfaction will serve to reduce bias.

We recommend that the group tasked with addressing this issue perform several activities by first consulting with multiple campus stakeholders (e.g., faculty, departmental and college leadership, students, student leaders, etc.) to understand current needs, frustrations, and points of satisfaction with the current evaluation process. We recommend working closely with the Teaching and Learning Transformation Center (that has performed a preliminary review of other Big 10 school practices and scholarship) as well as IRPA to improve the process of course evaluation. They should also evaluate the best practices of other institutions and the current scholarship on course evaluations. The group should make recommendations to revise the evaluation questions and processes based on what it learns about campus needs and evidence-based best practices. We would suggest that the committee should develop its recommendation through incremental and comparative studies, so that any changes are well understood before being broadly implemented. The University could enact an experimental process
that might include, for example, including new and old questions in the same class to compare them directly.
Active?Yes

Status

Status: Under Review
Under Review By: Academic Procedures & Standards (APAS) Committee
Received: 02/03/2017
Decision Due By: 12/15/2017
Actions: The APAS Committee met with the proposers on February 10, 2017 to discuss their concerns and suggestions for the course evaluation system. The committee reviewed past Senate action on course evaluations, including the work of the 2005 Task Force on Course Evaluations and Teaching at its meeting in early March, 2017. On April 10, 2017, the committee met with representatives from the Office of Institutional Research, Planning, and Assessment (IRPA) and the Course Eval Advisory Committee. After discussion, the committee agreed that APAS should work with the Course Eval Advisory Committee to move forward on the charge. The committee voted unanimously to form a joint subcommittee with members from the APAS Committee and the Course Eval Advisory Committee to continue considering the charge. The subcommittee will begin its review in summer, 2017, and will make recommendations to the APAS Committee during the 2017-2018 academic year.
Next Step: SEC Review
Related Files:


History

Reviewed By: Senate Executive Committee (SEC)
Received: 01/23/2017
Decision Date: 01/30/2017
Decision: The SEC voted to charge the APAS Committee with review of the proposal.
Actions: The SEC discussed the proposal at its January 30, 2017, meeting and voted to charge the APAS Committee with its review.
Next Step: APAS Committee Review
Related Files: