Senate Bill 16-17-24
| Bill ID: | 16-17-24 |
| Name: | Student Course Evaluation Improvement Project |
| Proposed: | 01/19/2017 |
| Sponsor: | Benjamin Bederson, Associate Provost for Learning Initiatives, and Alice Donlan, TLTC Director of Research |
| Proposal: | Over ten years ago, the University of Maryland instituted student course evaluations on campus based on work from the May 2004 report to the Senate from the Task Force on Course Evaluations and Teaching (Senate document #02-03-39) which preceded Senate bill 10-11-06 . There were 4 primary purposes of these evaluations articulated in the 2005 Final Report : a. Formative evaluation: to provide diagnostic feedback to faculty for the improvement of teaching b. Summative evaluation: to provide one measure of teaching effectiveness for use in the APT and post tenure review processes and in annual productivity reviews c. Informative evaluation: to provide information to students for their use in the selection of courses and instructors d. Outcome evaluation: for the purposes of documenting student learning. The Task Force outlined several recommendations to aid in the pursuit of these four purposes, including a recommendation that the University have a university-wide requirement for student evaluations in all undergraduate and graduate courses. Then, in 2010, the SEC received a proposal requesting a review of the current processes for course evaluations and the APAS Committee was tasked with reviewing the course evaluation system and considering whether it was consistent with the intent of the earlier Senate actions. The resulting Senate bill #10-11-06 recommended a few changes to the course evaluations system, including continued oversight of the CourseEvalUM system by a shared governance body, the development of unit-specific questions, and renewed consideration of a few specific issues, including how to better meet student needs through the course evaluations, how to educate students on the importance of civility in responses, and what efforts need to be made to ensure that APT dossiers include diverse documentation of teaching effectiveness. While the first instantiation of course evaluations made considerable progress, future efforts can build off of these recommendations to incorporate them into practice. We believe more can be done to improve the content and process of course evaluations to make the process more useful to campus stakeholders. Three concerns make this proposal particularly timely. First, the current system asks a parallel set of questions for student viewing, and personnel decisions, doubling the length of the survey instead of using questions for multiple-purposes. Second, principal components analysis of current evaluation data has shown that the current questions measure one overarching factor of course satisfaction, as opposed to measuring multiple, theoretically-grounded education constructs as it was originally designed to do. Third, recent research has identified significant bias in most student course evaluations that disadvantage female, ethnic minority, and other groups of instructors. We propose a process to evaluate and revise the current questions and procedures for course evaluations. In particular, we recommend designing the course evaluation to measure four pillars of effective education that comes from the education scholarly literature: classroom climate, course content, teaching practices, and assessment. ? Classroom Climate: Is the classroom environment constructed by the instructor inclusive and supportive of learning? ? Course Content: Is the content up-to-date, appropriate for the level of the course, and relevant for learners? ? Teaching Practices: Does the instructor include evidence-based teaching practices, such as providing timely feedback, scaffolding new information on to prior knowledge, and incorporating active learning assignments? ? Assessment: Are the assessments of learning (e.g., tests, quizzes, graded assignments) valid metrics of learning outcomes? Structuring the evaluation around these constructs will more effectively address the four stated purposes of course evaluations. We also anticipate that asking students about concrete classroom activities and practices instead of ambiguous questions about course satisfaction will serve to reduce bias. We recommend that the group tasked with addressing this issue perform several activities by first consulting with multiple campus stakeholders (e.g., faculty, departmental and college leadership, students, student leaders, etc.) to understand current needs, frustrations, and points of satisfaction with the current evaluation process. We recommend working closely with the Teaching and Learning Transformation Center (that has performed a preliminary review of other Big 10 school practices and scholarship) as well as IRPA to improve the process of course evaluation. They should also evaluate the best practices of other institutions and the current scholarship on course evaluations. The group should make recommendations to revise the evaluation questions and processes based on what it learns about campus needs and evidence-based best practices. We would suggest that the committee should develop its recommendation through incremental and comparative studies, so that any changes are well understood before being broadly implemented. The University could enact an experimental process that might include, for example, including new and old questions in the same class to compare them directly. |
| Active? | No |
Status
| Status: | Completed |
| Completed On: | 10/30/2019 |
History
| Status: | Complete |
| Related Files: |
| Status: | Complete |
| Reviewer: | Senate |
| Received: | 2019-04-18 |
| Decision Date: | 2019-10-02 |
| Decision: | The Senate voted to approve the recommendations as amended. |
| Actions: | The Senate considered the report at its April 24 meeting. The meeting adjourned before the Senate concluded its debate, so the item was moved to the agenda of the October 2, 2019 meeting for consideration. On October 2, the Senate reviewed the recommendations as presented by the committee. The Senate reviewed and approved two amendments, one to add an additional construct related to student learning and one to add student perceptions of teaching effectiveness to a recommendation on the focus of the survey. The Senate voted to approve the recommendations as amended. |
| Next Step: | Presidential Approval |
| Related Files: |
| Status: | Complete |
| Reviewer: | Senate Executive Committee (SEC) |
| Received: | 2019-04-01 |
| Decision Date: | 2019-04-08 |
| Decision: | The SEC voted to place the report on the agenda of the April 24, 2019 Senate meeting for consideration. |
| Next Step: | Senate Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Academic Procedures & Standards (APAS) Committee |
| Received: | 2018-08-27 |
| Decision Date: | 2019-03-29 |
| Decision: | The APAS Committee voted to forward its recommendations to the SEC for consideration. |
| Actions: | In fall 2018, the subcommittee worked with representatives from the Teaching and Learning Transformation Center (TLTC) and the Office of Transformational Learning in the Smith School of Business to consider how course evaluation items are understood and interpreted by students. The subcommittee considered issues related to implementation of course evaluations for a variety of purposes. In December and January, the subcommittee drafted and refined its recommendations, which it reported on at the APAS Committee's February 8, 2019, meeting. The APAS Committee shared the draft recommendations with various stakeholders, and provided an overview at the Senate's March 6 meeting. It revised the recommendations in light of the feedback and voted to approve them at its meeting on March 29, 2019. |
| Next Step: | SEC Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Senate Executive Committee (SEC) |
| Received: | 2018-08-20 |
| Decision Date: | 2018-08-27 |
| Decision: | The SEC voted to approve the extension until March 30, 2019. |
| Next Step: | APAS Committee Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Academic Procedures & Standards (APAS) Committee |
| Received: | 2018-01-23 |
| Decision Date: | 2018-08-20 |
| Decision Due By: | 2018-05-01 |
| Decision: | The APAS Committee requested an extension on its charge. |
| Actions: | In early 2018, the subcommittee reviewed feedback from a Campus Assessment Working Group, and examined college- and department-specific items included in course evaluations. The subcommittee met with the Associate Provost for Learning Initiatives & Executive Director of the Teaching and Learning Transformation Center (TLTC) to review results of the TLTC's pilot program on mid-semester course evaluations. In late spring, 2018, the subcommittee met with a focus group of students to better understand what students would find valuable in course evaluation results. The subcommittee was unable to complete its work by the deadline and requested an extension from the SEC. |
| Next Step: | SEC Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Senate Executive Committee (SEC) |
| Received: | 2018-01-16 |
| Decision Date: | 2018-01-23 |
| Decision: | The SEC voted to approve the extension until May 1, 2018. |
| Next Step: | APAS Committee Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Academic Procedures & Standards (APAS) Committee |
| Received: | 2017-02-03 |
| Decision Date: | 2018-01-16 |
| Decision: | The APAS Committee requested an extension on its charge. |
| Actions: | The APAS Committee met with the proposers on February 10, 2017 to discuss their concerns and suggestions for the course evaluation system. The committee reviewed past Senate action on course evaluations, including the work of the 2005 Task Force on Course Evaluations and Teaching at its meeting in early March, 2017. On April 10, 2017, the committee met with representatives from the Office of Institutional Research, Planning, and Assessment (IRPA) and the Course Eval Advisory Committee. After discussion, the committee agreed that APAS should work with the Course Eval Advisory Committee to move forward on the charge. The committee voted unanimously to form a joint subcommittee with members from the APAS Committee and the Course Eval Advisory Committee to continue considering the charge. The subcommittee began its review in summer, 2017. It met with the Director of the Office of Transformational Learning in the Smith School of Business, the Associate Provost for Faculty Affairs, and the Associate Provost for Learning Initiatives & Executive Director of the Teaching and Learning Transformation Center (TLTC). The subcommittee met with the Student Affairs Committee and reviewed scholarly research on student evaluation of teaching in higher education. It considered preliminary results from a mid-semester evaluation pilot conducted by TLTC, and met with the Undergraduate Academic Programs Committee and the Council of Associate Deans for Faculty Affairs. The subcommittee was unable to complete its work by the original deadline and requested an extension from the SEC. |
| Next Step: | SEC Review |
| Related Files: |
| Status: | Complete |
| Reviewer: | Senate Executive Committee (SEC) |
| Received: | 2017-01-23 |
| Decision Date: | 2017-01-30 |
| Decision: | The SEC voted to charge the APAS Committee with review of the proposal. |
| Actions: | The SEC discussed the proposal at its January 30, 2017, meeting and voted to charge the APAS Committee with its review. |
| Next Step: | APAS Committee Review |
| Related Files: |