Evaluation of Teaching Effectiveness Implementation Task Force
AY 2018-2019

Preface:  

UTA is home to outstanding faculty and student scholars across its many disciplines. As well, we count among our faculty 60 University of Texas System’s Regent’s Outstanding Teaching Award recipients since 2009, 54 current members of the UTA Academy of Distinguished Teachers, and numerous other faculty recognized for their outstanding accomplishments in teaching from other universities and professional organizations.

Effective teaching is determined by its effect on student achievement and learning. It involves using a set of evidence-based instructional practices that actively engage learners in the acquisition, maintenance, and application of knowledge.

Common themes around effective instructional practices used by effective teachers include:

• commitment to developing a deep knowledge of their content area,
• consistent communication of high expectations for each learner,
• practice of using feedback and reflection to continuously improve teaching effectiveness and learner outcomes,
• collaboration with other professionals to ensure student success,
• use of research-based (content and instructional) practices for instruction, and
• respect for the diverse knowledge, skills, and contributions each student brings to the classroom.

I. Why document teaching effectiveness?: 

“The University community believes that excellence in teaching and excellence in research go hand in hand, and as a matter of policy teaching and research are both essential duties of every faculty member. Promotion depends upon the demonstration of excellence in both areas.

The essential question in the evaluation of teaching is whether the candidate contributes in an effective, creative, and appropriate way to the teaching mission of the department. Attention should be paid to the varieties of demands placed on instructors and the range of teaching activities called for in various disciplines and at various levels. It is imperative that clear documentation of ability in teaching be included in all advancement and promotion cases. Incomplete advancement or promotion cases will be returned to the originating department. While no two cases will be alike, there are several recurring themes which may be addressed in the preparation of the teaching component and several useful techniques for verifying performance in these areas.”

– 1987 Policy for Evaluation of Teaching (for Advancement and Promotion)

II. Background: 

The educational literature includes substantial research which finds little association between student course evaluation results, student learning outcomes and teaching performance.  Multiple studies report that the favorability of course evaluation results vary with course rigor, instructor protected class characteristics, and course section contextual characteristics.  Studies of course rigor typically find increased rigor is associated with less favorable course evaluation results.  Although some studies of protected class characteristics find null results, many report that less favorable course evaluation results are associated with increased instructor age, female faculty members, minority faculty members, and faculty members born outside of the United States.  Regarding contextual characteristics, studies often report less favorable course evaluation results for larger course sections and certain technical subjects.

Due to these problematic research findings, in October 2014, members of the Faculty Senate expressed concerns about the use of Student Feedback Survey results in personnel evaluations.  To address these concerns the university initiated a joint undertaking between the provost’s office and the Faculty Senate for the purpose of testing SFS results.  A special project committee[1] was appointed and charged with investigating the relationship between SFS results and student learning.  The committee was also charged with investigating whether SFS results vary according to: (1) course-section rigor, (2) instructor protected class characteristics, and (3) course-section contextual characteristics.

In January 2017, the committee submitted it written findings to the provost’s office and Faculty Senate.  The final report consisted of two volumes titled, “An Examination of Instructor-Related Student Feedback Survey Results: Volume 1: Purpose, Methodology, and Findings (pdf)”[2] and “An Examination of Instructor-Related Student Feedback Survey Results: Volume 2: Supplementary Information (pdf)”[3].  Analyses found no meaningful association between SFS results and student learning.  Tests also disclosed: (1) an inverse relationship between the favorability of SFS results and course-section rigor, (2) SFS results vary with instructor protected class characteristics, and (3) SFS results vary with course-section contextual characteristics.  Given these findings, the committee concluded that SFS results lack validity as a measure of instructional performance and that reliance on SFS results as a measure of instructional performance should be reconsidered.

Following the university faculty senate report examining Student Feedback Surveys (SFSs), on February 24th, 2018, Dr. Vistasp Karbhari, President of the University of Texas at Arlington, formed and charged the UTA Teaching Effectiveness Task Force[4] with identifying best practices and developing guiding principles and strategies for assessing teaching effectiveness on its campus.  Specifically noted was a need for holistic measure of teaching effectiveness that were beyond the traditional end-of-course Student Feedback Surveys.

Headed by Drs. Teresa Taber, Dean of the College of Education and Dr. A. Raymond Elliott, Associate Professor of Spanish Linguistics, the task for worked to identity key teaching effectiveness principles and strategies and offered faculty and graduate students engaged in teaching at UT-Arlington students a set of recommendations for the evaluation of teaching effectiveness and continuous improvement efforts in teaching.  A complete copy of the pdf of the Teaching Effectiveness Task Force report can be obtained by contacting CRTLE at crtle@uta.edu. As a result, the members of the task force identified best practices in the assessment of teaching effectiveness through a review of the research literature in addition to consulting with peer and aspirational institutions.  These practices offer a holistic approach to supplement the required SFS process.  Additional strategies were identified for instructors to measure the effectiveness of their own teaching, engage in continuous improvement efforts as instructors, and ultimately offer models for preparing new instructors to successfully teach our student body. On May 21st, 2018, co-chairs Tabor and Elliott had a follow-up debriefing meeting to discuss the final report, the results and recommendations of the task force with the President Vistasp Karbhari, Provost Lim, Dr. David Coursey, Chair of Faculty Senate, and Dr. Antoinette Sol, Vice Provost for Faculty Affairs.  Also discussed at the meeting was a plan to best implement the recommendations put forth by the task force across campus.

In September of 2018, President Karbhari formed another task force[5] charged with finding concrete ways in which the university can implement the recommendations proffered by the members of the Teaching Effectiveness Task Force.  The Evaluation of Teaching Effectiveness Implementation Task Force was formally charged by President Karbhari at the beginning of October 2018.

To complete its charge, the members of the Evaluation of Teaching Effectiveness Implementation Task Force began a series of meetings aimed at identifying specific ways in which teaching effectiveness can be measured in reliable and valid way.  The ideas and suggestions presented by the task force are not intended to supersede but rather complement the traditional SFSs that are currently used to assess, albeit ineffectively, teaching effectiveness and student learning at the University of Texas at Arlington.  This website is intended to provide graduate teaching assistants, tenured and tenure-track faculty members, full- and part-time instructors as well as administrators ideas for effectively evaluating teaching performance beyond the traditional student feedback surveys that are currently in force.

At the University of Texas at Arlington, we believe that numerous methods beyond SFSs should be available for measuring the effectiveness of teaching at the university level.  In addition, teaching effectiveness should be determined on the basis of multiple holistic measures over time.

III. Overview of Sections: 

1. Introduction
2. Weekly or Mid-Semester Evaluations
3. Reflective Self-Evaluations
4. Peer Evaluations
5. Teaching Portfolios, e-Portfolios and Teaching Dossiers
6. Documenting Continuous Improvement in Teaching
7. How to Encourage Students to Complete Course Evaluations and Provide Informative Responses?
8. Student Interviews and Exit Interviews
9. Assessing Teaching Performance
10. Customizing Student Feedback Surveys
11. Uniformity in Assessing Teaching Effectiveness: A Guide for Supervisors, Department Chairs and Administrators
12. List of References
13. Members of the Evaluation of Teaching Effectiveness Implementation Committee

---

[1] The Student Feedback Study Committee was led by Dr. Thomas Hall and consisted of the following faculty members: Courtney Cronley, Dan Cavanaugh, David Coursey, A. Raymond Elliott, and Robert Kunovich.

[2] Hall, Thomas, Courtney Cronley, Dan Cavanaugh, David Coursey, A. Raymond Elliott, and Robert Kunovich. (January 2017). “An Examination of Instructor-Related Student Feedback Survey Results: Volume 1: Purpose, Methodology, and Findings.” Report of the Special Project Committee Faculty Senate.  Arlington: The University of Texas at Arlington. Pages i-viii, 1-238.

[3] Hall, Thomas, Courtney Cronley, Dan Cavanaugh, David Coursey, A. Raymond Elliott, and Robert Kunovich. (January 2017). “An Examination of Instructor-Related Student Feedback Survey Results: Volume 2: Supplementary Information.” Report of the Special Project Committee Faculty Senate.  Arlington: The University of Texas at Arlington. Pages i-iii, 1-138.

[4] The Teaching Effectiveness Task Force was comprised of the following faculty members: Raymond Elliott, Co-Chair, Teresa Taber Doughty, Co-Chair, Thomas W Hall, Laura B. Cameron, Mary E. (Beth) Mancini, Maria Martinez-Cosio, Frank W Foss, Paul J Componation, Peggy L Semingson, Diane B Mitschke, Antoinette Sol, Pauline Hudel Smith, Karabi C Bezboruah, Prajal Mishra, Barbara A Shipman, Cari Elizabeth Dighton, Esmeralda Sutton, Katie S. Gosa.

[5] The Teaching Effectiveness Task Implementation Task Force was comprised of the following faculty members listed by sub-committee membership: 1) Mid-semester survey, Self- and Peer-evaluation & Inspire: Peggy Semingson, Laura Mydlarz, Diane Mitschke; 2) GTA Preparation and New Faculty Fellows Program: Ann Cavallo, Barbara Shipman, Frank Foss; 3) Teaching Portfolios/Teaching Dossiers/E-Portfolios: Pauline Hudel-Smith; Theresa Jorgensen; Karabi Bezboruah, and 4) Uniformity in Assessing Teaching effectiveness Campus-wide: A. Raymond Elliott, Chair; María Martínez-Cosio, Paul Componation, Antoinette Sol and Andrew Pagel.