The Construct and Discriminant Validity of Assessing College Teaching Quality and Academic Rigor Using Observation, Survey, and Other Emerging Methods

The extensive program of observational research in k-12 education has generated countless theories and educational interventions. Yet, the study of college teaching and learning has only recently begun to utilize broad-scale observational approaches. Recent theories on the college teaching and learning process point to the importance of students’ prior knowledge and subject matter (via pedagogical content knowledge), as a basis for student learning. As theories on college teaching and learning continue to deepen, new methodological tools may provide richer data to support these highly contextualized theories. Toward this end, Dr. Campbell and her team have developed the College Educational Quality (CEQ) project, an observational protocol for college classrooms. This study focuses on the construct and methodological validation of the CEQ protocol by comparing CEQ observation ratings with ratings of syllabi, student work, and student and faculty survey using Multi-Trait Multi-Method modeling. The study also considers whether multiple raters or multiple time points during the semester yield the most valid observational results by using Generalizability Theory. The construct and discriminant validation of the CEQ observation protocol will provide a solid foundation for quantitative studies examining in-depth, context dependent college teaching and learning theories.,