Using Classroom Artifacts to Measure Instructional Practice in Middle School Science: A Two-State Field Test
Author(s): Hilda Borko, Brian M. Stecher, Felipe Martinez, Karin L. Kuffner, Dionne Barnes, Suzanne C. Arnold, Joi Spencer, Laura Creighton, and Mary Lou Gilbert
Abstract:
This report presents findings from two investigations of the use of classroom artifacts to measure the presence of reform-oriented teaching practices in middle-school science classes. It complements previous research on the use of artifacts to describe reform-oriented teaching practices in mathematics. In both studies, ratings based on collections of artifacts assembled by teachers following directions in the “Scoop
Notebook” are compared to judgments based on other sources of information, including direct classroom observations and transcripts of discourse recorded during classroom observations. For this purpose, we developed descriptions of 11 dimensions of reform-oriented science instruction, and procedures for rating each on a dimension-specific five-point scale.
Two investigations were conducted. In 2004, data were collected from 39 middle school science teachers in two states. Each teacher completed a Scoop Notebook, each was observed by a singe observer on two or three occasions, and eight of the teachers were also audio-taped, allowing us to create transcripts of classroom discourse. In 2005, 21 middle-school mathematics teachers participated in a similar study, in which each teacher was observed by a pair of observers, but no audio-taping occurred.
All data sources were rated independently on each of 11 dimensions. In addition, independent ratings were made using combinations of data sources. The person who observed in a classroom also reviewed the Scoop Notebook and assigned a “gold standard” rating reflecting all the information available from the Notebook and the classroom observations. Combined ratings were also assigned based on the transcripts and notebooks, and based on the observations and transcripts.
The results of these field studies suggest that the Scoop Notebook is a reasonable tool for describing instructional practice in broad terms. For example, it could be useful for providing an indication of changes in instruction over time that occur as a result of program reform efforts. There was a moderate degree of correspondence between judgments of classroom practice based on the Scoop Notebook and judgments based on direct classroom observation. Correspondence was particularly high for dimensions that did not exhibit great variation from one day to the next. Furthermore, judgments based on the Scoop Notebook corresponded moderately well to our “gold standard” ratings, which included all the information we had about practice.
APA Citation:
Borko, H., Stecher, B. M., Martinez, F., Kuffner, K. L., Barnes, D., Arnold, S. C., et al. (2006). Using Classroom Artifacts to Measure Instructional Practice in Middle School Science: A Two-State Field Test (CSE Technical Report 690). Los Angeles: University of California, National Center for Research on Evaluation, Standards and Student Testing (CRESST).
Link: http://www.cse.ucla.edu/products/download_report.asp?r=690
Items in XPress
- Research for High-Quality Urban Teaching: Defining It, Developing It, Assessing It
- Problems with the use of student test scores to evaluate teachers
- Using Classroom Artifacts to Measure Instructional Practice in Middle School Science: A Two-State Field Test
- Overview of the Instructional Quality Assessment
