This report is intended to extend previous work examining the reliability and validity of the Basic Skills and the Content-Analysis-Multiple Choice probes in two Iowa schools districts and to explore the use of the measures for monitoring student progress. One hundred thirty five students in grades nine to twelve participated in the study. Data were gathered from September 2005 to January 2006. Over four months of data collection, students completed two Basic Skills probes and two Content Analysis-Multiple Choice probes each month. We assessed the alternate form reliability and test-retest reliability of both types of probes. Our findings revealed that both types of probes possessed adequate levels of reliability with the Basic Skills probes demonstrating a higher level of reliability than the Content Analysis-Multiple Choice probes. To evaluate the validity of both types of probes, we gathered data from a variety of indicators of students’ proficiency in algebra including course grades, teachers’ evaluations of student proficiency and growth, and performance on standardized assessment instruments including the Iowa Test of Education Development (ITED) and the Iowa Algebra Aptitude Test (IAAT). We examined both concurrent and predictive validity. Our results revealed moderate correlations between both types of probes and other indicators of students’ proficiency in algebra. We examined whether the probes reflected student growth over time. The analyses revealed students grew .51 and .61 points each week on Basic Skills and Content Analysis- Multiple Choice probes, respectively. This result suggests that the Content Analysis-Multiple Choice probes may be slightly more sensitive in evaluating student growth. Finally, we were also interested in exploring whether students’ growth on the two types of probes was related to teachers’ rating of growth or gain scores on the IAAT. We found a small but significant correlation between teacher ratings of growth and both types of probes. No significant correlations existed between either type of probe and IAAT gain scores.
Perkmen, Serkan; Foegen, Anne; and Olson, Jeannette R., "Reliability, Criterion Validity and Sensitivity to Growth: Extending Work on Two Algebra Progress Monitoring Measures" (2006). Project AAIMS Technical Reports. 12.