This technical report summarizes the results of a study in which we examined the technical adequacy of three potential algebra measures for progress monitoring. One hundred thirteen students (18 of whom were receiving special education services) completed two forms of a Basic Skills probe, an Algebra Concepts probe, and a Content Analysis probe. In addition, we gathered data on criterion variables including grades, classroom assessment records, teacher ratings, and standardized test scores. We used multiple timing periods for the Basic Skills and Algebra Concepts probes to examine the efficacy of differing durations. We examined both test-retest and alternate form reliability for individual probes of all three types and for aggregated scores from two of the probe types. Criterion validity was examined using correlations between students’ probe scores and their scores from other indicators of algebra proficiency. The results of the study indicate that the Algebra Concepts probe is the most promising of the three measures investigated. It has adequate reliability and demonstrated the strongest correlations with the criterion measures. The Basic Skills probe had lower levels of reliability and more limited relations to the criterion measures (with the exception of the computation subtests of the standardized achievement tests). The Content Analysis probe had the highest levels of reliability among the three probes and moderate to moderately high correlations with several of the criterion measures. Concerns were identified about the difficulty of this probe because a large proportion of the students had scores of 10 or fewer points on the probe. Further research is needed to investigate more appropriate timing duration for the Basic Skills and Algebra Concepts probes. In the current study, the duration of the former was too short, while the duration of the latter was too long. The study should be replicated with additional, and more diverse student populations to determine the generalizability of the findings. Finally, subsequent research should examine the effects of routine progress monitoring on the measures’ stability and sensitivity to growth.
Foegen, Anne and Lind, Linda S., "Reliability and Criterion Validity for Three Potential Algebra Measures" (2004). Project AAIMS Technical Reports. 6.