Degree Type

Dissertation

Date of Award

1983

Degree Name

Doctor of Philosophy

Department

Psychology

Abstract

Two studies compared Huynh (1976) and Subkoviak (1976) estimation procedures for indices of classification consistency, rho and kappa, to be used when only one form of a test is administered. Rho indicates the degree of consistency and kappa indicates the degree of consistency beyond chance. Both studies manipulated test length (25, 50, 75 items), distribution shape (normal and two left-skewed) and passing standards (70%, 80%, 90% of items correct). Study I used simulated data; Study II used data from actual test administrations;Both Huynh and Subkoviak procedures yielded rho estimates of similar magnitude and acceptably high as reliability coefficients. Subkoviak kappa estimates were consistently higher than Huynh kappa estimates. No differences in behavior of coefficients was seen between Studies I and II. Test length affected all estimates, increasing in magnitude as test length increased. Magnitudes of rho coefficients decreased as skewness increased; kappa was at its lowest with less skewed distributions, but no consistent pattern of magnitude and skewness interaction was apparent. The effect of distribution shape was a function of the proximity between standard and distribution mode: rho was at its lowest value and kappa at its highest value when the standard neared the mode;Difficulties with interpretation of kappa coefficients are discussed. Suggestions regarding the practical use of and future research directions for one-form rho and kappa estimates are made;References;Huynh, H. On the reliability of decisions in domain-referenced testing. Journal of Educational Measurement, 1976, 13, 253-264. Subkoviak, M. J. Estimating reliability from a single administration of a criterion-referenced test. Journal of Educational Measurement, 1976, 13, 265-276.

DOI

https://doi.org/10.31274/rtd-180813-8572

Publisher

Digital Repository @ Iowa State University, http://lib.dr.iastate.edu/

Copyright Owner

Randi Louise Hagen

Language

en

Proquest ID

AAI8407073

File Format

application/pdf

File Size

113 pages

Share

COinS