The main purpose of this research is to provide an in-depth evaluation of the effectiveness of distance learning programs by conducting a comprehensive investigation of the learners perspective of quality in the Masters of Science in Agronomy distance program at the Iowa State university. It is presented in a manuscript format as it includes three publishable journal articles that represent Chapters 2, 3 and 4 of a traditional thesis along with a General Introduction and Conclusion Chapters.

This thesis argues that current quality standards and evaluation models applied in distance learning are insufficient in addressing the learner’s perspective of quality in distance learning and do not provide prescriptive theory-based procedures for conducting program evaluations in distance learning.

Current evaluation practices primarily represent distance-learning providers’ perspective. That is, quality is a value-laden concept that has a different meaning for different groups of people (stakeholders) that are involved in distance learning, and past research is limited in terms of bringing learners’ point of view of quality to distance learning. Considering the importance of bringing learners’ point of view to the discussion of quality in distance learning, a critical review was conducted of the literature on quality standards in distance learning. Specifically, a review was performed to compile quality indicators recommended by distance learners from research in the field. Four themes emerged in the literature that indicates the learners’ perspective of quality in distance learning: (1) Course design, (2) Interaction, (3) Support and (4) Technology use and within each theme, several quality indicators were identified.

A Logic model of evaluation was developed as a management tool to guide and focus the evaluation research of the Master of Science in Agronomy distance program. Building on the results of the Logic model an outcome evaluation was performed in the Master of Science in Agronomy distance program to examine participants’ recollections of quality in the program after completing the final oral examination. In this study, the quality indicators identified in the literature review were compiled and examined to ascertain: (1) the degree to which the quality indicators proposed from the literature are actually present in the program, and (2) how important are the specific quality indicators to learners. A significance difference was found for an indicator that addresses a need for course materials to be updated.

Building on the results of the outcome evaluation, a longitudinal mixed method study was designed and conducted by using prescriptive and context-sensitive methodologies. Specifically, the purpose of the second evaluation study was to evaluate the effectiveness of the Master of Science in Agronomy distance program by conducting an output evaluation. It was investigated what distance learners perceive as quality in the distance learning program by designing context-sensitive methodologies that use existing data of internal students’ course and program evaluations. According to the graduate students’ perspective, the educational program under evaluation is an effective learning environment that provides a learner-centered, knowledge-centered, community-centered and assessment-centered environment to its learners. In addition, areas of strengths and areas for improvement were identified within the four effective learning environments.

Together, these three articles address quality issues in the current evaluation practices in distance learning by using the Logic model and prescriptive, systematic and theory-based evaluation designs for measuring quality in distance learning through the learners’ perspective. Moreover, the two program evaluation studies provide empirical evidence of learners’ perceptions of quality in distance learning in a higher educational setting.