The issues related to defect formation are extremely important, but as important is the ability to determine the quality requirements of a casting purchaser and supplier. Therefore, visual inspection of the surface quality is an imperative task conducted during the processing of steel castings. These inspections identify the occurrence of unacceptable casting surface defects, such as inclusions, porosity, burnt on sand, and flash. The castings that are marked for surface defects are then taken through a series of cleaning operations, where the marked defects are welded and/or ground to surface quality specifications. Recent studies show that currently there is no satisfactory method, whereby the surface quality requirements can be communicated throughout the manufacturing and purchasing phases of casting production. The result is that the description of casting surfaces is uncertain at best and impossible at worst. The lack of a reasonable measurement system for quality causes several implications, including uncontrolled processing times. Undetected surface defects during the visual inspection process will result in unacceptable quality standards and returns from the customer. Marking minor surface imperfections as defects will result in excessive rework. The current visual inspection methods used have never been subjected to a statistical study to determine their usefulness to the industry. This study looks at the problems with the current visual surface inspection standards used in the casting industry today. The goal of this study is to assess the amount of variation introduced by these inspections to determine the measurement error associated with this process. The objective is to develop a methodology to quantify the amount of variation in terms of repeatability (variation within the same operator) and reproducibility (variation between different operators) and apply it to the visual assessment data collected at three different steel foundries to draw beneficial conclusions.