Lameness and other locomotor system dysfunctions are a major contributing factor to increased removal rates and reduced sow longevity currently observed in United States commercial breeding herds, particularly in younger parity sows. One possible solution to reduce culling and natural death due to locomotor system dysfunction within the breeding herd is implementing stringent selection criteria for feet and leg conformation. However, the success of any selection program depends on the accuracy and consistency of the individuals tasked with making selection decisions. The objectives for this study were to 1) identify and assess the phenotypic distribution for feet and leg conformation scores in replacement gilt populations when scored by persons with variable training levels, 2) estimate the repeatability for phenotypic scores measured by trained evaluators across multiple time points, 3) estimate the reproducibility for phenotypic scores measured by trained evaluators when compared with assessments measured by a gold-standard evaluator, and 4) estimate evaluator bias when comparing trained evaluators with the gold-standard evaluator. A total of 4,494 replacement gilts within a commercial production system were evaluated at approximately 22 weeks of age at 3 separate sites by 4 evaluators with varying evaluation experience, including a gold-standard evaluator, to evaluate interpersonal reproducibility and bias. Additionally, a subset of 1,492 replacement gilts were evaluated by the same evaluators at 17 weeks of age to evaluate intrapersonal repeatability. The replacement gilts were evaluated for ten structural traits, including front and rear leg conformation, front and rear pastern angle, front and rear foot position, rib shape, toe size differences, overall foot size, and locomotion score. Repeatability statistics evaluated the relationship between multiple measurements over time on the same replacement gilt by the same evaluator. Reproducibility statistics evaluated the relationship between evaluations on the same replacement gilt by multiple evaluators. The observed results demonstrated that there was inherent evaluator bias present between the trained evaluators and the gold-standard evaluator, resulting in over- or underestimation for the phenotypic expression for a trait. While there was inherent bias when evaluating traits, reproducibility statistics indicated that the evaluators observed the traits in a similar manner relative to the gold standard evaluator. Additionally, repeatability statistics within evaluators indicated that the evaluators observed the traits in a similar manner over time. Although there was exact agreement for both repeatability and reproducibility for a substantial amount of replacement gilts, in general, the agreement occurred in instances where an evaluator either over- or underestimated a particular trait by one level. These results indicate that individuals with varying evaluation experience are able to assign scores for a given trait in a similar manner. In conclusion, from a training perspective, an evaluator’s ability to assign the same score as others is not the most critical part when making gilt selection decisions, rather, it is imperative that they recognize the phenotypic trait expression that is not compatible with longevity in the breeding herd and remove those animals from the replacement gilt pool.