NDE modeling and simulation are important tools to support the development and validation of enhanced localization and characterization techniques. Previously, important achievements were made by the USAF to address crack detection in aircraft structures using angled-beam shear wave inspection techniques. However, new work on model benchmarking is needed to move beyond detection and achieve reliable crack characterization. To achieve this goal, simulated studies are needed to verify that models can accurately represent all of the key variables with the inspection of multilayer structures with fastener sites and varying crack conditions. Often with model benchmark studies, the accuracy of the model is evaluated based on the change in response relative to a selected reference signal. During recent simulated and experimental studies, some challenges were discovered concerning the creation and/or selection of a reference signal in a plate with a vertical hole and crack. The focus of this paper is on key findings concerning model benchmarking using CIVA-UT for angled-beam shear wave inspections. The use of a side drilled hole (SDH) in a plate was found to be somewhat problematic as a reference signal for angled beam shear wave inspection. Previously, only a limited number of studies have looked at model benchmarking for angled beam shear wave inspections. Systematic studies were performed with varying SDH depth and size, and varying the ultrasonic probe frequency, focal depth, and probe height. Care must be taken in understanding the precise beam properties with these experiments. One issue is that there is some increased error with the simulation of angled shear wave beams, especially in the near-field. Even more significant, asymmetry in real probes and the inherent sensitivity of signals in the near-field to subtle test conditions were found to provide a greater challenge with achieving model agreement. Through these studies, conditions of good and poor agreement were observed. For some inspection conditions, the skip signal off of the far wall from the side drilled hole can provide a better reference than the direct reflected signal. All in all, these seemingly mundane studies were found to be important with providing guidance on reference signal selection for model benchmarking work on the inspection of fastener sites with cracks.