Measurements of the anisotropic properties of single crystals play a crucial role in probing the physics of new materials. Determining a growth protocol that yields suitable high-quality single crystals can be particularly challenging for multicomponent compounds. Here we present a case study of how we refined a procedure to grow single crystals of CaKFe4As4 from a high temperature, quaternary liquid solution rich in iron and arsenic (“FeAs self-flux”). Temperature dependent resistance and magnetization measurements are emphasized, in addition to the x-ray diffraction, to detect intergrown CaKFe4As4, CaFe2As2, and KFe2As2 within what appear to be single crystals. Guided by the rules of phase equilibria and these data, we adjusted growth parameters to suppress formation of the impurity phases. The resulting optimized procedure yielded phase-pure single crystals of CaKFe4As4. This optimization process offers insight into the growth of quaternary compounds and a glimpse of the four-component phase diagram in the pseudoternary FeAs−CaFe2As2−KFe2As2 system.