We investigate how amphiphilic Janus particle assembly structures, including clusters and striped two-dimensional (2D) crystals, are influenced by the addition of surfactant molecules. Janus particles are fabricated using silica particles coated with Au on one side, which is further modified with a hydrophobic self-assembled monolayer. Analysis on the cluster assembly structures suggests that in addition to hydrophobic attraction, van der Waals (VDW) attraction plays a significant role in the assembly process, which is modulated by the Au coating thickness. This is manifested by the cluster formation induced primarily by VDW forces when the hydrophobic attraction between particles is diminished by adding the surfactant. In the 2D crystal case, sodium dodecyl sulfate (SDS) and Tween 20 show opposite trends in how they affect assembly structures and particle dynamics. SDS shortens the stripes in 2D crystals and accelerates the rotation of particles, whereas Tween 20 extends the straight stripes and slows down the particle rotation. We interpret the results by considering SDS adsorption on the Au-coated hemisphere of the Janus particles and Tween 20 forming hydrogen bonds with the silica hemisphere of Janus particles. Our study offers a simple approach to change the assembly structures of Janus particles, and it provides principles and guidance for potential applications of Janus particles coupled with small amphiphilic molecules.