We present detailed data from low-temperature magnetization, magnetoresistance, and specific heat measurements on single-crystal YbNiSi3 with the magnetic field applied along the easy magnetic axis, H∥b. An initially antiferromagnetic ground state changes into a field-stabilized metamagnetic phase at ∼16kOe (T→0). On further increase of the magnetic field, magnetic order is suppressed at ∼85kOe. No non-Fermi-liquid-like power law was observed in the resistivity in the vicinity of the critical field for T⩾0.4K. Heat capacity measurements suggest that the applied magnetic field splits the nearly degenerate crystal-electric-field levels that form the zero-field ground state of YbNiSi3. The functional behaviors of the resistivity and specific heat are discussed in comparison with those of the few other stoichiometric heavy fermion compounds with established field-induced quantum critical points.