Consider a multiple-input multiple-output (MIMO) downlink multi-user channel. A well-studied problem in such a system is the design of linear beamformers for power minimization with the quality of service (QoS) constraints. The most representative algorithms for solving this class of problems are the so-called minimum mean square error (MMSE)-second-order cone programming (SOCP) algorithm [Visotksy and Madhow, “Optimum Beamforming Using Transmit Antenna Arrays,” Proc. IEEE Veh. Technol. Conf., May 1999, vol. 1, pp. 851-856] , [Wong, Zheng, and Ng, “Convergence Analysis of Downlink MIMO Antenna System Using Second-Order Cone Programming,” Proc. 62nd IEEE Veh. Technol. Conf., Sep. 2005, pp. 492-496] and the uplink-downlink duality (UDD) algorithm [Codreanu, Tolli, Juntti, and Latva-Aho, “Joint Design of Tx-Rx Beamformers in MIMO Downlink Channel,” IEEE Trans. Signal Process., vol. 55, no. 9, pp. 4639-4655, Sep. 2007]. The former is based on alternating optimization of the transmit and receive beamformers; while the latter is based on the well-known uplink-dowlink duality theory. Despite their wide applicability, the convergence to Karush-Kuhn-Tucker (KKT) solutions of both algorithms is still open in the literature. In this paper, we rigorously establish the convergence of these algorithms for QoS-constrained power minimization (QCPM) problem with both single stream and multiple streams per user cases. Key to our analysis is the development and analysis of a new MMSE-DUAL algorithm, which connects the MMSE-SOCP and the UDD algorithm. Our numerical experiments show that 1) all these algorithms can almost always reach points with the same objective value irrespective of initialization and 2) the MMSE-SOCP/MMSE-DUAL algorithm works well while the UDD algorithm may fail with an infeasible initialization.