Reliability optimization originally developed for hardware systems is extended to incorporate software into an integrated system reliability optimization. This hardware-software reliability optimization problem is formulated into a mixed-integer programming problem. The integer variables are the number of redundancies, while the real variables are the components reliabilities;To search a common framework under which hardware systems and software systems can be combined, a review and classification of existing software reliability models is conducted. A software redundancy model with common-cause failure is developed to represent the objective function. This model includes hardware redundancy with independent failure as a special case. A software reliability-cost function is then derived based on a binomial-type software reliability model to represent the constraint function;Two techniques, the combination of heuristic redundancy method with sequential search method, and the Lagrange multiplier method with the branch-and-bound method, are proposed to solve this mixed-integer reliability optimization problem. The relative merits of four major heuristic redundancy methods and two sequential search methods are investigated through a simulation study. The results indicate that the sequential search method is a dominating factor of the combination method. Comparison of the two proposed mixed-integer programming techniques is also studied by solving two numerical problems, a series system with linear constraints and a bridge system with nonlinear constraints. The Lagrange multiplier method with the branch-and-bound method has been shown to be superior to all other existing methods in obtaining the optimal solution;Finally an illustration is performed for integrating software reliability model into systems reliability optimization.