In this dissertation, normal form (NF) theory is used to characterize and quantify nonlinear modal interaction near critical equilibria. The research focus is on the analysis of second-order modal interaction and the study of nonlinear aspects of system behavior of interest to the design and location of system controllers;A systematic approach to derive second-order NF representations in the neighborhood of equilibrium points is presented. Nonlinear interaction measures based on this model are then obtained to assess the extent and distribution of nonlinearity in the system. Finally, analytical criteria are developed to predict the existence of nonlinear modal interactions that significantly affect system dynamic performance;A nonlinear analysis framework based on normal form (NF) theory and center manifold reduction is proposed to most effectively select generating units which should be equipped with power system stabilizers (PSS). The effect of control action on nonlinear behavior is approximated via suitable modification of initial conditions in the nonlinear coordinate transformations that relate the physical system to the NF coordinates. Using this representation, nonlinear PSS sensitivity indices are then proposed to determine the optimum sites at which to locate PSS. The technique can predict aspects of a system's nonlinear behavior not obtainable from linear approaches and can therefore result in improved placement of system controllers;Test cases developed on standard test systems are presented to demonstrate the effect of nonlinear interaction and to estimate the controllers' effects on system dynamic performance. The efficacy and accuracy of the method is demonstrated through comparison with conventional analysis techniques.