Developing a proper model that could accurately represent highly nonlinear loads under various operating conditions is expected to be useful for power system planning, operation and online corrections. A novel approach of modeling this kind of loads such as an Electric Arc Furnace (EAF) is presented. First and second order Markov-like models are formulated based on the EAF time series in order to compare their effectiveness in the evaluation of the statistical stationarity of the process and the possibility of predicting the state variable such as arc current at least one step in advance. For the convenience of practical application, the models are further extended to a function space where a vector rather than a state or a point of the EAF current time series is treated as the basic element for modeling. In addition, several approximations for such a vector are investigated to reduce complexity of the problem and heavy burden of the computation. Based on both accuracy and efficiency indices, the results derived from FFT frequency decomposition method, when verified by actual EAF data, appears to have better performance than other approximations proposed and classic ARMA/Kalman Filtering approach. A simplified STATCOM is used to show arc current compensation based on the constructed Markov models. In this implementation, the harmonic components diminished and the power quality indices improved significantly after compensation. Thus the proposed Markov model is demonstrated to be effective in characterizing the dynamics of a seemingly chaotic load such as EAF as well as for real-time compensation of harmonics. The methodology can also be applied to other nonlinear time series.