The purpose of this thesis is to study, by using techniques of regenerative processes, the problem of estimation of the stationary density and the transition density of a real-valued recurrent Markov chain and the sojourn time distribution of a semi-Markov processes when the imbedded Markov chain is real-valued and recurrent. We propose kernel estimators for all these problems;Using the regeneration technique of Athreya and Ney and Nummelin, we are able to significantly reduce the strong hypotheses on the Markov chain such as Doeblin recurrence, stationarity, and mixing that were imposed in all the earlier works;In this thesis, we assume that the Markov chain satisfies a much weaker condition known as Harris recurrence. Our results hold for any initial distribution and we assume no mixing;In the first chapter we present basic definitions and review some known results necessary to our purpose. In the second chapter, by using techniques of regenerative processes, we prove consistency and asymptotic normality for a simple kernel estimator of the stationary density and of the transition density. In the third chapter, the results of chapter 2 are extended to a class of kernels satisfying some conditions. In the fourth chapter we present some results on the estimation of sojourn time distribution for semi-Markov processes. In chapters 2, 3, and 4 we also present some results from a simulation study.