We prove local asymptotic normality (resp. local asymptotic mixed normality) of a statistical experiment, when the observation is a positive-recurrent (resp. null-recurrent, with an additional technical assumption) Markov chain or Markov step process, under rather mild regularity assumptions on the transition kernel for Markov chains, and on the infinitesimal generator for Markov processes. The proof makes intensive use of Hellinger processes, thus avoiding almost completely to study the more complicated structure of the likelihoods themselves.