There are no announcements



This course is part of the MAGIC core.


Markov processes in discrete and continuous time will be presented for elementary and general state spaces and for some jump & so called linear-Markov (semi-Markov) processes in continuous times. Standard stuff will include generators, Dynkin’s formula, ergodicity for finite state spaces, strong Markov, Feller & strong Feller properties. A bit more advanced material will include coupling & recurrence applied to convergence rates and to queueing & reliability systems.


Spring 2019 (Monday, January 21 to Friday, March 29)


  • Live lecture hours: 20
  • Recorded lecture hours: 0
  • Total advised study hours: 80


  • Sun 00:05 - 00:55
  • Sun 00:05 - 00:55


Some basic knowledge about Markov chains is highly desirable.


1. Stochastic processes. Definitions of a Markov process. 2. Examples: Random Walks. Generators. Chapman-Kolmogorov equations. 3. Dynkin’s identity. Stopping times, strong Markov property, Feller & strong Feller processes. 4. Irreducible Markov processes, ergodic theorem for finite state spaces and in general case. 5. Doob-Doeblin's and Markov-Dobrushin's conditions of ergodicity. 6. Positive & null recurrent, polynomially & exponentially recurrent Markov processes. 7. Coupling method, lemma about three random variables, application to convergence rates. 8. Applications to queueing: Erlang telephone systems, stationary regimes, convergence rates. 9. Piecewise-linear Markov processes, extended Erlang formulae & convergence rates. 10. Applications to some reliability theory problems.


Alexander Veretennikov
Phone (0113) 3435183
Photo of Alexander Veretennikov


Stochastic Processes and ModelsStirzaker
Markov Processes: Characterization and ConvergenceEthier and Kurtz
A course in the theory of stochastic processesVent︠t︡selʹ (Wentzell)
Markov Models & OptimizationDavis
Ergodic Markov processes and Poisson equations (Lecture notes)A. Yu. Veretennikov


Clicking on the link for a book will take you to the relevant Google Book Search page. You may be able to preview the book there. On the right hand side you will see links to places where you can buy the book. There is also link marked 'Find this book in a library'. This sometimes works well, but not always. (You will need to enter your location, but it will be saved after you do that for the first time.)


No assessment information is available yet.

No assignments have been set for this course.


No files have yet been uploaded for this course.

Recorded Lectures

Please log in to view lecture recordings.