Subject description
Discrete time Mmarkov chains: Random processes and Markov property. Markov chain theory. Connections to graph theory and linear algebra. Basic structure of a chain. Times of first passage ant first return. Recurrent and transient states. Infinitely many visits of a state. Ergodic behaviour of a chain. Limit theorems. Specific results for the case of finite number of states.
Continuous time Markov chains: Poisson flow and Poisson process.Continuous time markov chains: Poisson flow and Poisson process. Birth processes: solving Kolmogorov equations. Continuous time Markov property. Forward and backward Kolmogorov equations and their solutions. Stacionary distribution. Reverse approach. Stability and explosions. Diferential equations and generator of a one-parameter semigroup.
Applications of markov chains: Waiting queue systems (birth&death system, M/M/1, introduction into the general theory, some important cases of waiting queue systems). Monte Carlo markov chains (Bayesian statistics and Monte Carlo simulations, Gibbs sampler and Metropolis-Hastings algorithm, convergence of MCMC algorithms, applications in Financial Mathematics).
The subject is taught in programs
Objectives and competences
The course provides a certain number of probability themes that do not need deep theoretical knowledge. However they are important in view of applications. The emphasys is on ergodic theory, both in discrete and continuous time. Appliacations include waiting queue systems and MCMC methods.
Teaching and learning methods
Lectures, exercises, homeworks, consultations
Part of the pedagogical process will be carried out with the help of ICT technologies and the opportunities they offer.
Expected study results
The knowledge of some of the most important applications of probability is acquired.
Basic sources and literature
- G. Grimmett, D. Stirzaker: Probability and Random Processes, 3rd edition, Oxford Univ. Press, Oxford, 2001.
- D. Williams: Probability with Martingales, Cambridge Univ. Press, Cambridge, 1995.
- L. C. G. Rogers, D. Williams: Diffusions, Markov Processes, and Martingales I : Foundations, 2nd edition, Cambridge Univ. Press, Cambridge, 2000.
- J. R. Norris: Markov Chains, Cambridge Univ. Press, Cambridge, 1999.
- S. I. Resnick: Adventures in Stochastic Processes, Birkhäuser, Boston, 1992.
Stay up to date
University of Ljubljana, Faculty of Electrical Engineering Tržaška cesta 25, 1000 Ljubljana