Stochastics and Probability

Stochastic Modeling and Simulation

Upon completing the module, the students master the basics of stochastic modelling and simulation. We first discuss discrete-time models, followed by two classic examples, and then continuous-time models.


Conditional probabilities, normal distributions, and scale-free distributions; Markov chains and their matrix representation, mixing times and Perron-Frobenius theory; Applications of Markov chains, such as the PageRank algorithm; Monte Carlo Methods: Convergence, Law of Large Numbers, Variance Reduction, Importance Sampling, Markov Chains Monte-Carlo Using Metropolis-Hastings & Gibbs Samplers; Random processes and Brownian motion: properties in 2, 3 and more dimensions, connection to the diffusion equation, Levy processes and anomalous diffusion; Stochastic differential equations (SDEs): Nonlinear transformations of Brownian motion (Ito calculus), Ornstein-Uhlenbeck process and other solvable equations; Examples from population dynamics, genetics, protein kinetics, etc.; Numerical simulation of SDEs: strong and weak error, Euler-Maruyama scheme, Milstein scheme.

Program / Module

M.Sc. Computational Modeling and Simulation
Module: CMS-COR-SAP - Stochastics and Probability

Winter Term

Lecture: Mondays, 4. DS (13:00-14:30) in APB-E008 (computer science building) / FIRST LECTURE: OCT 8
Exercises / Tutorials: Thursday, 5.DS (14:50 - 16:20) in APB-E009/U


2 SWS lecture, 1 SWS exercise, 1 SWS tutorial, self-study

5 credits


If there are more than 10 registered students, the module examination consists of a written examination, with a duration of 90 minutes. If there are 10 or fewer registered students, it consists of an oral examination as an individual examination performance amounting to 30 minutes; this will be announced to the enrolled students at the end of the enrollment period.

Registration to the course

For students of the Master program "Computational Modeling and Simulation: via CampusNet SELMA

For students of the Computer Science programs: via jExam


Lecture: Prof. Ivo F. Sbalzarini & Dr. Christoph Zechner
Exercises: David Gonzales

Teaching language: ENGLISH

Please find below the lecture syllabus and the handouts:
  • Lecture 1 (Sbalzarini) - Probability refresher, conditional probabilities, Bayes' rule, random variables, discrete and continuous probability distributions, scale-free distributions (Lecture Notes PDF, Exercise 1 PDF, Solution 1 PDF)
  • Lecture 2 (Sbalzarini) - transformation of random variables, pseudo- and quasi-random numbers, low discrepancy sequences, transformation algorithms: inversion, Box-Muller, accept-reject, composition-rejectio (Lecture Notes PDF, Exercise 2 PDF)
  • Lecture 3 (Sbalzarini) - Stochastic processes, discrete Markov chains and their matrix
  • Lecture 4 (Zechner) - Law of large numbers, Monte Carlo methods, example: MC integration, importance sampling
  • Lecture 5 (Zechner) - Monitoring variance, variance reduction, Rao-Blackwell
  • Lecture 6 (Zechner) - Markov Chain Monte Carlo (MCMC), detailed balance, convergence criteria, acceleration methods
  • Lecture 7 (Zechner) - Classic MCMC samplers 1: Gibbs sampling
  • Lecture 8 (Sbalzarini) - Classic MCMC samplers 2: Metropolis-Hastings, convergence dagnostics, stopping conditions
  • Lecture 9 (Sbalzarini) - Monte-Carlo optimization: stochastic gradient descent, simulated annealing, stochastic approximation, evolution strategies, CMA-ES.
  • Lecture 10 (Zechner) - Random Walks, Brownian motion in 1,2,3,n-dim, connection to diffusion, continuum limit of random walks
  • Lecture 11 (Zechner) - Stochastic calculus, Ito calculus, Ornstein-Uhlenbeck process (analytical)
  • Lecture 12 (Sbalzarini) - numerical methods for SDE: Euler-Maruyama, Milstein, strong and weak convergence
  • Lecture 13 (Zechner) - Master equation, Fokker-Planck, Kolmogorov forward, Example: chemical kinetics
  • Lecture 14 (Sbalzarini) - stochastic reaction network simulation: Gillespie SSA, partial propensity methods
Suggested Literature

Feller - an introduction to probability theory and its applications, Wiley+Sons, 1957.
Robert & Casella - Monte Carlo statistical methods, Springer, 2004.