## Stochastic Modeling and Simulation

Upon completing the module, the students master the basics of stochastic modelling and simulation. We first discuss discrete-time models, followed by two classic examples, and then continuous-time models.

**Contents**

Conditional probabilities, normal distributions, and scale-free distributions; Markov chains and their matrix representation, mixing times and Perron-Frobenius theory; Applications of Markov chains, such as the PageRank algorithm; Monte Carlo Methods: Convergence, Law of Large Numbers, Variance Reduction, Importance Sampling, Markov Chains Monte-Carlo Using Metropolis-Hastings & Gibbs Samplers; Random processes and Brownian motion: properties in 2, 3 and more dimensions, connection to the diffusion equation, Levy processes and anomalous diffusion; Stochastic differential equations (SDEs): Nonlinear transformations of Brownian motion (Ito calculus), Ornstein-Uhlenbeck process and other solvable equations; Examples from population dynamics, genetics, protein kinetics, etc.; Numerical simulation of SDEs: strong and weak error, Euler-Maruyama scheme, Milstein scheme.

**Program / Module**

M.Sc. Computational Modeling and Simulation

Module: CMS-COR-SAP - Stochastics and Probability

**Time/Place**

**Winter Term**

Lecture: Mondays, 4. DS (13:00-14:30) in APB-E008 (computer science building) / **FIRST LECTURE: OCT 8**

Exercises / Tutorials: Thursday, 5.DS (14:50 - 16:20) in APB-E009/U

**Format**

2 SWS lecture, 1 SWS exercise, 1 SWS tutorial, self-study

5 credits

**Exam**

If there are more than 10 registered students, the module examination consists of a written examination, with a duration of 90 minutes. If there are 10 or fewer registered students, it consists of an oral examination as an individual examination performance amounting to 30 minutes; this will be announced to the enrolled students at the end of the enrollment period.

**Registration to the course**

For students of the Master program "Computational Modeling and Simulation: via CampusNet SELMA

For students of the Computer Science programs: via jExam

Teachers

Lecture: Prof. Ivo F. Sbalzarini & Dr. Christoph Zechner

Exercises: David Gonzales

**Teaching language:** ENGLISH

Please find below the lecture syllabus and the handouts:

- Lecture 1 (Sbalzarini) - Probability refresher, conditional probabilities, Bayes' rule, random variables, discrete and continuous probability distributions, scale-free distributions (Lecture Notes PDF, Exercise 1 PDF, Solution 1 PDF)
- Lecture 2 (Sbalzarini) - transformation of random variables, pseudo- and quasi-random numbers, low discrepancy sequences, transformation algorithms: inversion, Box-Muller, accept-reject, composition-rejectio (Lecture Notes PDF, Exercise 2 PDF)
- Lecture 3 (Sbalzarini) - Stochastic processes, discrete Markov chains and their matrix
- Lecture 4 (Zechner) - Law of large numbers, Monte Carlo methods, example: MC integration, importance sampling
- Lecture 5 (Zechner) - Monitoring variance, variance reduction, Rao-Blackwell
- Lecture 6 (Zechner) - Markov Chain Monte Carlo (MCMC), detailed balance, convergence criteria, acceleration methods
- Lecture 7 (Zechner) - Classic MCMC samplers 1: Gibbs sampling
- Lecture 8 (Sbalzarini) - Classic MCMC samplers 2: Metropolis-Hastings, convergence dagnostics, stopping conditions
- Lecture 9 (Sbalzarini) - Monte-Carlo optimization: stochastic gradient descent, simulated annealing, stochastic approximation, evolution strategies, CMA-ES.
- Lecture 10 (Zechner) - Random Walks, Brownian motion in 1,2,3,n-dim, connection to diffusion, continuum limit of random walks
- Lecture 11 (Zechner) - Stochastic calculus, Ito calculus, Ornstein-Uhlenbeck process (analytical)
- Lecture 12 (Sbalzarini) - numerical methods for SDE: Euler-Maruyama, Milstein, strong and weak convergence
- Lecture 13 (Zechner) - Master equation, Fokker-Planck, Kolmogorov forward, Example: chemical kinetics
- Lecture 14 (Sbalzarini) - stochastic reaction network simulation: Gillespie SSA, partial propensity methods

**Suggested Literature**

Feller - an introduction to probability theory and its applications, Wiley+Sons, 1957.

Robert & Casella - Monte Carlo statistical methods, Springer, 2004.