## Stochastic Modeling and Simulation

Upon completing the module, the students master the basics of stochastic modelling and simulation. We first discuss discrete-time models, followed by two classic examples, and then continuous-time models.

**Contents**

Conditional probabilities, normal distributions, and scale-free distributions; Markov chains and their matrix representation, mixing times and Perron-Frobenius theory; Applications of Markov chains, such as the PageRank algorithm; Monte Carlo Methods: Convergence, Law of Large Numbers, Variance Reduction, Importance Sampling, Markov Chains Monte-Carlo Using Metropolis-Hastings & Gibbs Samplers; Random processes and Brownian motion: properties in 2, 3 and more dimensions, connection to the diffusion equation, Levy processes and anomalous diffusion; Stochastic differential equations (SDEs): Nonlinear transformations of Brownian motion (Ito calculus), Ornstein-Uhlenbeck process and other solvable equations; Examples from population dynamics, genetics, protein kinetics, etc.; Numerical simulation of SDEs: strong and weak error, Euler-Maruyama scheme, Milstein scheme.

**Program / Module**

M.Sc. Computational Modeling and Simulation

Module: CMS-COR-SAP - Stochastics and Probability

**Time/Place**

**Winter Term**

Lecture: Mondays, 4. DS (13:00-14:30) in HSZ-401 (Hörsaalzentrum) / **FIRST LECTURE: OCT 14**

Exercises / Tutorials: Thursday, 5.DS (14:50 - 16:20) in GÖR-229 (Görges-Bau)

**Format**

2 SWS lecture, 1 SWS exercise, 1 SWS tutorial, self-study

5 credits

**Exam**

**Wednesday, February 19, 2020, 09:20-10:50h, TRE/MATH/H (Trefftz-Bau, Haus A)**

**Since there are more than 10 registered students, module examination consists of a written examination with a duration of 90 minutes.**

At the exam, the following may be used:

- 4 A4 sheets (8 pages if you print duplex) of hand-written summary. We recommend writing the summary by hand, but it can also be machine-written. In the latter case, the font size must be 8 points or larger throughout.
- A standard pocket calculator (devices with network or bluetooth access, as well as devices capable of storing and displaying documents are not allowed)

**Exam Review**

You can come and look at your exam, and ask questions about its correction and the answers given during the following times:

- T.B.A.

All exam check session are going to happen at the CSBD (Pfotenhauerstr. 108) in the rooms of the Professorship.

**Registration to the course**

For students of the Master program "Computational Modeling and Simulation: via CampusNet SELMA

For students of the Computer Science programs: via jExam

Teachers

Lecture: Prof. Ivo F. Sbalzarini & Dr. Christoph Zechner

Exercises: David Gonzales

**Teaching language:** ENGLISH

Lecture notes are available as PDF here.

Below is the weekly syllabus and the exercise/solution handouts:

- Lecture 1 (Sbalzarini) - Probability refresher, conditional probabilities, Bayes' rule, random variables, discrete and continuous probability distributions, scale-free distributions (Exercise 01 PDF, Solution 01 PDF, Solution Jupyter Notebook)
- Lecture 2 (Sbalzarini) - transformation of random variables, pseudo- and quasi-random numbers, low discrepancy sequences, transformation algorithms: inversion, Box-Muller, accept-reject method, composition-rejection method (Exercise 02 PDF, Solution 02 PDF, Solution Jupyter Notebook)
- Lecture 3 (Sbalzarini) - Discrete-time stochastic processes, discrete Markov chains and their matrix (Exercise 03 PDF, Solution 03 PDF, Solution Jupyter Notebook)
- Lecture 4 (Sbalzarini) - Law of large numbers, Monte Carlo methods, example: MC integration, importance sampling (Exercise 04 PDF, Solution 04 PDF, Solution Jupyter Notebook)
- Lecture 5 (Zechner) - Monitoring variance, variance reduction (Exercise 05 PDF, Solution 05 PDF, Solution Jupyter Notebook)
- Lecture 6 (Zechner) - Rao-Blackwell, Markov Chain Monte Carlo (MCMC), detailed balance, convergence criteria, acceleration methods (Exercise 06 PDF, Solution 06 PDF, Solution Jupyter Notebook)
- Lecture 7 (Zechner) - Classic MCMC samplers 1: Gibbs sampling (Exercise 07 PDF, Solution 07 PDF, Solution Jupyter Notebook)
- Lecture 8 (Zechner) - Classic MCMC samplers 2: Metropolis-Hastings, convergence dagnostics, stopping conditions (Exercise 08 PDF, Solution 08 PDF, Solution Jupyter Notebook)
- Lecture 9 (Sbalzarini) - Monte-Carlo optimization: stochastic gradient descent, simulated annealing, evolution strategies, CMA-ES (Exercise 09 PDF, Solution 09 PDF, Solution Jupyter Notebook)
- Lecture 10 (Zechner) - Random Walks, Brownian motion in 1,2,3,n-dim, connection to diffusion, continuum limit of random walks (Exercise 10 PDF, Solution 10 PDF, Solution Jupyter Notebook)
- Lecture 11 (Zechner) - Stochastic calculus, Ito calculus, Ornstein-Uhlenbeck process (analytical) (Exercise 11 PDF, Solution 11 PDF, Solution Jupyter Notebook)
- Lecture 12 (Sbalzarini) - numerical methods for SDE: Euler-Maruyama, Milstein, strong and weak convergence (Exercise 12 PDF, Solution 12 PDF, Solution Jupyter Notebook)
- Lecture 13 (Zechner) - Master equation, Fokker-Planck, Kolmogorov forward, Example: chemical kinetics (Exercise 13 PDF, Solution 13 PDF)
- Lecture 14 (Sbalzarini) - Graphical representation and classification of reaction networks, exact stochastic simulation algorithms: first-reaction method, direct method (Exercise 14 PDF, Solution 14 PDF, Solution Jupyter Notebook)

**Suggested Literature**

Feller - an introduction to probability theory and its applications, Wiley+Sons, 1957.

Robert & Casella - Monte Carlo statistical methods, Springer, 2004.