Stochastik II – Stochastics II

Summer 2015




In a nutshell...

Starting: 14th April 2015

Place: Arnimallee 6

Lectures: Tuesday (SR 007/008) and Thursday (SR 025/026), 08:00-10:00

Problems: Wednesday (SR 025/026) 08:00-10:00 (Time and place changed!!!)

More fun: I will be available in my office at Tuesday 10:00-12:00. Of course meetings can be also arranged by appointment.

Prerequisites: a previous course on probability theory is required. Additionally, courses in analysis will be also desirable.

Synopsis: this course is the sequel of the course of Stochastik I. The main objective is to go beyond the first principles in probability theory by introducing the general language of measure theory, and the application of this framework in a wide variety of probabilistic scenarios. More precisely, the course will cover the following aspects of probability theory:

- Measure theory and the Lebesgue integral.
- Convergence of random variables and 0-1 laws.
- Generating functions: Branching processes and characteristic functions.
- Markov chains
- Introduction to martingales

Feel free to contact me at jrue at with any questions prior to the start of the course!

Requirements: the requirements of the course are here.

What we have seen so far...



Further Reading



Introduction: from Riemann to Lebesgue. Characteristic functions and step functions. Riemann integral. Motivation on why is needed to extend the notion of Riemann integral.

Sigma-algebras and measurable functions: definition of sigma-algebra, first observations and examples. Sigma-algebra generated by a set and Borel sets. Measurable functions: definition , equivalence of properties and first examples (constant functions, characteristic functions and continous functions).

Sigma-algebras, functions and measures

Important: Wednesdays session change to 08:00-10:00 (SR 025/026)

Sheet 1


Measurable functions: sum, product, absolute value and others are measurable (trick using rational numbers for proving measurability of the sum). Measurability in the extended line. Measurability of inf, sup, lim inf and lim sup. Product of functions in the extended line. How to build a monotne increasing sequence of functions that converges pontwise to a given function.


Measurable functions between (general) measurable spaces. Measure over a measurable space: definition, and properties. First examples and relation with monotone sequences of measurable sets.

Lebesgue measure over R: algebras and measures over algebras. The algebra generated by the intervals.


Lebesgue measure over R: outer measure, Caratheodory's condition and Caratheodory's Theorem (No proof). Lebesgue sets and Lebesgue measure over Borel sets.

Integral of positive functions: simple functions and canonical representation. Integral of a simple function and first consequences. Integral of positive functions and first consequences. Statement of the Monotone Convergent Theorem.

Integral of positive functions, and the Monotone Convergence Theorem

Sheet 2


Integral of positive functions: proof of the Monotone Convergence Theorem. Consequences: linearity of the integral, Fatou Lemma, construction of measures from integrable functions. Functions that are 0 mu-a.a. And extension of MCT to cover such functions.


Lecture Cancelled


Integrals over general functions: definitions and integral of the absolute value. The Dominated Convergence Theorem. Applications: integrals depending on parameters. Limits with respect to parameters: continuity.

Dominated Convergence Theorem, parameters and extranotions in measure thory

Sheet 3


Problem Session


Limits with respect to parameters: derivatives.

Extra notions: the relation between the Riemann integral and the Lebesgue integral. Absolutely continous measures, Radon-Nikodym Theorem (No proof) and the Radon-Nikodym derivative. Product of measure spaces, existence and unicity of product measure.


Problem Session

Lp spaces

Sheet 4

(correction in Probl 2)


Problem Session


Product of measures: Fubini's Theorem

Lp spaces: definition of equivalence relation over L and Lebesgue spaces. Lp spaces: definition and properties of the "norm". Hölder's inequality (via Young inequality) and consequences. Minkowsky inequality. Riesz-Fischer Theorem.


The ABC in Probability theory: Pprobability spaces, events and probability functions. Random variables: law of a random variable, density probability functions and probability distribution functions. Expectation of a random variable, rth moment and variance. Results for the expectation arising from measure theory. Independence of events and independence of random variables. Conditional probability and conditional expectation.

The Basics on Probability Theory

Sheet 5


Problem Session


Holiday (NO lecture)


The ABC in Probability theory: vectors of random variables. Law of a vector of random variables, and restriction to the case of independent random variables.

Modes of convergence for random variables: convergence almost surely, in probability, in r-th mean mode and in distribution. Implications: convergence in probability implies convergence in distribution. Convergence in 1-th mean mode implies convergence in probability.

Modes of convergence

Sheet 6


Problem Session


Modes of convergence for random variables: implications between mean modes of convergence. Convergence almost surely implies convergence in probability: proof and related concepts. Inverse results (convergence in distribution + additional condition implies convergence in probability; convergence in probability + additional condition implies convergence in rth mean). Skorokhod' Representation Theorem and consequences.


Problem Session

0-1 Laws

Correction of one of the Theorems

Sheet 7

Typo corrected in Probl 4


Problem Session


Modes of convergence: proof of Skorokhod's Representation Theorem.

0-1 Laws: First and second Borel-Cantelli lemmas. Independence of families of events. Sigma-algebra generated by a family of random variables. Tail sigma-algebra and tail event. Statement of Kolmogorov's 0-1 Law.


Proof of Kolmogorov's 0-1 Law: family of independent families of events implies that the sigma-algebras are also independent. Final arguments for the proof of Kolmogorov's 0-1 law.

Probability generating functions of positive discrete random variables: definitions and first examples. Properties of formal power series as analytic obtects. First properties of probability generating functions (computation of factorial moments, convexity).

Probability GF and Branching Processes

Sheet 8

(correction in Prob. 2)


Problem Session


Probability generating functions: sum of independent random variables and product of GFs. Sum of independent identically distributed random variables depending on a random index: composition of GFs. Branching processes: model and equations. Probability GF for the number of members in a given generation of the process. Expectation and variance of the number of members in a given generation. Extinction and probability of extinction. Three regimes (subcritical, critical, supercrticial) and behaviour.


Branching processes: probability of extinction: proofs for the three regimes.

Extinction time: definition and relation with the probability generating function. Behaviour in the subcritical case (exponential decay) and the critical case (polinomial decay). Proof of the subcritical situation.

Extinction time, and Random walks

Sheet 9


Problem Session


Branching processes: extinction time in the critical case (sketch).

Random walks: definition of random walk. Escape from 0, recurrence and transience. Polya's transient theorem. Strategy: encoding by means of generating functions. Study of the unidimensional and bidimensional case (recurrent families).


Random walks: Transient case: dimension 3. Further comments further comments on higher dimensions, random walks with barriers and self avoiding walks.

Moment GFs: definition, first properties related with the independence and sums, examples.

Moment generating functions and central limit Theorem

Sheet 10


Moment GFs: bounds for tail estimates in terms of moment gf. Theorems for moment gfs: continuity and limits. Law of large numbers and Central Limit Theorem. Characteristic functions and Fourier transform.


Stochastic processes: definition, and defintion of Markov chain. equivalent formulations and time-homogeneous Markov chain. Examples: random walks, random walks with barrier, branching processes. Transition matrix.



Markov chains: Chapman-Kolmogorov's relations, evolution of the probability distribution of the Markov chain in terms of the transition matrix. Classification of states: recurrent and transient states. Characterization. Mean recurrence time, null (and non-null) states, period of an state, ergodic states and absorving states.

Markov chains

Sheet 11


Problem Session


Markov chains: interaction between states: comunicacion and intercomunication. Closed and irreducible set of states. Decomposition Theorem for Markov chains. Stationary distributions: definitions and first properties.


Problem Session

Sheet 12


Problem Session


Problem Session


Stationary distributions: construction of stationary distributions from the mean recurrence time. Lemma building an stationary distribution by using a non-null recurrent state. Main theorem on irreducible non-null recurrent Markov chains vs stationary distributions. Limit theorems in terms of the mean recurrence time.

The Main Theorem and related results


Problem Session


Time-reversible Markov chains: definition and characterization. Ehrenfest model for urns.

Finite state Markov chains: Perron-Frobenius theorem for stochastic matrices and applications (finite irreducible Markov chains are non-null recurrent).


Martingales: the gambler ruin and definition of martingale. Properties of the conditional expectation. Examples of martingales. Convergence theorem for martingales.



Problem Session


Problem Session


FINAL EXAM: starts at 9:00 at Arnimallee 6, SR032

Solution to Problems 4 and 5


MAKE-UP EXAM 15th September: starts at 9:00 at Arnimallee 6, SR032


Preliminary grading of the course

The following books will be used as basic bibliography for the course:

[1] Bartle, R.: The Elements of Integration and Lebesgue Measure, 1995
[2] Grimmett, G. R. and Stirzaker, D. R. Probability and Random Processes, 2001
[3] Meintrup, D., Schäffler, S.: Stochastik: Theorie und Anwendungen, 2005
[4] Varadhan, S.: Probability Theory, 2001
[5] Stroock, D. : An introduction to Markov Processes, 2005

The main reference for the first part of the book will be [1]. For the rest of the course, we will use [2] as a reference book.