Introduction to Probability Theory

The following sections are included:

• Preface
• About the Author
• Acknowledgments
• Contents

We think of probability theory as a mathematical model for random events. Therefore, probability theory is widely used in many different topics, for example:

• Biology,
• Economics/Insurance/Stochastic finance theory,
• Physics (for example statistical mechanics/Quantum mechanics),
• Mathematics (Random matrix theory/Number theory/Group theory/etc.)

In this chapter, we are going to give an introduction to combinatorial aspects of probability theory and describe several simple methods for the computation of different problems. We will also already give some of the most basic examples of discrete distributions and consider combinatorial aspects of random events as a first look at a simple random walks. We will look at the reflection principle and basic terminology of random walks as a combinatorial theory. Finally, we will consider probabilities of events for special random walks.

This is the main chapter of Part I, covering a basic introduction of the modern probability theory. We will discuss the concept of distributions and the notion of expectation at the beginning. Afterwards, the concept of moments, variance and covariance and several properties of those will be covered. We will continue with the concept of the characteristic function and independence, both for σ-algebras and for random variables. Furthermore, we look at the Borel–Cantelli lemma and move on to the weak and strong law of large numbers. The concept of different convergences will follow and finally we are going to spend time on the central limit theorem. After one has read this chapter, the more advanced structures and notions of probability theory of Part II will be accessible and lead to a fundamental understanding of modern probability theory.

If we consider a probability space (Ω, ℱ, ℙ) with a sequence of i.i.d. random variables (X_n)_n≥1, we can look at the expectation 𝔼[|X₁|] = … = 𝔼[|X_n|] < ∞. Now limit theorems play a central role, as we have seen in Part I. For example, we have seen the strong law of large numbers (Theorem 2.7.2.3), that is

The following sections are included:

• L²(Ω, ℱ, ℙ) as a Hilbert Space and Orthogonal Projections
  • Convex sets in uniformly convex spaces
  • Orthogonal projection
• Conditional Expectation
  • Review of conditional probability
  • Discrete construction of the conditional expectation
  • Continuous construction of the conditional expectation
• The Radon–Nikodym Approach for the Conditional Expectation
• More Properties of the Conditional Expectation
  • Important examples
• Basic Facts on Gaussian Vectors
• Transition Kernel and Conditional Distribution

The following sections are included:

• Discrete Time Martingales
• Submartingales and Supermartingales
• Martingale Inequalities
  • Maximal inequality and Doob’s inequality
• Almost Sure Convergence for Martingales
• L^p-convergence for Martingales
• Uniform Integrability
• Stopping Theorems
• Applications of Martingale Limit Theorems
  • Backward martingales and the law of large numbers
  • Martingales bounded in L² and random series
  • A martingale central limit theorem

The following sections are included:

• Definition and First Properties
• The Canonical Markov Chain
• Classification of States
  • Random variable on a graph
• Invariant Measures
  • Interpretation

The following sections are included:

• Appendix A. Basics of Measure Theory
  • Measurable Spaces
  • Topological Spaces
  • Borel Sets
  • Positive Measures
  • Measurable Maps
  • The Theorems of Lusin and Egorov
  • The Limit Superior and Limit Inferior
  • Simple Functions
  • Monotone Classes
• Appendix B. Basics of Integration Theory
  • Integration for Positive (Non-negative) Functions
  • Integrable Functions
    • Extension to the complex case
  • Lebesgue’s Dominated Convergence Theorem
  • Parameter Integrals
  • Differentiation of Parameter Integrals

The following sections are included:

• Bibliography
• Index