A First Look at Rigorous Probability Theory

The following sections are included:

• Preface

• Contents

This introductory section is directed primarily to those readers who have some familiarity with undergraduate-level probability theory, and who may be unclear as to why it is necessary to introduce measure theory and other mathematical difficulties in order to study probability theory in a rigorous manner…

In this section we consider probability triples and how to construct them. In light of the previous section, we see that to study probability theory properly, it will be necessary to keep track of which subsets A have a probability P(A) defined for them.

Now that we understand probability triples well, we here discuss some additional essential ingredients of probability theory. Throughout Section 3 (and, indeed, throughout most of this text and most of probability theory in general), we shall assume that there is an underlying probability triple (Ω, F, P) with respect to which all further probability objects are defined. This assumption shall be so universal that we will often not even mention it.

There is one more notion that is fundamental to all of probability theory, that of expected values. The general definition of expected value will be developed in this section.

We begin with two very important inequalities about random variables.

The distribution or law of a random variable is defined as follows.

Now that we have covered most of the essential foundations of rigorous probability theory, it is time to get “moving”, i.e. to consider random processes rather than just static random variables…

In this section, we consider the general notion of a (discrete time, discrete space, time homogeneous) Markov chain…

In this section, we discuss a few probability ideas that we have not needed so far, but which will be important for the more advanced portions of the notes.

Given Borel probability distributions μ,μ₁, μ₂,… on R, we shall write μ_n ⇒ μ, and say that {μ_n} converges weakly to μ, if ∫_R f d μ_n → ∫_R f dμ for all bounded continuous Borel-measurable functions f : R → R.

This is a rather natural definition, though we draw the reader's attention to the fact that this convergence need hold only for continuous functions f. That is, the "topology" of R is being used here, not just its measure-theoretic properties. This will be evident in equivalent conditions of weak convergence that we shall develop, including the following. For condition (2), recall that the boundary of a set A ⊆ R is defined by ∂A = {x ∈ R; ∀∊ > 0, A∩(x - ∊, x + ∊) ≠ ∅, A^C ∩ (x - ∊, x + ∊) ≠ ∅}.

The following sections are included:

• The continuity theorem

• The Central Limit Theorem

• Generalisations of the Central Limit Theorem

• Method of moments

• Additional exercises

• Section summary

Let μ be a Borel probability measure on R. Recall that μ is discrete it takes all its mass at individual points, i.e. if Σ_x∈Rμ{x} = μ(R). Also μ is absolutely continuous if there is a non-negative Borel-measurable function f such that μ(A) = ∫_A f(x)λ(dx) = ∫ l_A(x) f(x)λ(dx) for all Borel sets A, where λ is Lebesgue measure on R…

“Conditioning” is a very important concept in probability, and we shall consider it here.

In this section we study a special kind of stochastic process called a martingale. A stochastic process X₀, X₁, … is a martingale if E (|X_n|) < ∞ for all n, and if with probability 1…

We end this text with a brief look at more general stochastic processes. We attempt to give an intuitive discussion of this area, without being overly careful about mathematical precision. (A full treatment of these processes would require another course, perhaps following one of the books in Subsection B.5.) In particular, in this section a number of results are stated without being proved, and a number of equations are derived in an intuitive and non-rigorous manner. For clarity, all exercises are grouped by subsection, rather than at the end of the entire section…

This section reviews some of the mathematics that is necessary as a prerequisite to understanding this text. The material is reviewed in a form which is most helpful for our purposes. Note, however, that this material is merely an outline of what is needed; if this section is not largely review for you, then perhaps your background is insufficient to follow this text in detail, and you may wish to first consult references such as those in Subsection B.1.

The books in Subsection B.1 below should be used for background study, especially if your mathematical background is on the weak side for understanding this text. The books in Subsection B.2 are appropriate for lower-level probability courses, and provide lots of intuition, examples, computations, etc. (though they largely avoid the measure theory issues). The books in Subsection B.3 cover material similar to that of this text, though perhaps at a more advanced level; they also contain much additional material of interest. The books in Subsection B.4 do not discuss probability theory, but they do discuss measure theory in detail. Finally, the books in Subsections B.5 and B.6 discuss advanced topics in the theory of stochastic processes and of mathematical finance, respectively, and may be appropriate for further reading after mastering the content of these notes.

The following sections are included:

• Index