Probability Theory

The following sections are included:

• CONTENTS

• PREFACE TO THE ENGLISH EDITION

• INTRODUCTION
  • A space of elementary events

This chapter deals with basic concepts of probability theory such as probability distribution, random variable and its mean (expectation).

In this chapter we first discuss the law of large numbers which lies in the background of probability theory and, in a sense, “justifies” it. Then we study the behavior of the number of “successes” in series of independent trials when the number of trials becomes large. The corresponding theorems may be of help in studying many practical problems and investigating rather complicated models.

The preceding Sections were devoted to the so-called direct methods, whose use permits us to calculate probabilities of some or other events directly. These methods are not much applicable. In this Section we employ one of the analytic methods of probability theory, namely the method of generating functions. It allows us to obtain a number of results quite simply, whose direct proof would be formidable or even impossible. In particular, we consider the extinction problem and the problem capable of describing random walks on the line.

This chapter is devoted to the axiomatics of probability theory, that is, to constructions of a rigorous probabilistic model of experiments with random outcomes. A part of the material will be used in the sequel as a source for references. For this reason some proofs are omitted; they can be found in many textbooks on functional analysis. In order to facilitate reading, the subsequent probabilistic scheme is explained in Section 1 on a heuristic level and a number of useful formulae is deduced before proceeding to rigorous constructions. The reader who is inclined to “take the scheme on trust” can only look through in the first reading the basic notions of the chapter or even omit Sections 2-3 and turn to concrete probabilistic-theoretic results of subsequent sections. Nevertheless, for more or less clear understanding of the theory one has to absorb the basic material of the chapter.

The chapter is devoted to probability distributions of random variables and vectors. After general constructions a number of concrete distributions playing an important role both in the theory and applications will be of special investigations.

Our nearest aim is to extend the notions of conditional probability and expectation to the general case. In particular, we give a rigorous meaning to the following intuitively clear analog of the total probability formula

where X,Y are random variables and D is a Borel set. Earlier, in Part I, the total probability formula has been obtained for a countable system of disjoint events B_j such that Σ_j,B_j = Ω. Eq. (1) is the analog of the mentioned formula for the case of uncountable system of events B_x = {X = x}. As we will see later, the fact that the system {B_x} is related with values of some random variable is of no importance…

The following sections are included:

• Conditionally Independent and Exchangeable Random Variables
  • Mixtures of distributions
  • Conditionally independent random variables
  • Symmetrically dependent or exchangeable random variables

• Martingales
  • Definitions, properties and examples
  • Markov moments and boundary problems
  • Continuity properties
  • Proof of De Finetti theorem

• Markov Chains
  • Basic notions and properties
  • Ergodicity property

This chapter is central in the present course. In dealing with a sequence of r.v.’s X₁, X₂, … we consider mainly the case of independent variables but also touch upon some kinds of dependence. Let Z_n = f_n(X₁,…,X_n) be a function of the first n random variables. In the sequel the main attention will be paid to the case f_n(x₁,…,x_n) = x₁+…+x_n, that is, to sums of random variables, although some nonlinear functions will be also considered: minima and maxima of random variables, quadratic forms, polynomials of arbitrary orders and more. In the theory under consideration the following result is typical and plays a key role…

In the preceding chapters we have been interested in distributions of various random variables. In particular, we have established convergence in distribution of some or other random variables. This section focuses on stronger types of convergence.

The following sections are included:

• REFERENCES