Elements of Stochastic Modelling

The following sections are included:

• Dedication
• Preface to the First Edition
• Preface to the Second Edition
• Preface to the Third Edition
• About the Author
• Contents

1. On the nature of mathematical modelling. Imagine a complex real-world system (say, the economy of a country or an ecological system) and suppose that one wants to know how the system will tend to evolve in the future. This is of particular importance when one is about to change something in the system (say, introduce a new tax or build a dam). Realworld experimentation is usually very risky and costly, on the one hand, and far too slow, on the other, to think about trying it seriously. Instead, one develops a model—an “imitation” of the system of interest—and first studies and/or “runs” or “simulates” that model. It is very important to be able to study the evolution of the modelled system in “compressed time” and compare the consequences of different policies or possible actions for the system…

To begin with, we would like to stress that probability theory is a rather special area of mathematics. Firstly, it is extremely broad in terms of its applicability: one can introduce probability on nearly any object belonging to any other area of mathematics, and this implies, in turn, its wide use in various applications. Secondly, unlike, say, Euclidean geometry and elementary calculus, probability theory is less supported by our everyday experience and hence is often counterintuitive. So usually it takes quite some time to develop “probabilistic intuition” enabling one to deal more confidently and feel more comfortable with probability, and successfully build and analyse meaningful stochastic models…

The following sections are included:

• Definitions
• Classification of States
• Further Examples
• The Limiting Behaviour of Markov Chains
• Random Walks
• Recommended Literature
• Problems

In this chapter we will discuss mathematical models enabling one to construct and execute optimal (in a certain sense) strategies in environments evolving in time in the presence of uncertainty. We assume that we have a process describing a system evolving in discrete time where, at each step, one is required to take an action. The consequences of the action are not certain, but the distribution of its possible outcomes depends only on the state at which the action was taken—and of the action itself as well, of course. A possible action could simply be the termination of the process at some stage. Graphically, this scheme can be illustrated by an example shown in the tree diagram in Fig. 4.1…

The following sections are included:

• Properties of the Exponential Distribution
• The Poisson Process
• Problems

The following sections are included:

• Definitions and Basic Results
• Inhomogeneous Processes
• Birth-and-Death Processes
• PASTA
• Recommended Literature
• Problems

The following sections are included:

• Definitions and Notation
• Exponential Queueing Systems
  • M/M/1 Systems
  • M/M/a Systems
  • M/M/a/N Systems
• The Machine Repair Problem
• Exponential Queueing Networks
• Recommended Literature
• Problems

The following sections are included:

• Definitions and Notation: Renewal Theorems
• Problems

A time series (TS) is a sequence of observations {x_t} recorded at (usually) regular time intervals. For convenience sake, we often assume that the time values are t = 1, 2, …, T. The difference from the notion of a random sample is that the order of observations is now important. In other words, a TS is a realisation of an SP in discrete time. The term is often used to mean both the data {x_t} (the x_t’s are numbers) and the process {X_t} itself (the X_t’s are RVs!), of which the former is a realisation…

The following sections are included:

• Basics: Random Number Generators
• The Inverse Function Method
• The Rejection Method
• Monte Carlo: Variance Reduction Methods
  • The Crude Monte Carlo
  • The Control Variates Method
  • The Stratified Sample Method
  • The Antithetic Variables Method
  • The Importance Sampling Method
• Markov Chain Monte Carlo
• Recommended Literature
• Problems

In this chapter we will present the basics of stochastic calculus, a part of Probability Theory that deals with constructing integrals with respect to random processes that, generally speaking, would make no “path-wise” sense. For more advanced treatment of stochastic calculus, the reader is referred to the books from the list of recommended literature in Section 11.6.

The following sections are included:

• Definitions
• Kolmogorov Differential Equations and Generators
• Stationary Distributions
• The Method of Differential Equations
• Some Applications
  • Branching Processes
  • The Wright–Fisher Model
  • The Brownian Bridge Process
• Recommended Literature
• Problems

The following sections are included:

• Introductory Remarks
• Binomial Markets
• The Single-Period Binomial Market
• Finite Single-Period Markets
• The Multi-Period Binomial Market
• Martingales and Claim Pricing
• The Black–Scholes Framework
• Pricing Barrier Options
• Recommended Literature
• Problems

This section contains the answers and solutions to problems at end of each chapter.

The following sections are included:

• Greek Alphabet
• Notations
• Abbreviations
• Index