Uncertainty and Optimality

The following sections are included:

• Preface

• About the Editor

• Contents

We examine the historical development of the three major paradigms in Statistics and how they have influenced each other in a positive way. We then go on to argue that it is still necessary to make a choice and that the Bayesian formulation appears to be the most appropriate. Since elicitation of priors remains difficult inspite of some progress, it is suggested that nonsubjective Bayesian analysis has an important role to play. We present an overview of how nonsubjective priors are constructed and how they are used in different problems of inference involving low or high dimensional models. In particular, it is shown that many of the common perceptions or criticisms of nonsubjective Bayesian analysis are not justified.

Stochastic partial differential equations (SPDE) are used for stochastic modelling, for instance, in the study of neuronal behaviour in neurophysiology, in modelling sea surface temparature and sea surface height in physical oceanography, in building stochastic models for turbulence and in modelling environmental pollution. Probabilistic theory underlying the subject of SPDE is discussed in Ito [2] and more recently in Kallianpur and Xiong [11] among others. The study of statistical inference for the parameters involved in SPDE is more recent. Asymptotic theory of maximum likelihood estimators for a class of SPDE is discussed in Huebner, Khasminskii and Rozovskii [7] and Huebner and Rozovskii [8] following the methods in Ibragimov and Khasminskii [9]. Bayes estimation problems for such a class of SPDE are investigated in Prakasa Rao [21,25] following the techniques developed in Borwanker et al. [2]. An analogue of the Bernstein-von Mises theorem for parabolic stochastic partial differential equations is proved in Prakasa Rao [21]. As a consequence, the asymptotic properties of the Bayes estimators of the parameters are investigated. Asymptotic properties of estimators obtained by the method of minimum distance estimation are discussed in Prakasa Rao [30]. Nonparametric estimation of a linear multiplier for some classes of SPDE is studied in Prakasa Rao [26,27] by the kernel method of density estimation following the techniques in Kutoyants [12]. In all the papers cited above, it was assumed that a continuous observation of the random field satisfying the SPDE is available. It is obvious that this assumption is not tenable in practice for various reasons. The question is how to study problem of estimation when there is only a discrete sampling on the random field. A simplified version of this problem is investigated in Prakasa Rao [28,29,30,31]. A review of these and related results is given.

Let X₁,…,X_n be any n random variables (rv's) and let X_1:n ≤ … ≤ X_n:n denote the same variables arranged in nondecreasing order. Then X_r:n is called the rth order statistic, r = 1,…,n. When one of the X's is dropped at random, there results a simple relation between the order statistics in the original and the reduced samples. This "dropping" argument will be shown to provide a unified approach to establishing recurrence relations between moments of order statistics, whatever the dependence structure of the observations. Also useful in studying recurrence relations is the classical theorem on the probability of occurrence of r events out of n.

It will also be shown that a simple general method of obtaining universal bounds for linear functions of order statistics in terms of the sample standard deviation can be based on Cauchy's inequality coupled with convexity arguments.

In this paper we review some of the results obtained recently in the area of stochastic comparisons of order statistics and sample spacings. We consider the cases when the parent observations are identically as well as non-identically distributed. But most of the time we shall be assuming that the observations are independent. The case of independent exponentials with unequal scale parameters is discussed in detail.

The following sections are included:

• Introduction
  • Regression: the basic model

• Parametric regression
  • Polynomial regression
  • Model selection
  • Other parametric models: logistic regression

• Nonparametric regression
  • Kernel smoothing
  • Cross-validation
  • Other nonparametric regression estimates: splines

• Resampling
  • Nonparametric bootstrap
  • Parametric bootstrap
  • Other methods of resampling: the jackknife

• Further reading

• Exercises

• References

Sometimes the experimenter has a suspicion or prior belief that the distribution belongs to a particular parametric family, like the normal, exponential, Poisson etc. This could be because the experimental conditions point to a particular distribution as the appropriate one or because of past experience of similar experiments. He then wishes to either confirm or reject this prior belief through a 'test of goodness of fit'. There are three major ways of carrying out such tests : (i) the chi-squared Goodness of fit test of Karl Pearson, (ii) the Kolmogorov - Smirnov goodness of fit test based on the empirical distribution function, and (iii) the Hellinger distance based methods. We shall describe these in successive sections. Then these will be followed by methods developed for testing goodness of fit of two specific popular distributions viz., the exponential and the normal. There are certain graphical procedures used as diagnostic indicators of the family governing the outcomes. These will be discussed in the last section.

After a quick introduction to some basic properties of U statistics with examples, we discuss M_m estimators and their asymptotic properties under easily verifiable conditions. In particular, these estimators are approximately U statistics and as a consequence, a huge collection of commonly used estimators are consistent and asymptotically normal. We also establish some higher order asymptotic properties of these estimates. The material is more or less self contained.

Assume that observations have a common distribution function , which belongs to a family of distributions indexed by . We are interested in making inferences about the unknown parameter vector based upon generalized rank set data, i.e. J independent order statistics X_ij:nj, j = 1, 2, …,J with a common parent distribution . We will discuss (i) the problem of estimating or deriving probability bounds for in the Bayesian sense, (ii) testing composite hypotheses concerning , and (iii) testing goodness of fit to the model .

We obtain expressions for the elements of the Fisher information matrix (FIM) for the three parameters of the Gumbel Type II bivariate exponential (G₂BVE) distribution. This distribution belongs to the Farlie-Gumbel-Morgenstern family and has exponential marginals. We evaluate the FIM for various values of the dependence parameter and discuss implications to finite-sample and asymptotic inference from the G₂BVE parent. We also conduct a similar study for the Marshall-Olkin bivariate exponential distribution and compare the results.

Invariance property in data envelopment analysis (DEA) allows negative data in efficiency analysis. In general, there are three cases of invariance under data transformation in DEA. The first case is the "classification invariance" where the classifications of efficiencies and inefficiencies are invariant to the data transformation. The second case is the "ordering invariance" of the inefficient decision making units (DMUs). The last case is the "solution invariance" in which the new DEA model (after data translation) must be equivalent to the old one. The current paper indicates that DMUs with negative output values may be classified as efficient when we use the classification invariance. Although such classification is mathematically correct, it may not be managerially acceptable. The method of finding well-defined facet is suggested to re-evaluate the performance of DMUs with negative values. The paper illustrates the approach with an application to textile firms where negative profit is present.

Inequalities and error bounds are derived for finite state, irreducible, time reversible Markov chains in continuous time. The results are illustrated in a reliability example involving a 2 out of 4 repairable system. The inequalities are derived via an isometry between two inner product spaces, one corresponding to the chain of interest, and the other to its companion star chain. This connection between the two Markov chains was originally exploited in Aldous and Brown (1992).

In this paper, we study the applications of information-theoretic concepts to characterise probability distributions as maximum entropy or minimum cross-entropy probability distributions. We also develop an entropic measure of stochastic dependence and apply it to obtain the measure of dependence in some multivariate distributions and also to measure dependence in contingency tables. We also derive the principle of maximum likelihood from both maximum entropy and minimum cross-entropy principles. We also compare entropic method of estimating parameters with Fishers and Pearson's methods. We also find probability distribution of a family which is closest to a mixture of distributions of some members of the same family.

The classical problem of providing a 'point estimator' for a survey population total along with an interval around it needs an appropriate estimator for its mean square error.

A brief resume is provided for modern approaches to solutions for this by model-motivated-cum-design-based methods covering multi-stage unequal probability sampling, small domain estimation randomized responses for sensitive issues and employing in particular adaptive sampling and bootstrap techniques. Relevant current thoughts and practices are especially accommodated.

A linear program (LP) Minimize c^tx subject to Ax = b, x ≥ 0 (null column vector), where A is an m×n real matrix, c and b are n- and m-vectors, respectively is a problem of great importance in numerous physical problems involving linear optimization such as diet problems, transport problems, industrial production problems. The algorithms such as simplex method, self-dual parametric algorithm, decomposition algorithm, primal-dual algorithm to solve an LP have been non-polynomial time. In order to appreciate the recent advances in this area the present chapter provides a background based on the simplex methods which completely ruled the scene during sixties, seventies and early eighties. Although the simplex methods are non-polynomial-time in the worst case, they did perform excellently in most real-world problems and behaved like a fast (polynomial-time) algorithm. The chapter then focuses on the development of several fast (polynomial-time) algorithms during the last two decades. It then briefly highlights heuristic and evolutionary approach to solve LPs including errorfree implementation.

Cell formation problems are practically important and are NP hard, which is very difficult to solve. Various operations research techniques, from the early use of the various mathematical programming techniques to the more recent neural fuzzy approaches, have been proposed to use to solve this problem. This chapter presents these operations research approaches. To save space and also to introduce the approaches in reasonably detail, at least one numerical example is used for each type of the technique discussed. A detailed list of references is also given.

Scheduling problems form the core of the operational planning problem in typically large State Road Transport Corporations in India. The problems include, scheduling of trips to satisfy the traffic demand, allocation of trips to depots, and scheduling of buses and crews to operate the trips while satisfying various operational constraints and efficiency considerations. The size and the structural complexity of these hard problems involve solution approaches that emphasize interplay between modelling, algorithms, and their efficient computer implementation, requiring blending of ideas from Transportation Science, Operations Research and Computer Science. The solutions need to be complete and closer to the real-life practice for effective implementation. This paper presents our experiences in addressing these issues and highlights the insights gained from our efforts to implement the solutions in real-life.

The dynamic power management is one of the most effective technologies for reducing the power consumption in computer systems. Especially, the sleep function based on the shutdown policy is usually installed in the almost operating systems. In this paper, we propose a stochastic model based on the dynamic power management concept to determine the optimal shutdown policy. More precisely, introducing the so-called power effectiveness criterion by taking account of the processing efficiency, the optimal shutdown policies maximizing the power effectiveness can be derived in two cases; single-user operating system and multi-tasking operating system. In a numerical example, we calculate the optimal shutdown policies numerically and perform the sensitivity analysis of model parameters.

Over the years combinatorial optimization problems have become of considerable importance and have been studied in literature extensively. In this chapter we describe a unified structure for such problems and concentrate on local solutions with respect to a given neighborhood. Such problems can be structured as search problems on hypercubes.