Exploration of a Nonlinear World

The following sections are included:

• Foreword

• Preface

• Acknowledgements

• Contents

• Publications of Howell Tong (In Reversed Chronological Order)

Please refer to full text.

The following sections are included:

• Prologue

• The Year 1977

• The Challenge

• Philosophy

• Non-linear Oscillations

• The Penny Drops

• Pride and Prejudice

• What Next?

• Epilogue

• References

The notion of a limit cycle, which can only exist in a non-linear system, plays the key role in the modelling of cyclical data. We have shown that the class of threshold autoregressive models is general enough to capture this notion, a definition of which in discrete time is proposed. The threshold value has an interesting interpretation. Simulation results are presented which demonstrate that this new class of models exhibits some well-known features of non-linear vibrations. Detailed analyses of several real data sets are discussed.

The following sections are included:

• Model Switching

• Density of the Threshold Autoregressive Functions

• Partly Linear Parametric Form

• SETAR Models and Conditional Variance

• Ergodicity and Stationary Solutions

• References

In 1980, the discussion paper, “Threshold Autoregression, Limit Cycles and Cyclical Data” (Tong and Lim (1980)) was presented to the Royal Statistical Society. In this article we review the contents of that paper and the impact of the paper on the study of nonlinear time series in the subsequent twenty nine years.

The Tong and Lim (1980) paper is shown to be a seminal paper in the statistical literature by examining the depth, breadth, and durability of its citation and subject area counts as tabulated from the ISI Web of Knowledge citation database. Two progenitor threshold models related to the SETAR model, Threshold Cointegration and Threshold GARCH, are presented, along with their citation and subject area counts, to illustrate two of the spillover effects generated by the Tong and Lim paper.

This discussion revisits Tong and Lim's seminal 1980 paper on the SETAR model in the context of advances in computation since that time. Using the Canadian lynx data set from that paper, it compares exact maximum likelihood estimates with those in the original paper. It illustrates the application of Bayesian MCMC methods, developed in the intervening years, to this model and data set. It shows that SETAR is a limiting case of mixture of experts models and studies the application of one variant of those models to the lynx data set. The application is successful, despite the small size of the data set and the complexity of the model. Predicive likelihood ratios favor Tong and Lim's original model.

Extensions of Tong's threshold approach to other fields of statistics abound. Among these, the application of the threshold approach to model volatility changes in financial time series has been particularly noteworthy. This paper aims to give a brief survey on this vast and important development since the birth of the threshold autoregression models.

The autoregressive moving average (ARMA) sequences have been used as time series models for a long time. It was clear that these linear models could not exhibit effects characteristic of data gathered on various wildlife populations and in their paper Tong and Lim 1980 proposed using threshold autoregressive schemes as a class of nonlinear models that might capture some of these effects. An ARMA sequence satisfies of system of equations

The following sections are included:

• Introduction

• Comment

• The Simple Idea that Works

• References

Please refer to full text.

We give a brief introduction to deterministic chaos and a link between chaotic deterministic models and stochastic time series models. We argue that it is often natural to determine the embedding dimension in a noisy environment first in any systematic study of chaos. Setting the stochastic models within the framework of non-linear autoregression, we introduce the notion of a generalized partial autocorrelation and an order. We approach the estimation of the embedding dimension via order determination of an unknown non-linear autoregression by cross-validation, and give justification by proving its consistency under global boundedness. As a by-product, we provide a theoretical justification of the final prediction error approach of Auestad and Tjøstheim. Some illustrations based on the Hénon map and several real data sets are given. The bias of the residual sum of squares as essentially a noise variance estimator is quantified.

This paper provides a survey of the recent development on model selection in semiparametric time series regression. In order to avoid using more data for model validation, this review briefly discusses a two-step model selection procedure, in which we extend the conventional nonparametric CV1 method proposed in Cheng and Tong (1992) to deal with both the optimum subset selection and optimum bandwidth choice. The main ideas and methodology of this review are based on the unpublished paper by Gao and Tong (2005).

The starting point of the paper by Cheng and Tong is a chaotic dynamic system. However an additional noise term is included making this into a “noisy” chaotic system, or in non-chaotic terminology, simply a nonlinear time series model

We have shown that within the setting of a difference equation it is possible to link ergodicity with stability via the physical notion of energy in the form of a Lyapunov function.

When introducing threshold time series models some 30 years ago, Howell Tong noted possible close connections with certain dynamical systems. This idea has generated much interest and study. One such connection compares stability of dynamical systems with that of nonlinear time series. I examine this relationship, noting when their conclusions agree and identifying parallels even when they diverge.

A personal overview of non-linear time series from a chaos perspective is given in an informal but, it is hoped, informative style. Recent developments which, in a radically new way, formulate the notion of initial-value sensitivity with special reference to stochastic dynamical systems are surveyed. Its practical importance in prediction is highlighted and its statistical estimation included by appealing to the modern technique of locally linear non-parametric regression. The related notions of an embedding dimension and correlation dimension are also surveyed from the statistical stand-point. It is shown that deterministic dynamical systems theory, including chaos, has much to offer to the subject. In return, some current results in the subject are summarized, which suggest that some of the standard practice in the former may have to be revised when dealing with real noisy data. Several open problems are identified.

The following sections are included:

• Introduction

• “On the Use of the Deterministic …” (Adv. Appl. Prob. 1985)

• “Some Comments On A Bridge …” (Physica D, 1992)

• “A Personal Overview of …” (Scand. J. Stat, 1995)

• References

This paper picks up some points in Howell Tong's work over the past 38 years which have common cause with the author's own work over the same period. Among these are an interest in time series reversibility, chaotic time series and applications of time series. In particular, chaotic communications engineering research, only then developed enough for a brief final mention in Tong's 1995 overview paper, is illustrated by a subsequent contribution to chaotic time series modelling and by some of the author's research on the performance of chaotic communication systems.

This is a selective review on two of Howell Tong's papers ^{3, 16} on the interplay between nonlinear (stochastic) time series and deterministic chaos.

Please refer to full text.

In this paper, we propose a general definition of the evolutionary (time-dependent) cross-spectrum between two non-stationary processes and describe its physical interpretation. We also study the estimation of the evolutionary cross-spectrum at each time instant t from a single realization of a bivariate process. Further, we propose a definition (and a method of estimation) for the coherency (spectrum) between the two components of the bivariate process and show that the notion of residual variance bound first introduced in the analysis of bivariate stationary processes can be extended to that of non-stationary processes. As an application of the evolutionary cross-spectral analysis of bivariate non-stationary stochastic processes, we consider the estimation of the transfer function of a linear open-loop time-dependent system. Numerical illustrations of the estimation of a time-dependent transfer function are included.

This paper addresses the null distribution of the likelihood ratio statistic for threshold autoregression with normally distributed noise. The problem is non-standard because the threshold parameter is a nuisance parameter which is absent under the null hypothesis. We reduce the problem to the first-passage probability associated with a Gaussian process which, in some special cases, turns out to be a Brownian bridge. It is also shown that, in some specific cases, the asymptotic null distribution of the test statistic depends only on the ‘degrees of freedom’ and not on the exact null joint distribution of the time series.

We have shown that the least squares estimator for a non-ergodic, first order, self-exciting, threshold autoregressive model is strongly consistent under quite general conditions.

This note makes some remarks on the two papers, “Strong Consistency of the Least Squares Estimator for a Non-Ergodic Threshold Autoregressive Model” in Statistica Sinica (1991) and “On Likelihood Ratio Tests for Threshold Autoregression” in Journal of the Royal Statistical Society, Series B (1990), by Professor Tong and his collaborators. This note also discusses the limiting distribution of the LSE for the unit root TAR(1) model and gives the rate of convergence of the LSE for an explosive TAR(1).

Please refer to full text.

Searching for an effective dimension reduction space is an important problem in regression, especially for high dimensional data. We propose an adaptive approach based on semiparametric models, which we call the (conditional) minimum average variance estimation (MAVE) method, within quite a general setting. The MAVE method has the following advantages. Most existing methods must undersmooth the nonparametric link function estimator to achieve a faster rate of consistency for the estimator of the parameters (than for that of the nonparametric function). In contrast, a faster consistency rate can be achieved by the MAVE method even without undersmoothing the nonparametric link function estimator. The MAVE method is applicable to a wide range of models, with fewer restrictions on the distribution of the covariates, to the extent that even time series can be included. Because of the faster rate of consistency for the parameter estimators, it is possible for us to estimate the dimension of the space conSistently. The relationship of the MAVE method with other methods is also investigated. In particular, a simple outer product gradient estimator is proposed as an initial estimator. In addition to theoretical results, we demonstrate the efficacy of the MAVE method for high dimensional data sets through simulation. Two real data sets are analysed by using the MAVE approach.

Xia, Tong, Li and Zhu (2002) proposed a general estimation method termed minimum average variance estimation (MAVE) for semiparametric models. The method has been found very useful in estimating complicated semiparametric models (Xia, Zhang and Tong, 2004; Xia and Härdle, 2006) and general dimension reduction (Xia, 2008; Wang and Xia, 2008). The method is also convenient to combine with other methods in order to incorporate additional statistical requirements (Wang and Yin, 2007). In this paper, we give a general review on the method and discuss some issues arising in estimating semiparametric models and dimension reduction (Li, 1991 and Cook, 1998) when complicated statistical requirements are imposed, including quantile regression, sparsity of variables and censored data.

Across the boreal forest of Canada, lynx populations undergo regular density cycles. Analysis of 21 time series from 1821 onward demonstrated structural similarity in these cycles within large regions of Canada. The observed population dynamics are consistent with a regional structure caused by climatic features, resulting in a grouping of lynx population dynamics into three types (corresponding to three climatic-based geographic regions): Pacific-maritime, Continental, and Atlantic-maritime. A possible link with the North Atlantic Oscillation is suggested.

The following sections are included:

• Introduction

• The TAR Lynx Models and Phase Dependence

• The Geographic Structuring of the Canadian Lynx: The Importance of Statistical Modelling

• Conclusions

• What Next?

• Acknowledgements

• References

Professor Howell Tong has distinguished himself for his work in non-linear time series analysis and especially for his seminal work on threshold autoregression, limit cycles and cyclical data. Among his extensive publications over the past four decades, there are two papers on reliability which he had published in the seventies. In this paper, a brief review of his two papers on the reliability function P(Y < X) is made.

A poem (in the Tang style) that Howell wrote on the occasion of his receiving the Guy medal. Below is a translation by Mr. Kwan Yee-Kwong:

My newly-mowed lawn showed me the Way

Non-linear vision finally carried the day.

On my threshold shone a silvery light

Late-coming, yet a joyous sight.