Narrow Results

Stochastic volatility models describe stock returns $r_t$ as driven by an unobserved process capturing the random dynamics of volatility $v_t$. The present paper quantifies how much information about volatility $v_t$ and future stock returns can be inferred from past returns in stochastic volatility models in terms of Shannon’s mutual information. In particular, we show that across a wide class of stochastic volatility models, including a two-factor model, returns observed on the scale of seconds would be needed to obtain reliable volatility estimates. In addition, we prove that volatility forecasts beyond several weeks are essentially impossible for fundamental information theoretic reasons.

The error rate in a current-controlled logic microprocessor dominated by shot noise has been investigated. It is shown that the error rate increases very rapidly with increasing cutoff frequency. The maximum clock frequency of the processor, which works without errors, is obtained as a function of the operational current. The information channel capacity of the system is also studied.

Shannon information in the genomes of all completely sequenced prokaryotes and eukaryotes are measured in word lengths of two to ten letters. It is found that in a scale-dependent way, the Shannon information in complete genomes are much greater than that in matching random sequences — thousands of times greater in the case of short words. Furthermore, with the exception of the 14 chromosomes of Plasmodium falciparum, the Shannon information in all available complete genomes belong to a universality class given by an extremely simple formula. The data are consistent with a model for genome growth composed of two main ingredients: random segmental duplications that increase the Shannon information in a scale-independent way, and random point mutations that preferentially reduces the larger-scale Shannon information. The inference drawn from the present study is that the large-scale and coarse-grained growth of genomes was selectively neutral and this suggests an independent corroboration of Kimura's neutral theory of evolution.

Artificial neural networks, due to their ability to find the underlying model even in complex highly nonlinear and highly coupled problems, have found significant use as prediction engines in many domains. However, in problems where the input space is of high dimensionality, there is the unsolved problem of reducing dimensionality in some optimal way such that Shannon information important to the prediction is preserved. The important Shannon information may be a subset of total information with an unknown partition, unknown coupling and linear or nonlinear in nature. Solving this problem is an important step in classes of machine learning problems and many data mining applications. This paper describes a semi-automatic algorithm that was developed over a 5-year period while solving problems with increasing dimensionality and difficulty in (a) flow prediction for a magnetically levitated artificial heart (13 dimensions), (b) simultaneous chemical identification/concentration in gas chromatography (22 detection dimensions with wavelet compressed time series of 180,000 points), and finally in (c) financial analytics portfolio prediction in credit card and sub-prime debt problems (80 to 300 dimensions of sparse data with a portfolio value of approximately US$300,000,000.00). The algorithm develops a map of input space combinations and their importance to the prediction. This information is used directly to construct the optimal neural network topology for a given error performance. Importantly, the algorithm also produces information that shows whether the space between input nodes is linear or nonlinear; an important parameter in determining the number of training points required in the reduced dimensionality of the training set. Software was developed in the MatLAB environment using the Artificial Neural Network Toolbox, Parallel and Distributed Computing toolboxes, and runs on Windows or Linux based supercomputers. Trained neural networks can be compiled and linked to server applications and run on normal servers or clusters for transaction or web based processing. In this paper, application of the algorithm to two separate financial analytics prediction problems with large dimensionality and sparse data sets are shown. The algorithm is an important development in machine learning for an important class of problems in prediction, clustering, image analysis, and data mining. In the first example application for subprime debt portfolio analysis, performance of the neural network provided a 98.4% prediction rate, compared to 33% rate using traditional linear methods. In the second example application regarding credit card debt, performance of the algorithm provided a 95% accurate prediction (in terms of match rate), and is 10% better than other methods we have compared against, primarily logistic regression.

We review a general mathematical link between utility and information theory appearing in a simple financial market model with two kinds of small investors: insiders, whose extra information is stored in an enlargement of the less informed agents' filtration. The insider's expected logarithmic utility increment is described in terms of the information drift, i.e. the drift one has to eliminate in order to perceive the price dynamics as a martingale from his perspective. We describe the information drift in a very general setting by natural quantities expressing the conditional laws of the better informed view of the world. This on the other hand allows to identify the additional utility by entropy related quantities known from information theory.