Narrow Results

This paper develops a unified algorithm to address the Conditional Value at Risk (CVaR) optimization problem characterized by two levels of expectations, where the inner-level expectation handles the repricing of the financial instruments and the outer-level expectation represents the risk measure. We propose an FA–SAA algorithm that tackles the inner-level expectation through function approximation (FA) method and addresses the outer-level expectation by sample average approximation (SAA). A variety of machine learning methods can be incorporated into this algorithm and eventually the optimization problem is reformulated into a linear programming problem, whose optimal value and optimal solution converge to the actual ones. Besides, we demonstrate that the convergence rate of optimal value can achieve $O_{\mathrm{\text{P}}}({\mathrm{\Gamma }}^{-1/2})$. The numerical results substantiate the efficacy of our proposed algorithm.

In function approximation problems, one of the most common ways to evaluate a learning algorithm consists in partitioning the original data set (input/output data) into two sets: learning, used for building models, and test, applied for genuine out-of-sample evaluation. When the partition into learning and test sets does not take into account the variability and geometry of the original data, it might lead to non-balanced and unrepresentative learning and test sets and, thus, to wrong conclusions in the accuracy of the learning algorithm. How the partitioning is made is therefore a key issue and becomes more important when the data set is small due to the need of reducing the pessimistic effects caused by the removal of instances from the original data set. Thus, in this work, we propose a deterministic data mining approach for a distribution of a data set (input/output data) into two representative and balanced sets of roughly equal size taking the variability of the data set into consideration with the purpose of allowing both a fair evaluation of learning's accuracy and to make reproducible machine learning experiments usually based on random distributions. The sets are generated using a combination of a clustering procedure, especially suited for function approximation problems, and a distribution algorithm which distributes the data set into two sets within each cluster based on a nearest-neighbor approach. In the experiments section, the performance of the proposed methodology is reported in a variety of situations through an ANOVA-based statistical study of the results.

In this paper, a measure of sensitivity is defined to evaluate fault tolerance of neural networks and we show that the sensitivity of a link is closely related to the amount of information passed through it. Based on this assumption, we prove that the distribution of output error caused by s-a-0 (stuck at 0) faults in an MLP network has a Gaussian distribution function. UDBP (Uniformly Distributed Back Propagation) algorithm is then introduced to minimize mean and variance of the output error. Then an MLP neural network trained with UDBP, contributes in an Algorithm-Based Fault Tolerant (ABFT) scheme to protect a nonlinear data process block. A systematic real convolution code guarantees that faults representing errors in the processed data will result in notable nonzero values in syndrome sequence. A majority logic decoder can now easily detect and correct single faults by observing the syndrome sequence. Simulation results demonstrating the error detection and correction behavior against random s-a-0 faults are presented too.

This paper intends to propose an integrated hybrid algorithm for training radial basis function neural network (RBFNN) learning. The proposed integrated of particle swarm and genetic algorithm based optimization (IPGO) algorithm is composed of two approaches based on particle swarm optimization (PSO) and genetic algorithm (GA) for gathering both their virtues to improve the learning performance of RBFNN. The diversity of individuals results in higher chance to search in the direction of global optimal instead of being confined to local optimal particularly in problem with higher complexity. The IPGO algorithm with PSO-based and GA-based approaches has shown promising results in some benchmark problems with three continuous test functions. After proposing the algorithm for these problems with result providing its outperforming performance, this paper supplements a practical application case for the papaya milk sales forecasting to expound the superiority of the IPGO algorithm. In addition, model evaluation results of the case have showed that the IPGO algorithm outperforms other algorithms and auto-regressive moving average (ARMA) models in terms of forecasting accuracy and execution time.

A popular approach to training feed-forward nets is to treat the problem of adaptation as a function approximation and to use curve fitting techniques. We discuss here the problems which the use of pure curve fitting techniques entail for the generalization capability and robustness of the net. These problems are in general inherently associated with the use of pure supervised learning techniques. We argue that a better approach to the training of feed-forward nets is to use adaptive techniques that combine properties of both supervised and unsupervised learning. A new formulation of the training problem is presented here. According to this formulation the net is viewed as two coupled sub-nets the first of which is trained by an unsupervised learning technique and the second by a supervised one. The same formulation gives rise to analytic expressions of the goals of the adaptation and leads to a new method for the adaptation of feed-forward nets.

A study of the function approximation capabilities of single hidden layer neural networks strongly motivates the investigation of constructive learning techniques as a means of realizing established error bounds. Learning characteristics employed by constructive algorithms provide ideas for development of new algorithms applicable to the function approximation problem. In addition, constructive techniques offer efficient methods for network construction and weight determination. The development of a novel neural network algorithm, the Constructive Locally Fit Sigmoids (CLFS) function approximation algorithm, is presented in detail. Basis functions of global extent (piecewise linear sigmoidal functions) are locally fit to the target function, resulting in a pool of candidate hidden layer nodes from which a function approximation is obtained. This algorithm provides a methodology of selecting nodes in a meaningful way from the infinite set of possibilities and synthesizes an n node single hidden layer network with empirical and analytical results that strongly indicate an O(1/n) mean squared training error bound under certain assumptions. The algorithm operates in polynomial time in the number of network nodes and the input dimension. Empirical results demonstrate its effectiveness on several multidimensional function approximate problems relative to contemporary constructive and nonconstructive algorithms.

Solutions of the evolution equation generally lie in certain Bochner–Sobolev spaces, in which the solutions may have regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework that shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner–Sobolev spaces. In our work, we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks. This activation function allows us to deduce approximation results of the neural networks while avoiding issues caused by the nonregularity of the most commonly used Rectified Linear Unit (ReLU) activation function.

In order to solve partial differential equations (PDEs) numerically, one first needs to approximate the field functions (such as the displacement functions), and then obtain the derivatives of the field functions (such as the strains), by directly differentiating the field functions. Using such direct-derivatives in formulating a numerical method is common and is used in the standard finite element method (FEM), but such models are often found to be “stiff”. In the weakened weak (W2) formulations, it is found that the use of properly re-constructed derivatives can be beneficial in ways because the model can become “softer”. This paper presents a novel “pick-out” theory and technique for re-constructing the derivatives (such as the strains) of functions defined in a local domain, using smoothing operations. The local domain can be a smoothing domain used in the smoothed finite element methods (S-FEMs), smoothed point interpolation methods (S-PIMs), and smoothed particle hydrodynamics (SPH). It is discovered that through the use of a set of linearly independent smoothing functions that are continuous in the local domain, one can simply pick out various orders of smoothed derivatives (at the center of a domain) from any given function that may discontinuous (strictly) inside the local domain. As long as the smoothing function is continuous in the smoothing domain, the picked out “smoothed derivatives” are equivalent (in a local integral sense) to the compatible direct-derivatives, which ensures the convergence of the smoothed model (such as the S-FEM) when the smoothing domains shrinking to zero. The pick-out technique can be used in strong, weak, local weak, weak-strong, or weakened weak formulations to create stable and convergent numerical models. It may offer a new window of opportunity to develop new effective numerical models using smoothed derivatives (strains) that are “softer” and can produce accurate solutions also in the derivatives (strains and stresses) of the field functions (displacements).

We propose an initialization method for feedforward artificial neural networks (FFANNs) trained to model physical systems. A polynomial solution of the physical system is obtained using a mathematical model and then mapped into the neural network to initialize its weights. The network can next be trained with a dataset to refine its accuracy. We focus attention on an elliptical partial differential equation modeled using a feedforward backpropagation network. We present a numerical example and compare our method with other initialization methods. Our method converges nearly 90% faster compared to random weights, with higher probability of convergence to an acceptable local minimum.

This paper proposes an asymmetric complex fuzzy inference system (ACFIS) that improves a conventional fuzzy inference system (FIS) in two ways. First, the proposed model uses the novel neural-net-like aim–object parts, making the model flexible, in terms of model size of parameters and terse asymmetric structure. Second, the enhanced complex fuzzy sets (ECFSs) are used to expand membership degree from a single real-valued state to complex-valued vector state. Hence, the ACFIS can have the ability to predict multiple targets simultaneously. In addition, a hybrid learning algorithm, combining the particle swarm optimization (PSO) and the recursive least-square estimator (RLSE), is utilized to optimize the proposed model. To test the proposed approach, we did experimentation on four-function approximation using one single model only with 10 repeated trails. Based on the experimental results, the ACFIS has shown excellent performance.

In this paper, a method for fuzzy modeling is presented. The framework of the method is the concept of Linguistic Integrity. The use of this framework present several advantages. The most important is transparency, this transparency can be exploited in two directions. The first direction is in data mining where the method can provide a linguistic relation (IF-THEN rules) among the variables. The second direction is to improve the completeness of a model by giving an easy interface to the user such that expert knowledge can be included. The algorithm starts from numerical data (input-output data) and generates a rule base with a limited number of membership functions on each input domain. The rules are created in the environment of fuzzy systems. The algorithm used for rule extraction is named (AFRELI).

If pattern generation is viewed as a stochastic process, which usually is appropriate for optical character recognition, speech recognition, and similar tasks, pattern classification can be based on the idea of function approximation. Starting from decision theory, it is shown in this chapter that minimum error decisions are obtained by the classifier if it approximates the a posteriori probabilities ruling the pattern source (Bayesian approach). Since for practical problems the properties of the stochastic pattern source are only implicitly given by the learning set, this set should truly represent the data population underlying the recognition task. Furthermore, the approximating functions should be taken from a family of functions which have — at least in principle — the power of approximating any desirable function (universal approximator). This contribution discusses in some detail the important paradigms for classification based on function approximation: polynomial classifier (including the linear approach) and multilayer perceptron, which are both global approximation schemes; radial basis functions, which are local approximators similar to nearest neighbor techniques; and finally classifiers which are based on certain distribution assumptions. All these paradigms, including different variants, are introduced based on a common notation, and special characteristics of the different approaches are pointed out using the classification of handwritten digits as a practical example.