Frontiers of Intelligent Control and Information Processing

The following sections are included:

• Preface
• Contents

The chapter develops a new class of multi-agent discrete-time dynamic games, known as dynamic graphical games. In graphical games, the information flow between agents is prescribed by a communication graph structure. For that reason a local performance index is defined for each agent that depends only on the local information available to this agent. This chapter brings together Discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, game theory, and reinforcement learning techniques to formulate and solve these multi-agent graphical games. Ideas from cooperative control are used to attain synchronization among the agents’ dynamics to the leader dynamics. The chapter lays the mathematical foundation to develop adaptive learning solutions for the dynamic graphical games. Novel coupled Hamilton Jacobi equations and coupled Bellman equations are developed for this class of the dynamic graphical games. The relationship between the coupled Hamilton Jacobi equations and the coupled Bellman equations is shown. The graphical game Bellman equations are shown to be equivalent to certain graphical game Hamilton Jacobi Bellman equations developed herein. Nash solutions are given in terms of solutions to a set of coupled discrete-time Hamilton-Jacobi-Bellman equations. Novel coupled Riccati recursions are developed for the dynamic graphical game. Reinforcement learning value iteration algorithms are developed to solve the dynamic graphical games online in real-time. The developed multi-agent value iteration algorithms learn the Nash solution online in real-time. Value iteration convergence proofs are provided. Approximate Dynamic Programming techniques namely Heuristic Dynamic Programming and Dual Heuristic Dynamic Programming are developed and extended to solve the dynamic graphical games. Online and offline adaptive learning structures are implemented using actor-critic network structures to solve these dynamic graphical games.

In this chapter, a reinforcement-learning-based online adaptive control is developed to deliver a desired tracking performance for a class of discrete-time (DT) nonlinear systems in the presence of unknown bounded disturbances. We study multi-input-multi-output unknown nonaffine nonlinear DT systems and employ two neural networks (NNs): an action NN is applied to generate the control signal and it is also designed to cancel the nonlinearity of unknown DT systems, and a critic NN is utilized to estimate the cost function, which satisfies the recursive equations derived from heuristic dynamic programming. The weights of both the action NN and the critic NN are directly updated online instead of preliminarily offline training. Based on Lyapunov’s direct method, the overall adaptive scheme guarantees that all signals involved in the closed-loop system are uniformly ultimately bounded and the tracking errors can converge to a small neighborhood of zero. Simulation results are presented to demonstrate the effectiveness of the developed approach.

Adaptive dynamic programming (ADP) has been a popular approach to seek the optimal control strategy in Markov decision process (MDP). Generally, this type of approach requires a complete set of system information/states to achieve the online optimal decision-making. However, the full system information/states are not usually available in practical situations. In many cases, the measured input/output data can only represent part of the system information and the system internal states are not available. In this chapter, we investigate a data-driven heuristic dynamic programming (HDP) architecture to tackle the partially observed Markov decision process (POMDP). In specific, we include a state estimator neural network to recover the full system information for the action network, so that the optimal control policy can still be achieved under the partially observed environment. We randomly initialize the weights in the state estimator network, and conduct online learning for the entire process. Both discrete-time and continuous-time system functions are tested. Simulation results and system trajectories justify the control performance of our proposed approach.

In this part, we focus on the online reinforcement learning for continuous state-action systems. Compared to offline problem, online case lacks of system dynamics. And continuous state-space problems are more difficult than finite states problem, as the number of states is infinite. Two methods are involved here, respectively RLSPI (Recursive Least-Square Policy Iteration) and MSEC (Multi-Samples in Each Cell). Both of them have their advantages and drawbacks. RLSPI combines LSPI (Least-Squares Policy Iteration) with continuous policy parametrization and RLS (Recursive Least-Squares). So it can be applied to continuous-action systems. It has simplicity of implementation and low cost of computation. But RLSPI uses part of observed samples in the past, so its utilization of samples is still not sufficient. Besides, RLSPI adopts ε-greedy exploration technique, which has low efficiency and selects explorative actions randomly. MSEC overcomes the drawbacks of RLSPI. It utilizes samples efficiently so that the algorithm can be sure to achieve a policy that is optimal or near-optimal to the agent. And it adopts PAC (Probably Approximately Correct) principle to explore, so its time bound is polynomial in some parameters of the system. Examples of both two algorithms are simulated to reveal their performance.

In this chapter, a novel adaptive iterative learning algorithm is proposed for a class of constraint robotic manipulators with uncertainties and external disturbances. The controller design is based on the reduced form of the robot model. The uncertain parameters are estimated in the time domain whereas the repetitive disturbances is compensated in the iteration domain. With the adoption of saturated learning, all the signals in the closed loop are guaranteed to be bounded. By constructing a Lyapunov-Krasovskii-like composite energy function, the proposed controller achieves both control objectives in the presence of dynamic parameter uncertainty. We prove the states of the closed system asymptotically converge to the desired trajectory while ensuring the constrained force remains bounded. Simulation results show the effectiveness of the proposed algorithm.

Stochastic adaptive control of a nonlinear system enclosed by a communication network or referred to as a nonlinear networked control system (NNCS) is a challenging problem due to the presence of unknown network imperfections such as network-induced delays and packet losses. Moreover, the known system dynamics of the original nonlinear system become uncertain and stochastic after the incorporation of the network imperfections due to the communication network within the feedback loop. Therefore, first, a novel NNCS representation incorporating the system uncertainties and network imperfections are derived in this chapter. Subsequently, an online neural network (NN) identifier is developed to identify the control coefficient matrix of the stochastic nonlinear discrete-time system for the purpose of the controller design. Further, critic and action NNs are proposed along with identified system dynamics to determine time-based stochastic optimal adaptive control of NNCS in a forward-in-time manner. Lyapunov stability theory is utilized to demonstrate that all the closed-loop signals are uniformly ultimately bounded (UUB) in the mean. Eventually, to reduce the network traffic between the nonlinear system and the controller, an emerging event-triggered control scheme is developed for the NNCS. The performance of both the controllers is contrasted via simulation.

Model predictive control is an optimization-based control strategy which has achieved enormous successes in numerous real-world applications. MPC generates control signals by means of real-time optimization of a performance index over a finite moving horizon of predicted future, subject to system constraints. A major challenge of the MPC research and development lies in the realization of nonlinear and robust MPC approaches, especially to cases where unmodeled dynamics exist. This chapter presents novel MPC approaches to nonlinear systems with unmodeled dynamics based on neural networks. At first, a nonlinear system with unmodeled dynamics is decomposed by means of Jacobian linearization to an affine part and a higher-order unknown term. The linearization residues, together with the unmodeled dynamics, are then modeled by using a feedforward neural network called the Extreme Learning Machine via supervised learning. When the controlled system is affected by bounded additive disturbances, the minimax methodology is exploited to achieve robustness. The nonlinear and robust MPC problems are formulated as constrained convex optimization problems and iteratively solved by applying neurodynamic optimization approaches. The applied neurodynamic optimization approaches can compute the optimal control signals in real-time, which shed a light for real-time implementability of MPC technology. Simulation results are provided to substantiate the effectiveness and characteristics of the proposed approaches.

A packet-based communication and control co-design framework is proposed for Networked Control Systems (NCSs). This framework takes advantage of the characteristic of the packet-based transmission in the networked control environment, which enables a sequence of control signals to be sent over the network simultaneously within one data packet. This consequently makes it possible to actively compensate for the communication constraints in NCSs with specially designed compensation mechanisms, which can not be achieved by conventional control approaches. These compensated communication constraints include all the major ones brought by the communication network to NCSs, i.e., the network-induced delay, data packet dropout and data packet disorder, thus making the packet-based co-design approach a unified framework for NCSs. Following the design of the packet-based framework, the resulting control system is mathematically formulated, its closed-loop stability is analyzed, and a receding horizon controller is designed to implement the scheme. Finally, the effectiveness of the co-design scheme is verified by numerical examples as well as an Internet-based test rig. It is believed that this packet-based communication and control co-design framework is an important step towards the convergence of control, communication and computation in the new era of the information technology.

Privacy preserving computational models have become an important research area due to the increasingly widespread usage of sensitive data in networked environments, as evidenced by distributed computing applications and game-theoretic settings (e.g., auctions). Over the years computer scientists have explored many quantifications of privacy in computation. Much of this research focused on designing perfectly privacy-preserving protocols, i.e., protocols whose execution reveals no information about the parties’ private inputs beyond that implied by the outcome of the computation. Unfortunately, perfect privacy is often either impossible, or infeasibly costly to achieve (e.g., requiring impractically extensive communication steps). To overcome this, researchers have also investigated various notions of approximate privacy. In this chapter, we review a few such notions and known results for them.

The ability to detect online the occurrence of concept drift in a datastream is a hot research topic with striking implications. In addition to applications such as network traffic monitoring and credit-card fraud detection, the issue is relevant in several monitoring-based and big data applications. This chapter describes a change-detection approach based on encoding-decoding machines, which are computationally simple, yet effective mechanisms for detecting concept drift in a datastream. As an illustrative example we describe a linear machine based on SVD and contrast the change-detection capabilities with a linear predictive modeling-based inspection. The proposed solution can be easily embedded in intelligent sensors and adopted in sensor networks where the change-detection outcomes of several units can be aggregated in a cognitive level.

It is a challenge to achieve a satisfactory rate for the sEMG pattern recognition, which is becoming the main focus of the on-going research in rehabilitation and prosthetics. This chapter introduces nonlinear feature extraction and nonlinear classification approaches to efficiently identify different human hand manipulations based on surface electromyography (sEMG) signals. The recurrence plot is employed to represent dynamical characteristics of sEMG during hand movements as nonlinear features. Fuzzy Gaussian Mixture Models (FGMMs) are proposed and employed as a nonlinear classifier to recognise these hand grasps and in-hand manipulations captured from different subjects. Results from a variety of experiments comparing 14 individual features, 19 multi-features and 4 classifiers demonstrate the proposed nonlinear measures provide essential supplemental information to the good performance in multi-features. It also proves that FGMMs with capacity of modelling nonlinear datasets outperform commonly used approaches including Linear Discriminant Analysis (LDA), Gaussian Mixture Models (GMMs) and Support Vector Machine (SVM). Specially, the best performance with the recognition rate of 96.7% is achieved by FGMMs with the multi-feature combining Willison Amplitude (WAMP) and Determinism (DET).

Although Smart Grids may represent the solution to the limits of nowadays Power Grid, the turnover may not occur in the next future yet due the complex nature of energy distribution. Thus, as a more short term effort, to improve the responsiveness of the energy demand to the power grid load, more and more energy providers apply dynamic pricing schemes for grid users. Believing that dynamic pricing policies may be an effective asset even at a micro-grid level, an hybrid energy management scheme is proposed in this contribution. While the nonlinear nature of a micro grid, involving the task allocation and the thermal constraint satisfaction, can be modeled as a mixed integer nonlinear programming problem, neural-network forecasting abilities can provide a sustainable support under realistic operating conditions. Based on the forecast of solar energy production and grid energy prices and outdoor temperature, the optimization of tasks allocation is aimed to lower both the user costs and the grid burden while accounting the thermal comfort of the user. Through computer simulations, whose degree of realism is enhanced by the adoption of forecast data, the shift of the grid load towards low energy price hours is confirmed.

In recent years, the design and evaluation of MOEAs, specifically for many-objective optimization problems, have been gaining an increasing attention because a large number of real-world problems involves simultaneously optimizing multiple conflicting objectives (in most cases, more than five) which are commonly referred to as many-objective optimization problems (MaOPs). Evolutionary algorithms have been successfully exploited to solve multiobjective optimization problems. In literature, a heuristic approach is often taken. For a chosen benchmark problem with specific problem characteristics, the performance of multiobjective evolutionary algorithms (MOEAs) is evaluated via some heuristic chosen performance metrics. The conclusion is then drawn based on statistical findings given the preferable choices of performance metrics. The conclusion, if any, is often indecisive and reveals no insight pertaining to specific problem characteristics that the underlying MOEA could perform the best. In this chapter, we introduce an ensemble method to compare six state-of-the-art MOEAs designed specifically for many-objective optimization problems under a number of carefully crafted benchmark problems, by combining a number of performance metrics using double elimination tournament selection. The double elimination design allows characteristically poor performance of a quality algorithm to still be able to win it all. Experimental results show that the proposed metrics ensemble can offer a more comprehensive comparison among various MOEAs than what could be obtained from single performance metric alone. The end result is a ranking order among all chosen MOEAs, but not quantifiable measures pertaining to the underlying MOEAs.

This chapter investigates the synchronization control problem of memristive chaotic circuits and their applications in image encryptions. First, several memristive chaotic circuits such as the memristive Chua’s circuit, and memristive Lorenz circuit are designed. Then, considering the state-dependent properties of the memristor, a new fuzzy model employing parallel distributed compensation (PDC) gives a new way to analyze the complicated memristive circuits with only two subsystems. Several examples are also given to illustrate the effectiveness and potential applications in image encryption of the results.

Twin support vector machine (TSVM) was proposed recently as a novel binary classifier which aims to seek a pair of nonparallel planes such that each one is closest to the samples of its own class and is at least one distance far from the samples of the other class. However, the samples of the other class with distances to such plane larger than one are discarded, thus implying TSVM fails to exploit the data information residing in those samples. Based on this observation, we extend the original TSVM and propose a novel graph embedded total margin twin support vector machine (GTM-TSVM) in this chapter. The central idea of GTM-TSVM is the plane of one class is required to be far away from overall samples of the other class. Moreover, the intra-class and inter-class graphs which respectively characterize the proximity relationships between samples of within and between classes are embedded into GTM-TSVM formulation so as to exploit the underlying manifold structure of data. The nonlinear classification with kernels is also studied. The experimental results on several publicly available benchmark data sets confirm the feasibility and effectiveness of the proposed method.

When Gaussian Mixture Model (GMM) is used for classification for small sample problem with high dimension, the estimation of the sample covariance matrix will be singular, which can lead to lower classification accuracy or can not achieve credible results. KLIM is a regularization method proposed for solving the problem. However, the regularization parameter in the KLIM is single and fixed which makes it suitable for a certain class of problems. In this chapter, under the framework of minimum description length (MDL) principle, the extensions of KLIM are investigated. One is the covariance matrix estimation with multi-regularization parameters, and the other is the covariance matrix estimation with variable regularization parameters. In the developed methods, based on MDL principle, regularization parameters are selected by the criterion of minimization the KL divergence and approximated efficiently by second-order Taylor expansion. The feasibility and effectiveness of developed methods are validated with experiment studies.

We use this chapter to illustrate the correlation analysis for big data based on evolutionary algorithms. Since there exists a huge amount of information in big data, traditional methods which process and analyze the database based on causality is powerless in dealing with big data. To exhibit the influences of data volume to methods, numerical simulations with low and high dimensional benchmarks are employed, which points out that the performances of algorithms with small data is not scalable to evaluate the performances in the cases of high-dimensional problems. Besides, to overcome the disadvantage that there is no “one-fits-all” evolutionary algorithm for various problems, we design a framework of hyper evolutionary algorithms and the simulation results validate the feasibility and effectiveness of the framework. An example of business intelligence to conduct the correlation analysis for big data is presented and based on the evolutionary algorithms demonstrates that the evolutionary algorithm is useful and helpful for e-business to make correct judgements for commodities storage, transfer and sales.

The following section is included:

• Index