Narrow Results

In multi-agent systems (MAS), the interactions and credit allocation among agents are essential for achieving efficient cooperation. To enhance the interactivity and efficiency of credit allocation in multi-agent reinforcement learning, we introduce a credit allocation for interactive multi-agents method (CAIM). CAIM not only considers the effects of various actions on other agents but also leverages attention mechanisms to handle the mismatch between observations and actions. With a unique credit allocation strategy, agents can more precisely assess their contributions during collaboration. Experiments in various adversarial scenarios within the SMAC benchmark environment indicate that CAIM markedly outperforms existing multi-agent reinforcement learning approaches. Further ablation studies confirm the effectiveness of each CAIM component. This research presents a new paradigm for enhancing collaboration efficiency and overall performance in MAS.

With the rapid development of unmanned aerial vehicles (UAVs) technologies, a substantial increase on the employ of UAV swarms in a wide range of civilian and military tasks has been witnessed. Advanced confrontation control approach can greatly improve UAVs’ capabilities and effectively free pilots from dangerous, boring, and burdensome confrontation missions. How to efficiently control UAV swarms in the air-to-air confrontation is still a hard problem. In this paper, considering the influence of the defending angle of UAV, we propose a general attacking cost function and an adaptive attacking strategy (AAS) to improve the capability of UAV swarm against another UAV swarm in an airborne battlefield. A multi-agent based UAV swarm air-to-air confrontation model is established, where red UAV swarm versus blue UAV swarm were simulated in a visual 3D and discrete-event environment. Extensive simulations are performed to verify the performance of AAS, the results show that AAS outperforms other traditional strategies by a large margin. In particular, UAV swarm adopting AAS can obtain a very high winning percentage even though the size of the swarm is only half of its opposing swarm that uses random or low velocity strategy. Meanwhile, AAS is quite robust to cope with different UAV swarm sizes. To improve the usability and practicability of AAS, we also propose a lightweight strategy called empirical adaptive attacking strategy (EAAS). The simulation results indicate that EAAS is easy to use and can retain the similar effects to AAS especially for large scale UAV swarms. Our work will illuminate new insights into the area of UAV swarm versus UAV swarm.

Results of multi-agent based simulations of the immune response to HIV during the acute phase of infection are presented here. The model successfully recreates the viral dynamics associated with the acute phase of infection, i.e., a rapid rise in viral load followed by a sharp decline to what is often referred to as a "set point", a result of T-cell response and emergence of HIV neutralizing antibodies. The results indicate that sufficient T Killer cell response is the key factor in controlling viral growth during this phase with antibody levels of critical importance only in the absence of a sufficient T Killer response.

The giant amoeboid organism true slime mould Physarum polycephalum dynamically adapts its body plan in response to changing environmental conditions and its protoplasmic transport network is used to distribute nutrients within the organism. These networks are efficient in terms of network length and network resilience and are parallel approximations of a range of proximity graphs and plane division problems. The complex parallel distributed computation exhibited by this simple organism has since served as an inspiration for intensive research into distributed computing and robotics within the last decade. P. polycephalum may be considered as a spatially represented parallel unconventional computing substrate, but how can this ‘computer’ be programmed? In this paper we examine and catalogue individual low-level mechanisms which may be used to induce network formation and adaptation in a multi-agent model of P. polycephalum. These mechanisms include those intrinsic to the model (particle sensor angle, rotation angle, and scaling parameters) and those mediated by the environment (stimulus location, distance, angle, concentration, engulfment and consumption of nutrients, and the presence of simulated light irradiation, repellents and obstacles). The mechanisms induce a concurrent integration of chemoattractant and chemorepellent gradients diffusing within the 2D lattice upon which the agent population resides, stimulating growth, movement, morphological adaptation and network minimisation. Chemoattractant gradients, and their modulation by the engulfment and consumption of nutrients by the model population, represent an efficient outsourcing of spatial computation. The mechanisms may prove useful in understanding the search strategies and adaptation of distributed organisms within their environment, in understanding the minimal requirements for complex adaptive behaviours, and in developing methods of spatially programming parallel unconventional computers and robotic devices.

An artificial stock market is established with the modeling method and ideas of cellular automata. Cells are used to represent stockholders, who have the capability of self-teaching and are affected by the investing history of the neighboring ones. The neighborhood relationship among the stockholders is the expanded Von Neumann relationship, and the interaction among them is realized through selection operator and crossover operator. Experiment shows that the large events are frequent in the fluctuations of the stock price generated by the artificial stock market when compared with a normal process and the price returns distribution is a Lévy distribution in the central part followed by an approximately exponential truncation.

Learning with very large-scale datasets is always necessary when handling real problems using artificial neural networks. However, it is still an open question how to balance computing efficiency and learning stability, when traditional neural networks spend a large amount of running time and memory to solve a problem with large-scale learning dataset. In this paper, we report the first evaluation of neural network distributed-learning strategies in large-scale classification over protein secondary structure. Our accomplishments include: (1) an architecture analysis on distributed-learning, (2) the development of scalable distributed system for large-scale dataset classification, (3) the description of a novel distributed-learning strategy based on chips, (4) a theoretical analysis of distributed-learning strategies for structure-distributed and data-distributed, (5) an investigation and experimental evaluation of distributed-learning strategy based-on chips with respect to time complexity and their effect on the classification accuracy of artificial neural networks. It is demonstrated that the novel distributed-learning strategy is better-balanced in parallel computing efficiency and stability as compared with the previous algorithms. The application of the protein secondary structure prediction demonstrates that this method is feasible and effective in practical applications.

For the limited transmission capacity and compressed images in the network environment, a compressed-domain image filtering and re-ranking approach for multi-agent image retrieval is proposed in this paper. Firstly, the distributed image retrieval platform with multi-agent is constructed by using Aglet development system, the lifecycle and the migration mechanism of agent is designed and planned for multi-agent image retrieval by using the characteristics of mobile agent. Then, considering the redundant image brought by distributed multi-agent retrieval, the duplicate images in distributed retrieval results are filtered based on the perceptual hashing feature extracted in the compressed-domain. Finally, weight-based hamming distance is utilized to re-rank the retrieval results. The experimental results show that the proposed approach can effectively filter the duplicate images in distributed image retrieval results as well as improve the accuracy and speed of compressed-domain image retrieval.

Supervisory Control and Data Acquisition (SCADA) system in the modern industrial automation control network is facing an increasing number of serious security threats. In order to meet the security defense requirements of oil and gas SCADA system, an intelligent security defense model based on multi-agent was designed by analyzing the security risks in oil and gas SCADA system and combining the advantages of multi-agent technology in distributed intrusion detection system. First, the whole structure of this model was divided into three layers: monitoring layer, decision layer and control layer. Then, the defense model was verified by C4.5 decision tree algorithm, and obtained a good result. Finally, the security defense prototype system of large-scale oil and gas SCADA system based on this model was realized. Results demonstrate that the application of multi-agent technology in the security defense of oil and gas SCADA system can achieve more comprehensive defense, more accurate detection, which can handle large-scale distributed attacks and improve the robustness and stability of security defenses. This study makes full use of the multi-agent architecture and has the advantage of accurate detection, high detection efficiency and timely response.

Under the actual combat background, prior information on radar targets has great uncertainty. The waveform designed based on prior information does not meet the requirements for the estimation of parameter. Thus, an algorithm for designing a waveform based on reinforcement learning is proposed to solve the above-mentioned problem. The problem on radar target parameter estimation is modeled as a framework for multi-agent reinforcement learning. Each frequency band acts as an agent, collectively interacts with the environment, independently receives observation results, shares rewards, and constantly updates the Q-network. The results of the simulation experiments indicate that the algorithm exhibits a significant improvement in terms of the mutual information obtained using the water injection method. In the case of simulation experiment, the SINR of the waveform designed based on multi-agent reinforcement learning is more than 3dB higher than that of LFM waveform. Under the condition of different time width and power, the mutual information obtained by the algorithm is better than that of water injection method. Moreover, such algorithm is also found to effectively improve the parameter estimation performance of target detection.

In the era of 5G/B5G, computing-intensive, delay-sensitive applications such as virtual reality inevitably bring huge amounts of data to the network. In order to meet the real-time requirements of applications, Mobile Edge Computing (MEC) pushes computing resources and data from the centralized cloud to the edge network, providing users with computing offload technology. However, the mismatch between the great computing requirements of computing-intensive tasks and the limited computing power of a single edge server poses a great challenge to computing offload technology. In this paper, a multi-agent cooperation mechanism for MEC and a routing mechanism based on deep reinforcement learning (DRL) are proposed. First of all, a multi-agent cooperation mechanism is proposed to realize the cooperative processing of computing-intensive and delay-sensitive applications, and the task unloading decision-making problem based on multi-agent cooperation is studied. Secondly, the cooperative processing of tasks by multi-agents involves data transmission. Considering the real-time requirements of tasks, this paper proposes an intelligent routing mechanism based on DRL to plan the optimal routing path. Finally, the simulation implementation and performance evaluation of the multi-agent cooperation mechanism and routing mechanism for MEC are carried out. The experimental results show that the intelligent routing mechanism based on DRL and graph neural network is superior to the comparison mechanism in terms of network average delay, throughput and maximum link bandwidth utilization. At the same time, the superiority of graph neural network in model generalization is verified on a new network topology National Science Foundation (NSF) Net. The results of route optimization are applied to the multi-agent cooperation mechanism, and the experimental results show that the mechanism is superior to the comparison scheme in terms of task success rate and average task response delay. The combination of these two mechanisms well solves the problem that it is difficult to deal with computing-intensive and delay-sensitive applications in mobile edge computing because of its limited resources.

Highly dynamic Internet of Vehicles spectrum sharing can share spectrum owned by vehicle-to-infrastructure links through multiple workshop links to achieve efficient resource allocation. Aiming at the problem that the rapid variations in channel states in highly dynamic vehicular environments can make it challenging for base stations to gather and manage information about instantaneous channel states, we present a multi-agent deep reinforcement learning-based V2X spectrum access algorithm. The algorithm is designed to optimize the throughput of V2I user under V2V user delay and reliability constraints, and uses the experience gained from interacting with the communication environment to update the Q network to improve spectrum and power allocation strategies. Implicit collaborative agents are trained through an improved DQN model combined with dueling network architecture and long short-term memory network layers and public rewards. With lagged Q-learning and concurrent experience replay trajectories, the training process was stabilized and the non-stationarity problem caused by concurrent learning of multiple agents was resolved. Simulation results demonstrate that our presented algorithm achieves a mean successful payload delivery rate of 95.89%, which is 16.48% greater than that of the randomized baseline algorithm. Our algorithm obtains approximately the optimal value and shows performance close to the centralized brute force algorithm, which provides a better strategy for further minimizing the signaling overhead of the Internet of Vehicles communication system.

A market mechanism is basically driven by a superposition of decisions of many agents optimizing their profit. The macroeconomic price dynamic is a consequence of the cumulated excess demand/supply created on this micro level. The behavior analysis of a small number of agents is well understood through the game theory. In case of a large number of agents one may use the limiting case that an individual agent does not have an influence on the market, which allows the aggregation of agents by statistic methods. In contrast to this restriction, we can omit the assumption of an atomic market structure, if we model the market through a multi-agent approach.

The contribution of the mathematical theory of neural networks to the market price formation is mostly seen on the econometric side: neural networks allow the fitting of high dimensional nonlinear dynamic models. Furthermore, in our opinion, there is a close relationship between economics and the modeling ability of neural networks because a neuron can be interpreted as a simple model of decision making. With this in mind, a neural network models the interaction of many decisions and, hence, can be interpreted as the price formation mechanism of a market.

Distillations utilize multi-agent based modeling and simulation techniques to study warfare as a complex adaptive system at the conceptual level. The focus is placed on the interactions between the agents to facilitate study of cause and effect between individual interactions and overall system behavior. Current distillations do not utilize machine-learning techniques to model the cognitive abilities of individual combatants but employ agent control paradigms to represent agents as highly instinctual entities. For a team of agents implementing a reinforcement-learning paradigm, the rate of learning is not sufficient for agents to adapt to this hostile environment. However, by allowing the agents to communicate their respective rewards for actions performed as the simulation progresses, the rate of learning can be increased sufficiently to significantly increase the teams chances of survival. This paper presents the results of trials to measure the success of a team-based approach to the reinforcement-learning problem in a distillation, using reward communication to increase learning rates.

This paper investigates the impact of reward shaping in multi-agent reinforcement learning as a way to incorporate domain knowledge about good strategies. In theory, potential-based reward shaping does not alter the Nash Equilibria of a stochastic game, only the exploration of the shaped agent. We demonstrate empirically the performance of reward shaping in two problem domains within the context of RoboCup KeepAway by designing three reward shaping schemes, encouraging specific behaviour such as keeping a minimum distance from other players on the same team and taking on specific roles. The results illustrate that reward shaping with multiple, simultaneous learning agents can reduce the time needed to learn a suitable policy and can alter the final group performance.

This paper, describes an agent based model of the spreading of a population over a territory. The models aims at reproducing a distribution of settlements with statistical and spatial characteristics similar to a historically produced pattern. The model operates on a representation of a real territory, taking into account hydrography and relief. The two main goals are to obtain a rank size distribution of the size of settlements which corresponds to a power law (also known as the Zipf Law of settlements) and to place the settlements in the territory in patterns that are close to the real ones, in zones where settlements were the result of a long historical process. The goal of the project was to demonstrate that a set of relatively simple rules could produce a complex pattern, similar to the result of a long and complex historical process. Therefore, it is an assumed reductionist approach.

Our conclusions show that a simple territorial logic, taking into account the quality of land, accessibility, population growth and migration preferences could reproduce Zipf distributions and interesting patterns of agent flow among the settlements created. However, achieving spatial patterns closer to the historical record needs an extra dimension involving field of sight. The best results were achieved by creating an artifical population which chooses to create settlements in places where a wide field of view exists of quality territory.

In order to react to volatile market demand and uncertain production objectives, distributed manufacturing resources have to be dynamically configured. However, the dynamics of process planning and shop floor information makes shop floor reconfiguration a challenging problem. The shortcomings of the existing research on shop floor reconfiguration are first discussed. Then a multi-agent-based framework for dynamic shop floor reconfiguration is presented, and a mathematical programming model taking process planning into consideration is constructed as well. To coordinate the resource assignment among agents, a cooperative coevolutionary algorithm is also put forward to find the optimal solution of the reconfiguration model. The advantages of the proposed model are using combined multi-agent and mathematical programming method to decompose and optimize the problem of reconfiguration and considering alternative process plans. Furthermore, a prototype system for dynamic shop reconfiguration is developed. The results of this research will help solve the problem of shop floor reconfiguration with complex and dynamic interactive structure.

Trustworthy and reliable applications built using intelligent software agents aim to provide improved performance using its characteristics. Agents introduced in various architectures represent its functionality as functional elements of the architecture and shows the interaction between other components present in the architecture. The Internet of things (IoT) reveals as a frequent technology that allows accessing the physical objects present in the world. IoT systems utilize wireless sensor network to transmit and receive data by establishing communication. Wireless Sensor Networks transmits digital signals to the cyber-world for analyzing and processing the information into useful data by either formulating or communicating with the intelligent and innovative system. While talking about IoT and WSN, agents introduced in such environments assist in making decisions quickly by perceiving the input from the environment. The number of agents needed for an application depends upon the complexity of the problem. Multi-Agent architectures discussed in the article describe their association, roles, functionality and interaction. This paper gives a detailed survey of various agent/multi-agent learning architectures introduced over IoT and WSN. Moreover, this survey with the performance and the SWOT analysis on the Agent-based learning architecture helps the reader and paves a way to pursue research on Agent-based architectural deployment over IoT and WSN paradigms.

Knowledge of the structural class of a given protein is important for understanding its folding patterns. Although a lot of efforts have been made, it still remains a challenging problem for prediction of protein structural class solely from protein sequences. The feature extraction and classification of proteins are the main problems in prediction. In this research, we extended our earlier work regarding these two aspects. In protein feature extraction, we proposed a scheme by calculating the word frequency and word position from sequences of amino acid, reduced amino acid, and secondary structure. For an accurate classification of the structural class of protein, we developed a novel Multi-Agent Ada-Boost (MA-Ada) method by integrating the features of Multi-Agent system into Ada-Boost algorithm. Extensive experiments were taken to test and compare the proposed method using four benchmark datasets in low homology. The results showed classification accuracies of 88.5%, 96.0%, 88.4%, and 85.5%, respectively, which are much better compared with the existing methods. The source code and dataset are available on request.

In this paper, we propose an evacuation support system that provides evacuation routes in case of disasters, and verify the usefulness of the system. Current popular wireless communication infrastructure is supported by a series of base stations and one base station handles a lot of communication. Therefore, when communication base stations break down due to disasters such as an earthquake, it may become difficult for people to use their smartphones based on the Internet. When the communication infrastructure is paralyzed, people will have great difficulty collecting information about the conditions of transportation systems and about the safety of family and friends using smartphones. Our proposed system addresses this problem by using multiple kinds of mobile agents in addition to static agents on smartphones that use a mobile ad hoc network (MANET). The proposed system collects information with mobile agents and diffuses information via mobile agents so that the system is able to provide an optimized evacuation route for each user in a dynamically changing disaster environment. In this paper, we extend our previously proposed evacuation support system to consider elevation information when constructing evacuation routes. When a tsunami or a flood tide occurs, low elevations may be under water. Therefore, this revised evacuation support system helps people to move to safer places by selecting higher elevation routes when warranted by the situation.

An application-oriented homogeneous control protocol is proposed to solve the consensus problem of multi-agent system (MAS) modeled by higher-order integrator. This nonlinear control protocol is able to homogenize the linear system with a special degree called homogeneity degree. This homogeneous control protocol ensures asymptotically/finite-time stable multi-agent systems (MASs) (or fixed-time attractive to compact sets containing the origin) by selecting different homogeneity degrees. If the linear control protocol for each agent is provided, the proposed nonlinear homogeneous control protocol in this paper can be easily implemented without requiring any tuning of the control parameters. A bounded homogeneous control protocol, which is a special form of the controller proposed, is also introduced to address the same problem with input constraints. Finally, numerical simulations are conducted to demonstrate the effectiveness of the proposed approach.