Narrow Results

Small Unmanned Aircraft Systems have grown in autonomy and capability and continue to complement Department of Defense mission objectives. Teaming unmanned aircraft with manned vehicles can expand mission profiles and reduce risk to human life. To fully leverage unmanned systems, vehicles must be efficient and autonomous in path planning development. The work herein explores direct orthogonal collocation optimal control techniques combined with fast geometric path planning algorithms to reduce computation time and increase solution accuracy for small unmanned aircraft systems path planning missions. Previous work in the two-dimensional plane demonstrated a methodology to provide optimal flight paths through defined simplex corridors and simplified the optimal control parameter bounds by formulating the problem in the barycentric coordinate system. These methodologies are extended in this paper for three-dimensional flight and are solved with two different formulations for flight in an urban environment. The first formulation solves the constrained optimal control problem using a single phase while modeling the building constraints with superquadric functions. The second formulation implements the simplex methodology, eliminating polygonal constraints from the search domain, and solving the optimal path in a multiple phase approach. Results illustrate the benefits gained in computation time and accuracy when implementing simplex methods into the optimal control design and provide a foundation for closing the gap to real-time, onboard operations for unmanned vehicle path planning.

Time-optimal model predictive control is important for achieving fast racing drones but is computationally intensive and thereby rarely used onboard small quadcopters with limited computational resources. In this work, we simplify the optimal control problem (OCP) of the position loop for several maneuvers by exploiting the fact that the solution resembles a so-called ‘bang-bang’ in the critical direction, where only the switching time needs to be found. The noncritical direction uses a ‘minimum effort’ approach. The control parameters are obtained through bisection search schemes on an analytical path prediction model. The approach is compared with a classical PID controller and theoretical time-optimal trajectories in simulations. We explain the effects of the OCP simplifications and introduce a method of mitigating one of these effects. Finally, we have implemented the ‘bang-bang’ controller as a model predictive controller (MPC) onboard a Parrot Bebop and performed indoor flights to compare the controller’s performance to a PID controller. We show that the light novel controller outperforms the PID controller in waypoint-to-waypoint flight while requiring only minimal knowledge of the quadcopter’s dynamics.

This paper briefly reviews the dynamics and the control architectures of unmanned vehicles; reinforcement learning (RL) in optimal control theory; and RL-based applications in unmanned vehicles. Nonlinearities and uncertainties in the dynamics of unmanned vehicles (e.g. aerial, underwater, and tailsitter vehicles) pose critical challenges to their control systems. Solving Hamilton–Jacobi–Bellman (HJB) equations to find optimal controllers becomes difficult in the presence of nonlinearities, uncertainties, and actuator faults. Therefore, RL-based approaches are widely used in unmanned vehicle systems to solve the HJB equations. To this end, they learn the optimal solutions by using online data measured along the system trajectories. This approach is very practical in partially or completely model-free optimal control design and optimal fault-tolerant control design for unmanned vehicle systems.

In this paper, a task assignment method is proposed to deal with the multi-agent pursuit-evade game for heterogeneous unnamed aerial vehicles via reinforcement learning. The mathematical model based on the local position error dynamics is established to describe the interactions among the vehicles in the pursuit-evade game, subject to high nonlinearities and parameter uncertainties involved in the vehicle model. The execution costs and the corresponding optimal control policies of the agent pursuing each target are calculated, and the policy with minimum execution cost is determined as the objective of the multi-agent pursuit-evade game. Min-Max strategy is introduced to estimate and counteract the interaction effects in the mathematical model, and the reinforcement learning-based algorithm is proposed to obtain the optimal solution to the assignment problem based on the Hamilton–Jacobi–Bellman equation without the interaction effects. Simulation results are given to show the effectiveness of the proposed task assignment method.