Control and Estimation in Dynamic System Networks A Decentralized Perspective from Multiple UAV Coordination1

Gokhan Inalhan Department of Aeronautics & Astronautics Stanford University, Stanford, CA 94305

KEYWORDS
Decentralized Optimization, Nash Equilibrium, Dynamic System Networks

ABSTRACT
Control and estimation system design for interconnected autonomous systems has been a major research area of development in large-scale dynamic systems and communication networks. Specifically, with advances in nano-technology, such networks can span numerous devices with spatial and temporal differences working under limited centralized coordination and information. For such systems, traditional approach to control and estimation structures result in distributed structures decentralized over fixed coordination and communication links. However, in the case of reconfigurable large-scale networks, these two major assumptions indeed present the main challenge. To remedy these challenges, we present an overview of a new decentralized optimization concept. The methodology embeds the inherent nature of the problem which evolves across independent decision makers with incomplete mathematical models. The results are illustrated using a multiple unmanned air vehicle (UAV) coordination system.

1. INTRODUCTION
Our work focuses on a decentralized optimization methodology for a general class of interconnected autonomous systems governed by multiple decision makers. Control and estimation system design for such interconnected autonomous systems [1] has been a major research area in large-scale dynamic systems and communication networks [2]. Recent advances in nano-technology, resulted in development of reconfigurable networks that span numerous devices with spatial and temporal differences. A feature of such systems is that, information on state, dynamics and operational constraints is distributed throughout the system. To account for this distribution of information, traditional approaches considered distributed control and estimation structures, decentralized over known coordination and communication structures [1].
This work is a part of ongoing Ph.D. research under guidance and supervision of Prof. Claire J. Tomlin and is supported by Office of Naval Research under Grant N00014-00-1-0637. A subset of these results has appeared in Stanford University Department of Aeronautics & Astronautics Report SUDAAR 759 in February 2003.
1

However, prior to design, this requires global knowledge of all the components and is naturally limited to static interconnection structures which dont dynamically evolve. However, in the case of reconfigurable large-scale networks, these two major assumptions present the main challenge. To remedy these challenges, a new decentralized optimization concept is presented. The methodology embeds the inherent nature of the problem which evolves across independent decision makers with incomplete mathematical models. For a complete mathematical treatment of the subject and the optimization methodology, we refer the readers to the published report and journal manuscript [3, 4]. In this work, we present an overview of these results. This paper is organized in two main sections. First, we present the multiple unmanned air vehicle safety coordination problem and the associated mathematical model. This problem serves as the example on which the decentralized optimization methodology is illustrated. In the second section, we provide some numeric results on various four vehicle safety coordination problems using the computational test-bed that was developed.

2. MULTIPLE UNMANNED AIR VEHICLE (UAV) COORDINATION STRUCTURE

Our interest in a decentralized methodology stems from the effort to design multiple unmanned air vehicle (UAV) control systems which are much more flexible and responsive than a centralized paradigm allows. In many applications of Unmanned Aerial Vehicles (UAVs) such as distributed sensing, imaging, or reconnaissance, autonomous operation with limited ground support is desired. This is seen both for civilian (agriculture, weather sensing) or military purposes. However, of particular importance to military applications is the replacement of traditional complex aircraft fighter systems with an array of simpler and lower-cost UAVs that improves flexibility and offers a high degree of redundancy and reconfigurability in the event of a vehicle failure.

Safety Assurance

V1

Interconnection Constraints V1 V2 V3 Notion of Neighborhood V3

Communication Zone

V2

V4

V4

Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

FIG.1 DYNAMICAL SYSTEM NETWORKS ELEMENTS OF INCOMPLETE MATHEMATICAL MODELS AND DYNAMIC INTERCONNECTIONS A natural solution to such problems would utilize the distributed computational power where each aircraft has more responsibility over their own path planning and control routines; and thus creating a decentralized control paradigm in which each aircraft transmits its own information to a neighborhood, receives information about the aircraft in its neighborhood, and plans a path for itself accordingly. This concept is illustrated in Fig. 1 where limited communication zones result in incomplete mathematical knowledge about vehicles within a neighborhood. Considering that the vehicles operate in common airspace, a major aspect of coordination would be to avoid collisions by not entering into each others safety zone. In this work, we consider this problem as the main example to illustrate the decentralized optimization framework. In the next section, we provide the underlying mathematical structure which captures the mathematical model of each independent aircraft and the local optimization structures interconnected through collisionavoidance constraints. 2.1 MATHEMATICAL MODEL AND DECENTRALIZED COORDINATION ALGORITHM In our analysis, each aircraft is modeled as a nonlinear discrete-time dynamic system. Specifically, for ease of computation, we use a two-dimensional kinematics motion in (x,y) plane where velocity u, and angular turn rate w are the control inputs. This dynamics is given under equality constraints of the form hd. In addition, to provide a realistic operational characteristic, we impose input constraints for minimum and maximum velocity and turn rates which is illustrated by constraints ge. The dynamics and input constraints are embedded under a local optimization where a possible cost function can be described via quadratic deviation from an initial flight plan. The coupling in the system arises from the common (or conflicting) objectives and constraints between each subsystem. In this case, the coupling is through the minimum safety assurance zone circle with radius Rmin illustrated by constraints of the form gs. For example, under this structure, the local optimization for aircraft number one takes the form of Equation 1.

(1) Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

As each aircraft has control over its own flight path, each local optimizations should be able to work in an unfeasible mode. To account for this, we formulate the optimization problem under a penalty function form, where instead of constraints being penalized, local cost function is penalized. This specifically allows us to overcome the unbounded cost functions properties (which dont exist in standard distributed optimization techniques) and show global decentralized convergence and decentralized optimality of the system. [3, 4]

Decentralized Coordination Algorithm

SET TIMER=0
WHILE TIMER<T_ITER REPEAT
INITIALIZE ASSUMING ORIGINAL FLIGHT PATHS OF ALL VEHICLES

RECEIVE FLIGHT SCENARIOS FROM OTHER A/C

RECEIVE

V1
OPTIMIZE INDIVIDUAL FLIGHT PLAN FOR A GIVEN FLIGHT SCENARIO
STORE SOLUTION
X-LINK COMMUNICATION

V2
V3
V4

TIMER >= T_ITER

SEND

SELECT A FLIGHT SCENARIO FOR THE SET OF AVAILABLE SOLUTIONS

SEND FLIGHT SCENARIO TO OTHER A/C

FIG.2 DECENTRALIZED COORDINATION ALGORITHM SYSTEM SCHEMATIC FOR UAV COORDINATION. The decentralized coordination algorithm as captured in Fig. 2 is structured around local optimization in the penalty function form. The system evolves as a natural bargaining scheme where flight scenarios from vehicle within the neighborhood are received, the local penalty parameter is chosen. For this parameter, the cost function is optimized. Working in a timer mode, from all the available solutions received in unit process time, one is selected via arbitrary local selection criteria

Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

and transmitted to vehicles within the neighborhood. This indeed emulates a standard bargaining structure. In our analysis, we have shown that this algorithm based on sequential local optimizations is globally convergent. Under assumptions of differentiability and linear independence constraint qualification, we show that the method results in global convergence to epsilon-feasible Nash equilibria that satisfy the Karush-Kuhn-Tucker necessary conditions for Pareto-optimality. Analysis of the second order sufficiency conditions, provide insight to structures and solutions with strong local convexity or weak interconnections which guarantee Pareto-optimality. [3, 4]

3. NUMERICAL IMPLEMENTATION
As a part of this framework, we have developed a computational test-bed for rapid prototyping and analysis of coordination algorithms. The decentralized coordination network allows two MATLAB processes to run and communicate with each other using standard TCP-IP network protocol. We utilize the demo version of RBNB Matlink libraries provided by Creare Inc. [5] to publish and subscribe time stamped data through a hybrid network architecture with different types and rates of data sources and sinks.

FIG. 3 - DECENTRALIZED COORDINATION NETWORK AND A SCREEN-CAPTURE FROM THE SETUP. Fig. 3 captures our initial setup where a formation of four aircraft and a world model run the decentralized coordination algorithm where the individual aircrafts publish/subscribe local iterations (flight paths i.e. (x, y) for the given time horizon) and the local cost decrease to the world model in an asynchronous fashion.

Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

FIG. 4 - DYNAMICALLY CHANGING NETWORKS UNDER LIMITED COMMUNICATION RADIUSES UAV COORDINATION EXAMPLE This computational tool can provide a realistic assessment of our coordination algorithms under real-world situations such data delays, communication losses and global clock differences in localized vehicles. In addition, utilizing the world-model, we can capture dynamically changing networks as illustrated in Fig. 4. In this particular example, each aircraft `connect' and `disconnect' to the environment captured in local optimizations which dynamically change in time as the vehicles come in or leave close proximity of each other. The black connection lines correspond to the coordination link between aircraft. As the vehicles move around in time, other vehicles are identified to be in close proximity either via sensors and communication links. This results in new connections to appear (or disappear if the vehicles leave local proximity). 3.1 FOUR UAV COORDINATION EXAMPLE We have extensively tested the numerical implementation of the algorithm for a range of four-UAV safety coordination maneuvers. The example presented within is a general flight scenario, where the vehicles have to coordinate to satisfy the minimum separation constraint. Here, the imperfect communication channels are modeled via binary erasure channels where p1=0.7, p2=0.8, p3=0.6, p4=0.9. Fig. 5a shows the result of the decentralized optimization algorithm, where four UAVs change their initial straight flight routes to avoid collision conflict at the end bargaining process which involves sequential optimization and message passing. In this example, each vehicle tries to minimize its deviation from its local initial flight path while satisfying the minimum separation requirements. Fig.6a, shows that at the end of the local optimization and message passing process, the vehicles converge to a feasible solution where minimum safety distance constraint, 5km is satisfied.

Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

FIG. 5 FLIGHT SCENARIO AFTER COORDINATION AND GLOBAL CONVERGENCE OF SOLUTIONS An interesting aspect of our approach is that multiple solutions exist at any time within the scheme. Each solution is identified as thread. Fig. 5b captures the convergence of the algorithm under communication losses. Here groups of three lines with identical line patterns, correspond to the value of the iteration done by the vehicle to the solutions sent by the other three vehicles. The straight plateaus represent the places where we receive no information from that vehicle. For example, in Fig. 6b, we show vehicle 1's perspective of the whole iteration and the pattern of the communications with the other vehicle. Here zero corresponds to no communication or lost communication at that iteration with the other vehicle.

FIG. 6 SAFETY ASSURANCE CONSTRAINTS AFTER COORDINATION AND DECENTRALIZED COORDINATION FROM VEHICLE #1S PERSPECTIVE Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org

Notice that, especially a very bad link with vehicle 3 results in that solution to be not updated as frequently as the others. This results in it to have high values and thus not selected. Whenever the communication is restored, we obtain better solutions from vehicle 3. However, the graph indicates a solution of similar quality (cost) was indeed received beforehand. This is a property of the algorithm where solutions can flow down from different routes even though the communication between two vehicles is unreliable.

4. CONCLUSIONS
Recent advances in nano-technology resulted in development of reconfigurable networks that span numerous devices with spatial and temporal differences. A feature of such systems is that, information on state, dynamics and operational constraints is distributed throughout the system. For control and estimation system design for such structures, we presented an overview of a new decentralized optimization concept. The methodology embeds the inherent nature of the problem which evolves across independent decision makers with incomplete mathematical models. The results are illustrated using a multiple unmanned air vehicle (UAV) coordination system.

ACKNOWLEDGEMENTS
The author would like to thank Mr. Pierguiseppe Zani for his encouragement and guidance for publication of the results under the Instrumentation, Systems and Automation Society.

REFERENCES
[1] Siljak D.D., Decentralized Control of Complex Systems, Academic Press, New York, NY, 1991 [2] DoD Multidisciplinary Research Initiative Decision-Making Under Uncertainty: http://www.cs.berkeley.edu/~jordan/muri/ [3] Inalhan G., Stipanovic D. M., Tomlin C. J., Decentralized Optimization, with Application to Multiple Aircraft Coordination, SUDAAR-759, Stanford University, February 2003 [4] Inalhan G., Stipanovic D. M., Tomlin C. J., Penalty-based Methods for Decentralized Optimization Across Independent Decision Makers, Submitted to the Journal of Optimization Theory and Applications, January 2003 [5] Data Turbine Matlab Toolkit Reference V1.1, Creare Inc., October 1998

Copyright 2003 by ISA - The Instrumentation, Systems and Automation Society. Presented at ISA EXPO 2003; http:/www.isa.org