Superlinearly Convergent PMP-Based Algorithms for Nash-Coupled Nonlinear Discrete-Time Control

Abstract:
This paper investigates the optimal control of discrete-time nonlinear systems with two strategically coupled control inputs arising from a Nash-type interaction. By augmenting the coupled inputs into a unified control vector and embedding the equilibrium conditions into a single performance index, the original game-based problem is reformulated as anaugmented optimal control problem. Under standard regularity assumptions, we establish that the Nash stationarity conditions are locally equivalent to the first-order optimality conditions characterized by the Pontryagin maximum principle (PMP). Based on this reformulation, two PMP-based iterative algorithms are developed to solve the resulting forward-backward difference equations (FBDEs). The first algorithm exploits exact second-order information through the inversion of a regularized Hessian matrix, while the second constructs a Hessian-free recursive update that avoids explicit Hessian inversion. For both methods, a rigorous local convergence analysis is provided, showing that the effective linear contraction factor vanishes as the iteration horizon increases, leading to a superlinear-type local convergence behavior. Numerical experiments on representative nonlinear systems validate the theoretical findings and demonstrate improved convergence efficiency compared with classical gradient-based and Newton-type methods. The proposed theoretical framework and algorithms are particularly relevant to cooperative control problems in unmanned aerial vehicle (UAV) swarms, where multiple decision-makers interact through coupled dynamics and performance objectives.
Index Terms: Nonlinear discrete-time optimal control, Nash equilibrium and game-theoretic control, Pontryagin maximum principle, multi-agent cooperative control
Published in:The International Journal of Intelligent Control and Systems (Volume: 31, Issue: 2, 2026-06-25)
Page(s):1 - 11