Adaptive learning in two-player Stackelberg games with application to network security

Oregon Convention Center 

This paper proposes an adaptive learning approach to solve two-player Stackelberg games with incomplete information. Specifically, the leader lacks knowledge of the follower's cost function, but knows that the follower's response function to the leader's action belongs to a known parametric family with unknown parameters. Our algorithm simultaneously estimates these parameters and optimizes the leader's action. It guarantees that the estimates of the follower's action and the leader's cost converge to their true values within a finite time, with a preselected error bound that can be arbitrarily small. Additionally, the first-order necessary condition for optimality is asymptotically satisfied for the leader's estimated cost. Under persistent excitation conditions, the parameter estimation error remains within a preselected, arbitrarily small bound as well. Even with mismatches between the known parametric family and the follower's actual response function, our algorithm achieves convergence robustly with error bounds proportional to the mismatch size. Simulation examples in the domain of network security illustrate the algorithm's effectiveness and the convergence of results.