Rolling horizon procedures in Semi-Markov Games: The Discounted Case

Abstract

We study the properties of the rolling horizon and the approximate rolling horizon procedures for the case of two-person zero-sum discounted semi-Markov games with infinite horizon, when the state space is a borelian set and the action spaces are considered compact. Under suitable conditions, we prove that the equilibrium is the unique solution of a dynamic programming equation, and we prove bounds which imply the convergence of the procedures when the horizon length tends to infinity. The approach is based on the formalism for Semi-Markov games developed by Luque-Vásquez in [11], together with extensions of the results of Hernández-Lerma and Lasserre [4] for Markov Decision Processes and Chang and Marcus [2] for Markov Games, both in discrete time. In this way we generalize the results on the rolling horizon and approximate rolling horizon procedures previously obtained for discrete-time problems