On distinguished orbits of reductive representations

Abstract

Let G be a real reductive Lie group and let τ : G −→ GL(V ) be a real reductive representation of G with (restricted) moment map mg : V {0} −→ g. In this work, we introduce the notion of nice space of a real reductive representation to study the problem of how to determine if a G-orbit is distinguished (i.e. it contains a critical point of the norm squared of mg). We give an elementary proof of the well-known convexity theorem of Atiyah–Guillemin– Sternberg in our particular case and we use it to give an easyto-check sufficient condition for a G-orbit of an element in a nice space to be distinguished. In the case where G is algebraic and τ is a rational representation, the above condition is also necessary (making heavy use of recent results of Michael Jablonski), obtaining a generalization of Nikolayevsky’s nice basis criterion. We also provide useful characterizations of nice spaces in terms of the weights of τ . Finally, some applications to ternary forms are presented.