Detecting exoplanets with the false inclusion probability: Comparison with other detection criteria in the context of radial velocities

Abstract

Context. It is common practice to claim the detection of a signal if, for a certain statistical significance metric, the signal significance exceeds a certain threshold fixed in advance. In the context of exoplanet searches in radial velocity data, the most common statistical significance metrics are the Bayes factor and the false alarm probability (FAP). Both criteria have proved useful, but do not directly address whether an exoplanet detection should be claimed. Furthermore, it is unclear which detection threshold should be taken and how robust the detections are to model misspecification. Aims. The aim of the present work is to define a detection criterion that conveys as precisely as possible the information needed to claim an exoplanet detection, as well as efficient numerical methods to compute it. We compare this new criterion to existing ones in terms of sensitivity and robustness to a change in the model. Methods. We define a general detection criterion called the false inclusion probability (FIP). In the context of exoplanet detections it provides the posterior probability of presence of a planet with a period in a certain interval. Posterior distributions are computed with the nested sampling package POLYCHORD. We show that for FIP and Bayes factor calculations, defining priors on linear parameters as Gaussian mixture models can significantly speed up computations. The performance of the FAP, Bayes factor, and FIP are studied via simulations and analytical arguments. We compare the methods assuming the model is correct, then evaluate their sensitivity to the prior and likelihood choices. Results. Among other properties, the FIP offers ways to test the reliability of the significance levels; it is a particularly efficient way to account for aliasing, and it allows the presence of planets to be excluded with a certain confidence. In our simulations, we find that the FIP outperforms existing detection metrics. We show that low amplitude planet detections are sensitive to priors on period and semi-amplitude, which will require further attention for the detection of Earth-like planets. We recommend to let the parameters of the noise model free in the analysis, rather than fixing a noise model based on a fit to ancillary indicators.