Artículo
Estimating the mutual information between two discrete, asymmetric variables with limited samples
Fecha de publicación:
06/2019
Editorial:
Molecular Diversity Preservation International
Revista:
Entropy
e-ISSN:
1099-4300
Idioma:
Inglés
Tipo de recurso:
Artículo publicado
Clasificación temática:
Resumen
Determining the strength of nonlinear, statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual information from limited samples is a challenging task. Since the mutual information is the difference of two entropies, the existing Bayesian estimators of entropy may be used to estimate information. This procedure, however, is still biased in the severely under-sampled regime. Here, we propose an alternative estimator that is applicable to those cases in which the marginal distribution of one of the two variables?the one with minimal entropy?is well sampled. The other variable, as well as the joint and conditional distributions, can be severely undersampled. We obtain a consistent estimator that presents very low bias, outperforming previous methods even when the sampled data contain few coincidences. As with other Bayesian estimators, our proposal focuses on the strength of the interaction between the two variables, without seeking to model the specific way in which they are related. A distinctive property of our method is that the main data statistics determining the amount of mutual information is the inhomogeneity of the conditional distribution of the low-entropy variable in those states in which the large-entropy variable registers coincidences.
Palabras clave:
BAYESIAN ESTIMATION
,
MUTUAL INFORMATION
,
BIAS
,
SAMPLING
Archivos asociados
Licencia
Identificadores
Colecciones
Articulos(CCT - PATAGONIA NORTE)
Articulos de CTRO.CIENTIFICO TECNOL.CONICET - PATAGONIA NORTE
Articulos de CTRO.CIENTIFICO TECNOL.CONICET - PATAGONIA NORTE
Citación
Hernández Lahme, Damián Gabriel; Samengo, Ines; Estimating the mutual information between two discrete, asymmetric variables with limited samples; Molecular Diversity Preservation International; Entropy; 21; 6; 6-2019; 1-20
Compartir
Altmétricas