Mostrar el registro sencillo del ítem
dc.contributor.author
Vera, Matías Alejandro
dc.contributor.author
Rey Vega, Leonardo Javier
dc.contributor.author
Piantanida, Pablo
dc.date.available
2023-04-14T18:48:04Z
dc.date.issued
2022-10
dc.identifier.citation
Vera, Matías Alejandro; Rey Vega, Leonardo Javier; Piantanida, Pablo; Information flow in Deep Restricted Boltzmann Machines: An analysis of mutual information between inputs and outputs; Elsevier Science; Neurocomputing; 507; 10-2022; 235-246
dc.identifier.issn
0925-2312
dc.identifier.uri
http://hdl.handle.net/11336/193980
dc.description.abstract
Empirical evidence suggests the existence of an entangled relationship between the information flow from inputs features to hidden representations of a deep neural network and its ability to generalize from training samples to unobserved data. For instance, regularization techniques often used to control statistical generalization, are expected to impact this information flow. In this work, we study MI (mutual information) between inputs and representation outputs, and its relationship with various regularization methods commonly used in Restricted Boltzmann Machines (RBM) and their generalizations: Deep Belief Networks and Deep Boltzmann Machines. Our theoretical findings show the existence of fundamental connections between the hyperparameters associated with the regularization and the MI, including relevant practical ingredients such as: network dimension, matrix norms and dropout probability, which are well-known to influence the generalization ability of the network. These results are experimentally corroborated on various visual datasets.
dc.format
application/pdf
dc.language.iso
eng
dc.publisher
Elsevier Science
dc.rights
info:eu-repo/semantics/restrictedAccess
dc.rights.uri
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.subject
GRAPHICAL MODELS
dc.subject
MUTUAL INFORMATION
dc.subject
REGULARIZATION
dc.subject
RESTRICTED BOLTZMANN MACHINE
dc.subject
UNSUPERVISED LEARNING
dc.subject.classification
Ciencias de la Computación
dc.subject.classification
Ciencias de la Computación e Información
dc.subject.classification
CIENCIAS NATURALES Y EXACTAS
dc.title
Information flow in Deep Restricted Boltzmann Machines: An analysis of mutual information between inputs and outputs
dc.type
info:eu-repo/semantics/article
dc.type
info:ar-repo/semantics/artículo
dc.type
info:eu-repo/semantics/publishedVersion
dc.date.updated
2023-04-14T17:26:58Z
dc.journal.volume
507
dc.journal.pagination
235-246
dc.journal.pais
Países Bajos
dc.journal.ciudad
Amsterdam
dc.description.fil
Fil: Vera, Matías Alejandro. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Centro de Simulación Computacional para Aplicaciones Tecnológicas; Argentina
dc.description.fil
Fil: Rey Vega, Leonardo Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Centro de Simulación Computacional para Aplicaciones Tecnológicas; Argentina
dc.description.fil
Fil: Piantanida, Pablo. Centre National de la Recherche Scientifique; Francia
dc.journal.title
Neurocomputing
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/doi/https://doi.org/10.1016/j.neucom.2022.08.014
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/url/https://www.sciencedirect.com/science/article/abs/pii/S0925231222009833
Archivos asociados