Mostrar el registro sencillo del ítem

dc.contributor.author
Vera, Matías Alejandro  
dc.contributor.author
Rey Vega, Leonardo Javier  
dc.contributor.author
Piantanida, Pablo  
dc.date.available
2019-11-13T17:26:42Z  
dc.date.issued
2018-10  
dc.identifier.citation
Vera, Matías Alejandro; Rey Vega, Leonardo Javier; Piantanida, Pablo; Compression-based regularization with an application to multitask learning; Institute of Electrical and Electronics Engineers; Ieee Journal Of Selected Topics In Signal Processing; 12; 5; 10-2018; 1063-1076  
dc.identifier.issn
1932-4553  
dc.identifier.uri
http://hdl.handle.net/11336/88736  
dc.description.abstract
This paper investigates, from information theoretic grounds, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, i.e., using fewer bits than needed to fully describe the data itself, in order to build meaningful representations of a relevant content (multiple labels). We begin studying a multitask learning (MTL) problem from the average (over the tasks) of misclassification probability point of view and linking it with the popular cross-entropy criterion. Our approach allows an information theoretic formulation of an MTL problem as a supervised learning framework, in which the prediction models for several related tasks are learned jointly from common representations to achieve better generalization performance. More precisely, our formulation of the MTL problem can be interpreted as an information bottleneck problem with side information at the decoder. Based on that, we present an iterative algorithm for computing the optimal tradeoffs and some of its convergence properties are studied. An important feature of this algorithm is to provide a natural safeguard against overfitting, because it minimizes the average risk taking into account a penalization induced by the model complexity. Remarkably, empirical results illustrate that there exists an optimal information rate minimizing the excess risk, which depends on the nature and the amount of available training data. Applications to hierarchical text categorization and distributional word clusters are also investigated, extending previous works.  
dc.format
application/pdf  
dc.language.iso
eng  
dc.publisher
Institute of Electrical and Electronics Engineers  
dc.rights
info:eu-repo/semantics/openAccess  
dc.rights.uri
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/  
dc.subject
ARIMOTO-BLAHUT ALGORITHM  
dc.subject
INFORMATION BOTTLENECK  
dc.subject
MULTITASK LEARNING  
dc.subject
REGULARIZATION  
dc.subject
SIDE INFORMATION  
dc.subject.classification
Otras Ingeniería Eléctrica, Ingeniería Electrónica e Ingeniería de la Información  
dc.subject.classification
Ingeniería Eléctrica, Ingeniería Electrónica e Ingeniería de la Información  
dc.subject.classification
INGENIERÍAS Y TECNOLOGÍAS  
dc.title
Compression-based regularization with an application to multitask learning  
dc.type
info:eu-repo/semantics/article  
dc.type
info:ar-repo/semantics/artículo  
dc.type
info:eu-repo/semantics/publishedVersion  
dc.date.updated
2019-10-28T18:17:47Z  
dc.journal.volume
12  
dc.journal.number
5  
dc.journal.pagination
1063-1076  
dc.journal.pais
Estados Unidos  
dc.description.fil
Fil: Vera, Matías Alejandro. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina  
dc.description.fil
Fil: Rey Vega, Leonardo Javier. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Centro de Simulación Computacional para Aplicaciones Tecnológicas; Argentina  
dc.description.fil
Fil: Piantanida, Pablo. Université Paris Sud; Francia. Centre National de la Recherche Scientifique; Francia  
dc.journal.title
Ieee Journal Of Selected Topics In Signal Processing  
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/url/https://ieeexplore.ieee.org/document/8379424  
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/doi/http://dx.doi.org/10.1109/JSTSP.2018.2846218