Mostrar el registro sencillo del ítem

dc.contributor.author
Vera, Matías Alejandro
dc.contributor.author
Rey Vega, Leonardo Javier
dc.contributor.author
Piantanida, Pablo
dc.date.available
2019-11-13T17:26:42Z
dc.date.issued
2018-10
dc.identifier.citation
Vera, Matías Alejandro; Rey Vega, Leonardo Javier; Piantanida, Pablo; Compression-based regularization with an application to multitask learning; Institute of Electrical and Electronics Engineers; Ieee Journal Of Selected Topics In Signal Processing; 12; 5; 10-2018; 1063-1076
dc.identifier.issn
1932-4553
dc.identifier.uri
http://hdl.handle.net/11336/88736
dc.description.abstract
This paper investigates, from information theoretic grounds, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, i.e., using fewer bits than needed to fully describe the data itself, in order to build meaningful representations of a relevant content (multiple labels). We begin studying a multitask learning (MTL) problem from the average (over the tasks) of misclassification probability point of view and linking it with the popular cross-entropy criterion. Our approach allows an information theoretic formulation of an MTL problem as a supervised learning framework, in which the prediction models for several related tasks are learned jointly from common representations to achieve better generalization performance. More precisely, our formulation of the MTL problem can be interpreted as an information bottleneck problem with side information at the decoder. Based on that, we present an iterative algorithm for computing the optimal tradeoffs and some of its convergence properties are studied. An important feature of this algorithm is to provide a natural safeguard against overfitting, because it minimizes the average risk taking into account a penalization induced by the model complexity. Remarkably, empirical results illustrate that there exists an optimal information rate minimizing the excess risk, which depends on the nature and the amount of available training data. Applications to hierarchical text categorization and distributional word clusters are also investigated, extending previous works.
dc.format
application/pdf
dc.language.iso
eng
dc.publisher
Institute of Electrical and Electronics Engineers
dc.rights
info:eu-repo/semantics/restrictedAccess
dc.rights.uri
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.subject
ARIMOTO-BLAHUT ALGORITHM
dc.subject
INFORMATION BOTTLENECK
dc.subject
MULTITASK LEARNING
dc.subject
REGULARIZATION
dc.subject
SIDE INFORMATION
dc.subject.classification
Otras Ingeniería Eléctrica, Ingeniería Electrónica e Ingeniería de la Información
dc.subject.classification
Ingeniería Eléctrica, Ingeniería Electrónica e Ingeniería de la Información
dc.subject.classification
INGENIERÍAS Y TECNOLOGÍAS
dc.title
Compression-based regularization with an application to multitask learning
dc.type
info:eu-repo/semantics/article
dc.type
info:ar-repo/semantics/artículo
dc.type
info:eu-repo/semantics/publishedVersion
dc.date.updated
2019-10-28T18:17:47Z
dc.journal.volume
12
dc.journal.number
5
dc.journal.pagination
1063-1076
dc.journal.pais
Estados Unidos
dc.description.fil
Fil: Vera, Matías Alejandro. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina
dc.description.fil
Fil: Rey Vega, Leonardo Javier. Universidad de Buenos Aires. Facultad de Ingeniería. Departamento de Electronica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Centro de Simulación Computacional para Aplicaciones Tecnológicas; Argentina
dc.description.fil
Fil: Piantanida, Pablo. Université Paris Sud; Francia. Centre National de la Recherche Scientifique; Francia
dc.journal.title
Ieee Journal Of Selected Topics In Signal Processing
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/url/https://ieeexplore.ieee.org/document/8379424
dc.relation.alternativeid
info:eu-repo/semantics/altIdentifier/doi/http://dx.doi.org/10.1109/JSTSP.2018.2846218


Archivos asociados

Documento no disponible

Comunidades y colecciones

  • Articulos(CSC) [34]
    Articulos de CENTRO DE SIMULACION COMPUTACIONAL P/APLIC. TECNOLOGICAS

Mostrar el registro sencillo del ítem