flâneur

[1511.03643] Unifying distillation and privileged information

arxiv.org · 5,323 words · saved by 1 readers

Abstract:Distillation (Hinton et al., 2015) and privileged information (Vapnik & Izmailov, 2015) are two techniques that enable machines to learn from other machines. This paper unifies these two techniques into generalized distillation, a framework to learn from multiple machines and data representations. We provide theoretical and causal insight about the inner workings of generalized distillation, extend it to unsupervised, semisupervised and multitask learning scenarios, and illustrate its efficacy on a variety of numerical simulations on both synthetic and real-world data.

# link_26m3zkjke69.pdf ## Metadata - PDFFormatVersion=1.5 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - CreationDate=D:20160229015023Z - Creator=LaTeX with hyperref package - ModDate=D:20160229015023Z - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.1415926-2.3-1.40.12 (TeX Live 2011) kpathsea version 6.0.1 - Producer=pdfTeX-1.40.12 - Trapped=False ## Contents ### Page 1 Published as a conference paper at ICLR 2016UNIFYING DISTILLATION AND PRIVILEGED INFORMATIONDavid Lopez-PazFacebook AI Research, Paris, Franc

saved by

related reading