Title
Unsupervised learning of binary vectors: a Gaussian scenario Unsupervised learning of binary vectors: a Gaussian scenario
Author
Faculty/Department
Faculty of Sciences. Physics
Publication type
article
Publication
Melville, N.Y. :American Physical Society ,
Source (journal)
Physical review : E : statistical, nonlinear, and soft matter physics / American Physical Society. - Melville, N.Y., 2001 - 2015
Volume/pages
61(2000) , p. 6971-6980
ISSN
1539-3755
1550-2376
ISI
000087575400034
Carrier
E
Target language
English (eng)
Full text (Publishers DOI)
Abstract
We study a model of unsupervised learning where the real-valued data vectors are isotropically distributed, except for a single symmetry-breaking binary direction B∈{-1,+1}N, onto which the projections have a Gaussian distribution. We show that a candidate vector J undergoing Gibbs learning in this discrete space, approaches the perfect match J=B exponentially. In addition to the second-order retarded learning phase transition for unbiased distributions, we show that first-order transitions can also occur. Extending the known result that the center of mass of the Gibbs ensemble has Bayes-optimal performance, we show that taking the sign of the components of this vector (clipping) leads to the vector with optimal performance in the binary space. These upper bounds are shown generally not to be saturated with the technique of transforming the components of a special continuous vector, except in asymptotic limits and in a special linear case. Simulations are presented which are in excellent agreement with the theoretical results.
Full text (open access)
https://repository.uantwerpen.be/docman/irua/2db951/8078.pdf
E-info
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000087575400034&DestLinkType=RelatedRecords&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000087575400034&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000087575400034&DestLinkType=CitingArticles&DestApp=ALL_WOS&UsrCustomerID=ef845e08c439e550330acc77c7d2d848