Learning by examples from a non-uniform distribution

Reimann, P.; van den Broeck, C.

doi:10.1103/PHYSREVE.53.3989

Title

Learning by examples from a non-uniform distribution

Author

Reimann, P.

van den Broeck, C.

Abstract

We present a general replica calculation for learning from examples generated by a nonuniform pattern distribution with a single symmetry-breaking orientation. Our results cover the three main learning scenarios: storage of patterns with random classifications by a perceptron, supervised learning from a teacher, and unsupervised learning. We show that for a perceptron the critical storage capacity αc=2 is completely independent of the pattern distribution provided it is point symmetric or provided the classification as ± 1 is unbiased. In a particular model for supervised learning we find that an ideal (Bayes) student learns most from a few examples if they are easy and from a large number if they are difficult. Learning based on the minimization of a specific class of (quadratic) cost functions is solved completely for all three scenarios.

Language

English

Source (journal)

Physical review : E : statistical, nonlinear, and soft matter physics / American Physical Society. - Melville, N.Y., 2001 - 2015

Publication

Melville, N.Y. : American Physical Society , 1996

ISSN

1539-3755 [print]

1550-2376 [online]

DOI

10.1103/PHYSREVE.53.3989

Volume/pages