Large deviations in the perceptron model and consequences for active learning
Copyright policy )Large deviations in the perceptron model and consequences for active learning
Abstract Active learning (AL) is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibility of querying a limited number of samples to obtain the corresponding labels, subsequently used for supervised learning. In this work, we consider the task of choosing the subset of samples to be labeled from a fixed finite pool of samples. We assume the pool of samples to be a random matrix and the ground truth labels to be generated by a single-layer teacher random neural network. We employ replica methods to analyze the large deviations for the accuracy achieved after supervised learning on a subset of the original pool. These large deviations then provide optimal achievable performance boundaries for any AL algorithm. We show that the optimal learning performance can be efficiently approached by simple message-passing AL algorithms. We also provide a comparison with the performance of some other popular active learning strategies.
FOS: Computer and information sciences, ACTIVE LEARNING, LARGE DEVIATIONS, PERCEPTRON MODEL, MESSAGE-PASSING ALGORITHMS, Computer Science - Machine Learning, Active learning, Large Deviations, Active learning Large Deviations Perceptron Model Message-passing algorithms, FOS: Physical sciences, PERCEPTRON MODEL, LARGE DEVIATIONS, ACTIVE LEARNING, Disordered Systems and Neural Networks (cond-mat.dis-nn), Condensed Matter - Disordered Systems and Neural Networks, Perceptron Model, Message-passing algorithms, [PHYS] Physics [physics], Machine Learning (cs.LG)
FOS: Computer and information sciences, ACTIVE LEARNING, LARGE DEVIATIONS, PERCEPTRON MODEL, MESSAGE-PASSING ALGORITHMS, Computer Science - Machine Learning, Active learning, Large Deviations, Active learning Large Deviations Perceptron Model Message-passing algorithms, FOS: Physical sciences, PERCEPTRON MODEL, LARGE DEVIATIONS, ACTIVE LEARNING, Disordered Systems and Neural Networks (cond-mat.dis-nn), Condensed Matter - Disordered Systems and Neural Networks, Perceptron Model, Message-passing algorithms, [PHYS] Physics [physics], Machine Learning (cs.LG)
selected citations These citations are derived from selected sources.This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).5 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Top 10% influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average
Citations provided by BIP!Popularity provided by BIP!