Obtain a PDF of the paper titled The committee machine: Computational to statistical gaps in studying a two-layers neural community, by Benjamin Aubin and 4 different authors

Obtain PDF

Summary:Heuristic instruments from statistical physics have been used previously to find the section transitions and compute the optimum studying and generalization errors within the teacher-student state of affairs in multi-layer neural networks. On this contribution, we offer a rigorous justification of those approaches for a two-layers neural community mannequin referred to as the committee machine. We additionally introduce a model of the approximate message passing (AMP) algorithm for the committee machine that permits to carry out optimum studying in polynomial time for a big set of parameters. We discover that there are regimes by which a low generalization error is information-theoretically achievable whereas the AMP algorithm fails to ship it, strongly suggesting that no environment friendly algorithm exists for these circumstances, and unveiling a big computational hole.