RANDOM NEURAL NETWORKS IN THE INFINITE WIDTH LIMIT AS GAUSSIAN PROCESSES

TY - JOUR

T1 - RANDOM NEURAL NETWORKS IN THE INFINITE WIDTH LIMIT AS GAUSSIAN PROCESSES

AU - Hanin, Boris

N1 - Publisher Copyright: © Institute of Mathematical Statistics, 2023.

PY - 2023/12

Y1 - 2023/12

N2 - This article gives a new proof that fully connected neural networks with random weights and biases converge to Gaussian processes in the regime where the input dimension, output dimension, and depth are kept fixed, while the hidden layer widths tend to infinity. Unlike prior work, convergence is shown assuming only moment conditions for the distribution of weights and for quite general nonlinearities.

AB - This article gives a new proof that fully connected neural networks with random weights and biases converge to Gaussian processes in the regime where the input dimension, output dimension, and depth are kept fixed, while the hidden layer widths tend to infinity. Unlike prior work, convergence is shown assuming only moment conditions for the distribution of weights and for quite general nonlinearities.

KW - Gaussian processes

KW - Neural networks

KW - limit theorems

UR - http://www.scopus.com/inward/record.url?scp=85180508472&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85180508472&partnerID=8YFLogxK

U2 - 10.1214/22-AAP1933

DO - 10.1214/22-AAP1933

M3 - Article

SN - 1050-5164

VL - 33

SP - 4798

EP - 4819

JO - Annals of Applied Probability

JF - Annals of Applied Probability

IS - 6A

ER -