Optimal nonparametric inference via deep neural network

Ruiqi Liu; Ben Boukai; Zuofeng Shang

doi:10.1016/j.jmaa.2021.125561

Optimal nonparametric inference via deep neural network

Ruiqi Liu, Ben Boukai, Zuofeng Shang

Mathematical Sciences

New Jersey Institute of Technology

Research output: Contribution to journal › Article › peer-review

Abstract

Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the L² minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proved as minimax optimal under suitable network architectures.

Original language	English (US)
Article number	125561
Journal	Journal of Mathematical Analysis and Applications
Volume	505
Issue number	2
DOIs	https://doi.org/10.1016/j.jmaa.2021.125561
State	Published - Jan 15 2022

All Science Journal Classification (ASJC) codes

Analysis
Applied Mathematics

Keywords

Asymptotic distribution
Deep neural network
Nonparametric inference
Nonparametric testing
Optimal minimax risk bound
Tensor product B-splines

Access to Document

10.1016/j.jmaa.2021.125561

Cite this

@article{461b343af82143ec9f6ceca1bf1a60e8,

title = "Optimal nonparametric inference via deep neural network",

abstract = "Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the L2 minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proved as minimax optimal under suitable network architectures.",

keywords = "Asymptotic distribution, Deep neural network, Nonparametric inference, Nonparametric testing, Optimal minimax risk bound, Tensor product B-splines",

author = "Ruiqi Liu and Ben Boukai and Zuofeng Shang",

note = "Publisher Copyright: {\textcopyright} 2021 Elsevier Inc.",

year = "2022",

month = jan,

day = "15",

doi = "10.1016/j.jmaa.2021.125561",

language = "English (US)",

volume = "505",

journal = "Journal of Mathematical Analysis and Applications",

issn = "0022-247X",

publisher = "Academic Press Inc.",

number = "2",

}

TY - JOUR

T1 - Optimal nonparametric inference via deep neural network

AU - Liu, Ruiqi

AU - Boukai, Ben

AU - Shang, Zuofeng

PY - 2022/1/15

Y1 - 2022/1/15

N2 - Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the L2 minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proved as minimax optimal under suitable network architectures.

AB - Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the L2 minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proved as minimax optimal under suitable network architectures.

KW - Asymptotic distribution

KW - Deep neural network

KW - Nonparametric inference

KW - Nonparametric testing

KW - Optimal minimax risk bound

KW - Tensor product B-splines

UR - http://www.scopus.com/inward/record.url?scp=85112785555&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85112785555&partnerID=8YFLogxK

U2 - 10.1016/j.jmaa.2021.125561

DO - 10.1016/j.jmaa.2021.125561

M3 - Article

SN - 0022-247X

VL - 505

JO - Journal of Mathematical Analysis and Applications

JF - Journal of Mathematical Analysis and Applications

IS - 2

M1 - 125561

ER -

Optimal nonparametric inference via deep neural network

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this