Adaptive Two-Layer ReLU Neural Network: I. Best Least-squares Approximation

19 Jul 2021 · Min Liu, Zhiqiang Cai, Jingshuang Chen ·

In this paper, we introduce adaptive neuron enhancement (ANE) method for the best least-squares approximation using two-layer ReLU neural networks (NNs). For a given function f(x), the ANE method generates a two-layer ReLU NN and a numerical integration mesh such that the approximation accuracy is within the prescribed tolerance. The ANE method provides a natural process for obtaining a good initialization which is crucial for training nonlinear optimization problems. Numerical results of the ANE method are presented for functions of two variables exhibiting either intersecting interface singularities or sharp interior layers.

PDF Abstract

Code

Add Remove Mark official

No code implementations yet. Submit your code now

Edit Social Preview

Adaptive Two-Layer ReLU Neural Network: I. Best Least-squares Approximation

Code Edit Add Remove Mark official

Categories

Code

Add Remove Mark official