A Numerical Implementation of Kolmogorov's Superpositions

Abstract

Hecht-Nielsen proposed a feedforward neural network based on Kolmogorov's superpositions

$$\text{f(x}{}_{\mathrm{i}}\text{,…,x}{}_{\mathrm{n}}\text{)=}\text{∑}\text{q=0}\text{2π}\text{Φ}{}_{\mathrm{q}}\text{(y}{}_{\mathrm{q}}\text{)}$$

that apply to all real valued continuous functions f(x₁, …, x_n) defined on a Euclidean unit cube of dimension n ≥ 2. This network has a hidden layer that is independent of f and that transforms the n-tuples (x₁, …, x_n) into the 2n + 1 variables y_q, and an output layer in which f is computed. Kůrková has shown that such a network has an approximate implementation with arbitrary activation functions of sigmoidal type. Actual implementation is, however, impeded by the lack of numerical algorithms for the hidden layer which contain continuous functions of the formy_q=Σⁿ_p=1α_pψ(x_p+qa) with constants a and α_p. This paper gives an explicit numerical implementation of the hidden layer that also enables the implementation of the output layer. Copyright © 1996 Elsevier Science Ltd