Neural Systems as Nonlinear Filters
Wolfgang Maass and Eduardo D.Sontag
Technische Univ. Graz Rutgers University
Abstract:
Experimental data show that biological synapses behave
quite differently from the symbolic synapses in all common
artificial neural network models. Biological synapses are dynamic,
i.e., their ``weight'' changes on a short time scale by several
hundred percent in dependence of the past input to the synapse.
In this article we address the question how this inherent synaptic
dynamics-- which should not be confused with long term ``learning'' --
affects the computational power of a neural network. In particular we
analyze computations on temporal and spatio-temporal patterns, and
we give a complete mathematical characterization of all filters that
can be approximated by feedforward neural networks with dynamic
synapses. It turns out that even with just a single hidden layer
such networks can approximate a very rich class of nonlinear filters:
all filters that can be characterized by Volterra series.
This result is robust with regard to various changes in the model for
synaptic dynamics. Our characterization result provides for all
nonlinear filters that are approximable by Volterra series a new
complexity hierarchy which is related to the cost of implementing such
filters in neural systems.
The article will appear in Neural Computation.
It is online available as # 107 from
http://www.tu-graz.ac.at/igi/maass/#Publications
and from
http://www.math.rutgers.edu/~sontag/FTP_DIR/spiking.ps.gz