The local learning idea is a very core idea that drives research in
a number of different fields. I welcome comments on the questions
and issues raised here.
This note is being sent to many listserves. I will collect all of
the responses from different sources and redistribute them to all
of the participating listserves. The last such discussion was very
productive. It has led to the realization by some key researchers
in the connectionist area that "memoryless" learning perhaps is not
a very "valid" idea. That recognition by itself will lead to more
robust and reliable learning algorithms in the future. Perhaps a
more active debate on the local learning issue will help us resolve
this issue too.
A) Does plasticity imply local learning?
The physical changes that are observed in synapses/cells in
experimental neuroscience when some kind of external stimuli is
applied to the cells may not result at all from any specific
"learning" at the cells. The cells might simply be responding to a
"signal to change" - that is, to change by a specific amount in a
specific direction. In animal brains, it is possible that the
"actual" learning occurs in some other part(s) of the brain, say
perhaps by a global learning mechanism. This global mechanism can
then send "change signals" to the various cells it is using to
learn a specific task. So it is possible that in these neuroscience
experiments, the external stimuli generates signals for change
similar to those of a global learning agent in the brain and that
the changes are not due to "learning" at the cells themselves.
Please note that scientific facts/phenomenon like LTP/LTD or
synaptic plasticity can probably be explained equally well by many
theories of learning (e.g. local learning vs. global learning,
etc.). However, the correctness of an explanation would have to be
judged from its consistency with other behavioral and biological
facts, not just one single biological phenomemon or fact.
B) "Pure" local learning does not explain a number of other
activities that are part of the process of learning!!
When learning is to take place by means of "local learning" in a
network of cells, the network has to be designed prior to its
training. Setting up the net before "local" learning can proceed
implies that an external mechanism is involved in this part of the
learning process. This "design" part of learning precedes actual
training or learning by a collection of "local learners" whose only
knowledge about anything is limited to the local learning law to
use! In addition, these "local learners" may have to be told what
type of local learning law to use, given that a variety of
different types can be used under different circumstances. Imagine
who is to instruct such local learners which type of learning law
to use? In
addition to these, the "passing" of appropriate information to the
appropriate set of cells also has to be "coordinated" by some
external or global learning mechanism. This coordination cannot
just happen by itself, like magic. It has to be directed from some
place by some agent or mechanism.
In order to learn properly and quickly, humans generally collect
and store relevant information in their brains and then "think"
about it (e.g. what problem features are relevant, problem
complexity, etc.). So prior to any "local learning," there must be
processes in the brain that examine this "body of
information/facts" about a problem in order to design the
appropriate network that would fit the problem complexity, select
the problem features that are meaningful, etc. It would be very
difficult to answer the questions "What size net?" and "What
features to use?" without looking at the problem in great detail. A
bunch of "pure" local learners, armed with their local learning
laws, would have no clue to these issues of net design,
generalization and feature selection.
So, in the whole, there are a "number of activities" that need to
be
performed before any kind of "local learning" can take place. These
aforementioned learning activities "cannot" be performed by a
collection of "local learning" cells! There is more to the process
of learning than simple local learning by individual cells. Many
learning "decisions/tasks" must precede actual training by "local
learners." A group of independent "local learners" simply cannot
start learning and be able to reproduce the learning
characteristics and processes of an "autonomous system" like the
brain.
Local learning, however, is still a feasible idea, but only within
a general global learning context. A global learning mechanism
would be the one that "guides" and "exploits" these local learners.
However, it is also possible that the global mechanism actually
does all of the computations (learning) and "simply sends signals"
to the network cells for appropriate synaptic adjustment. Both of
these possibilities seem logical: (a) a "pure" global mechanism
that learns by itself and then sends signals to the cells to
adjust, or (b) a global/local combination where the global
mechanism performs certain tasks and then uses the local mechanism
for training/learning.
Note that the global learning mechanism may actually be implemented
with a collection of local learners!!
The basic argument being made here is that there are many tasks in
a "learning process" and that a set of "local learners" armed with
their local learning laws is incapable of performing all of those
tasks. So local learning can only exist in the context of global
learning and thus is only "a part" of the total learning process.
It will be much easier to develop a consistent learning theory
using the global/local idea. The global/local idea perhaps will
also give us a better handle on the processes that we call
"developmental" and "evolutionary." And it will, perhaps, allow us
to better explain many of the puzzles and inconsistencies in our
current body of discoveries about the brain. And, not the least, it
will help us construct far better algorithms by removing the
"unwarranted restrictions" imposed on us by the current ideas. Any
comments
on these ideas and possibilities are welcome.
Asim Roy
Arizona State University