My intention in writing this book was to describe what we know of the
anatomy and physiology of the visual pathway up to the striate cortex. The knowledge we have now is really only the beginning of an effort to understand the physiological basis of perception, a story whose next stages are
just coming into view; we can see major mountain ranges in the middle distance,
but the end is nowhere in sight.
The striate cortex is just the first of over a dozen separate visual
areas, each of which maps the whole visual field. These areas collectively form
the patchwork quilt that constitutes the occipital cortex and extends forward
into the posterior temporal cortex and posterior parietal cortex. Beginning with
the striate cortex, each area feeds into two or more areas higher in the
and the connections are topographically organized so that any given
area contains an orderly representation of the visual field, just as the striate
The ascending connections presumably take the visual information from
one region to the next for further processing. For each'of these areas our
problem is to find out how the information is processed—the same problem
we faced earlier when we asked what the striate cortex does with the information
it gets from the geniculate.
Although we have only recently come to realize how numerous these visual areas are, we are already building up knowledge about the connections
and single-cell physiology of some of them. Just as area 17 is a mosaic
of two sets of regions, blob and nonblob, so the next visual area, area 18 or visual
is a mosaic of three sets. Unlike the blobs and interblobs, which formed
islands in an ocean, the mosaic in area 18 takes the form of parallel stripes.
In these subdivisions we find a striking segregation of function. In the set
of thick stripes, most of the cells are highly sensitive to the relative horizontal
positions of the stimuli in the two eyes, as described in Chapter 7; we therefore
conclude that this thick-stripe subdivision is concerned at least in part with
stereopsis. In the second set, the thin stripes, cells lack orientation selectivity
and often show specific color responses. In the third set, the pale stripes, cells
are orientation selective and most are end stopped. Thus the three sets of subdivisions
that make up area 18 seem to be concerned with stereopsis, color, and form.
A similar division of labor occurs in the areas beyond area 18, but
now entire areas seem to be given over to one or perhaps two visual functions.
An area called MT (for middle temporal gyrus) is devoted to movement and stereopsis;
one called V 4 (V for visual) seems to be concerned mainly with color.
We can thus discern two processes that go hand in hand. The first is hierarchical.
To solve the various problems in vision outlined in previous chapters—color,
stereopsis, movement, form—information is operated upon in one
area after the next, with progressive abstraction and increasing complexity of
representation. The second process consists of a divergence of pathways. Apparently the problems require such different strategies and hardware that it
becomes more efficient to handle them in entirely separate channels.
This surprising tendency for attributes such as form, color, and movement to be handled by separate structures in the brain immediately raises
the question of how all the information is finally assembled, say for perceiving
a bouncing red ball. It obviously must be assembled somewhere, if only
at the motor nerves that subserve the action of catching. Where it's assembled,
and how, we have no idea.
This is where we are, in 1995, in the step-by-step analysis of the visual
In terms of numbers of synapses (perhaps eight or ten) and complexity
of transformations, it may seem a long way from the rods and cones in the
retina to areas MT or visual area 2 in the cortex, but it is surely a far longer
way from such processes as orientation tuning, end-stopping, disparity tuning,
or color opponency to the recognition of any of the shapes that we perceive in our everyday life. We are far from understanding the perception
of objects, even such comparatively simple ones as a circle, a triangle,
or the letter A—indeed, we are far from even being able to come up with
We should not be particularly surprised or disconcerted over our relative ignorance in the face of such mysteries. Those who work in the field
of artificial intelligence (AI) cannot design a machine that begins to rival
the brain at carrying out such special tasks as processing the written word, driving
a car along a road, or distinguishing faces. They have, however, shown that
the theoretical difficulties in accomplishing any of these tasks are formidable.
It is not that the difficulties cannot be solved—the brain clearly has
but rather that the methods the brain applies cannot be simple: in the
lingo of AI, the problems are "nontrivial". So the brain solves nontrivial
The remarkable thing is that it solves not just two or three but thousands
In the question period following a lecture, a sensory physiologist or
psychologist soon gets used to being asked what the best guess is as to
how objects are recognized. Do cells continue to become more and more specialized
at more and more central levels, so that at some stage we can expect to
find cells so specialized that they respond to one single person's face—say,
one's grandmother's? This notion, called the grandmother cell theory, is hard to
entertain seriously. Would we expect to find separate cells for grandmother smiling,
grandmother weeping, or grandmother sewing? Separate cells for the concept or definition of grandmother: one's mother's or father's mother? And
if we did have grandmother cells, then what? Where would they project?
The alternative is to suppose that a given object leads to the firing
of a particular constellation of cells, any member of which could also belong
to other constellations. Knowing as we do that destroying a small region
of brain does not generally destroy specific memories, we have to suppose that
the cells in a constellation are not localized to a single cortical area, but
extend over many areas. Grandmother sewing then becomes a bigger constellation comprising grandmother-by-definition, grandmother's face, and sewing.
It is admittedly not easy to think of a way to get at such ideas experimentally. To record from one cell alone and make sense of the results even
in the striate cortex is not easy: it is hard even to imagine coming to terms
with a cell that may be a member of a hundred constellations, each consisting of
a thousand cells. Having tried to record from three cells simultaneously and
understand what they all are doing in the animal's daily life, I can only