Donald D. Hoffman
released October 1998
After his stroke, Mr. P still had outstanding memory and intelligence. He could still read and talk, and mixed well with the other patients on his ward. His vision was in most respects normal---with one notable exception: He couldn't recognize the faces of people or animals. As he put it himself, "I can see the eyes, nose, and mouth quite clearly, but they just don't add up. They all seem chalked in, like on a blackboard ... I have to tell by the clothes or by the voice whether it is a man or a woman ...The hair may help a lot, or if there is a mustache ... ." Even his own face, seen in a mirror, looked to him strange and unfamiliar. Mr. P had lost a critical aspect of his visual intelligence.
We have long known about IQ and rational intelligence. And, due in part to recent advances in neuroscience and psychology, we have begun to appreciate the importance of emotional intelligence. But we are largely ignorant that there is even such a thing as visual intelligence---that is, until it is severely impaired, as in the case of Mr. P, by a stroke or other insult to visual cortex. The culprit in our ignorance is visual intelligence itself. Vision is normally so swift and sure, so dependable and informative, and apparently so effortless that we naturally assume that it is, indeed, effortless. But the swift ease of vision, like the graceful ease of an Olympic ice skater, is deceptive. Behind the graceful ease of the skater are years of rigorous training, and behind the swift ease of vision is an intelligence so great that it occupies nearly half of the brain's cortex. Our visual intelligence richly interacts with, and in many cases precedes and drives, our rational and emotional intelligence. To understand visual intelligence is to understand, in large part, who we are.
It is also to understand much about our highly visual culture in which, as the saying goes, image is everything. Consider, for instance, our entertainment. Visual effects lure us into theaters, and propel films like Star Wars and Jurassic Park to record sales. Music videos usher us before surreal visual worlds, and spawn TV stations like MTV and VH-1. Video games swallow kids (and adults) for hours on end, and swell the bottom lines of companies like Sega and Nintendo. Virtual reality, popularized in movies like Disclosure and Lawnmower Man, can immerse us in visual worlds of unprecedented realism, and promises to transform not only entertainment but also architecture, education, manufacturing, and medicine. As a culture we vote with our time and wallets and, in the case of entertainment, our vote is clear. Just as we enjoy rich literature that stimulates our rational intelligence, or a moving story that engages our emotional intelligence, so we also seek out and enjoy new media that challenge our visual intelligence.
Or consider marketing and advertisement, which daily manipulate our buying habits with sophisticated images. Corporations spend millions each year on billboards, packaging, magazine ads, and television commercials. Their images can so powerfully influence our behavior that they sometimes generate controversy---witness the uproar over Joe Camel. If you're out to sell something, understanding visual intelligence is, without question, critical to the design of effective visual marketing. And if you're out to buy something, understanding visual intelligence can help clue you in to what is being done to you as a consumer, and how it's being done.
This book is a highly illustrated and accessible introduction to visual intelligence, informed by the latest breakthroughs in vision research. Perhaps the most surprising insight that has emerged from vision research is this: Vision is not merely a matter of passive perception, it is an intelligent process of active construction. What you see is, invariably, what your visual intelligence constructs. Just as scientists intelligently construct useful theories based on experimental evidence, so vision intelligently constructs useful visual worlds based on images at the eyes. The main difference is that the constructions of scientists are done consciously, but those of vision are done, for the most part, unconsciously.
The constructive power of visual intelligence has long fascinated vision researchers. How can vision conjure up the endless panoply of colors and shapes and motions that we see about us in the ``real'' world? How can it morph a mass of metal into a murdering maniac in the world of Terminator 2? How can it stretch before us a world in three dimensions when we watch, with special glasses, a 3D movie like Dial M for Murder? How can it "boldly go where no man has gone before", not just through the "strange new worlds" of Star Trek but through the stranger new worlds revealed by the cameras of the Hubble telescope and of probes like Voyager and Pioneer, worlds for which the eye has not obviously been adapted?
Vision, as we shall see, has divulged many of its secrets to physicists, neurobiologists, perceptual psychologists, and researchers in computer vision. But, as we shall also discover, many interesting secrets are yet to be unveiled. These secrets, both revealed and unrevealed, of your visual intelligence are sure to pique in you the same admiration and curiosity that animate vision researchers.
For some, science is like a "reaper binder" that methodically mows down wheat: Science methodically mows down unanswered questions and leaves ever less room for wonder. But our exploration of visual intelligence will suggest that science is more like an island: "The larger the island of knowledge, the longer the shoreline of wonder." For each question we answer ten new ones arise, and for each new probe at nature entire vistas arise.
Our island of knowledge about visual intelligence has grown immensely in the last two decades. A book-length tour of the new terrain can hit but a few of the interesting sights. Visual Intelligence hits those sights that may inspire you to linger on the shoreline of wonder.
Visual Intelligence has several tourists in mind. For those in marketing, advertising, and graphic design, Visual Intelligence will better acquaint you with your first client: visual intelligence. Your images must pass the scrutiny of a customer's visual intelligence before they can go on to convince her rational intelligence of a need and her emotional intelligence of a desire for your product. Visual intelligence is your path to the head and heart of a customer, and understanding visual intelligence is key to successfully navigating that path.
For the reader of popular science, Visual Intelligence explains why your brain devotes billions of its valuable neurons and trillions of its valuable synapses to vision, why each of your eyes contains within it more computing power than the fastest supercomputers made today, why you can buy a chess machine that beats a Master but can't yet buy a vision machine that beats a toddler's vision, why computer vision is not only possible but is destined soon to be a multi-billion dollar industry that alters our day-to-day lives no less than the computer or the telephone. Visual Intelligence will leave you astonished at what happens when you simply open your eyes.
For the undergraduate with an undeclared major, beware, Visual Intelligence might lure you away from a high-paying career as a doctor or lawyer or CEO, and lure you into a career as a vision researcher. The unsolved puzzles of visual intelligence are a worthy challenge for the sharpest of minds (and the pay isn't half bad). There is much to be done, and the field could use your help.
For those building virtual worlds, Visual Intelligence documents that human vision (and touch and hearing) is the real creator of virtual reality (VR), and that the role of VR systems is to "trick" human vision into creating those realities that you, the VR designer, want created. To build compelling virtual worlds, one must understand visual intelligence and how it constructs visual realities.
For lawyers concerned with eye-witness testimony, Visual Intelligence conveys the latest scientific insights and perspectives on the visual processes that underlie such testimony. The visual reality of eye witnesses is a constructed reality. Understanding the rules by which eye-witness reality is constructed can be critical to a proper evaluation of eye-witness testimony.
For philosophers interested in the epistemological and ontological issues raised by perception, Visual Intelligence provides an accessible entrance into the latest empirical and theoretical literature on vision, and suggests that an idealist reading of this literature can be at least as compelling as the best physicalist readings.
For vision researchers, faced with a burgeoning literature from a variety of disciplines, Visual Intelligence provides a provocative synthesis. Unifying themes can be discerned in the diversity of our technical results, and contemplating these themes is not only worthwhile in its own right but can alter how we conceive and conduct our technical work.
For anyone who is just plain curious about how they see, or who isn't curious because they can't imagine there's that much to it, Visual Intelligence is for you.
That should include just about everyone.
As in many scientific fields, mathematics has cropped up everywhere in vision research---a help to researchers and a hindrance to laymen. Fortunately, the key discoveries about visual intelligence can be conveyed without mathematics. To that end, all mathematics are banished to endnotes for those so inclined. This approach is, of course, not new. The Greek physician Galen (AD 130--200), for instance, adopted it when he wrote about vision:
I have explained nearly everything pertaining to the eyes with the exception of one point which I had intended to omit ... since it necessarily involves the theory of geometry ... [But] I felt impelled to take up again what I had omitted and add it to the end of this book.
Also banished to the back of the book are all notes and citations. These are listed by page number. So if you find an intriguing fact or picture or quote on page 73 and want to know more, chances are you'll find more in Note 73 at the back of the book.
Many people have helped in the development of Visual Intelligence: Laura Andes, Joe Arpaia, Margaret Atherton, Kimberley Babb, Bruce Bennett, Francis Crick, Alan Gilchrist, Heiko Hecht, Dieter Heyer, David Hoffman, Loretta Hoffman, Nicole Huber, Stan Klein, Larry Maloney, Rainer Mausfeld, Brian McLaughlin, Alan Nelson, Chetan Prakash, Whitman Richards, Scott Richman, Craig Sauer, Armin Schwegler, Stan Schein, Robert Schwartz, Greg Seyranian, Manish Singh, Carol Skrenes, Dejan Todorovíc, VaNessa Vollmer, and many students in my class on the mind-body problem at UC Irvine, and in Stan Klein's class on sensation and perception at UCLA, who read an earlier version of this book for the class and gave helpful feedback. My editor at Norton, Angela von der Lippe, expertly helped me to streamline the text and broaden its appeal. My agents Katinka Matson and John Brockman encouraged me early on, and guided me through the proposal process. Thank you all for your help.
Work on this book was supported in part by a sabbatical leave from the University of California, Irvine, and by a grant from the Zentrum für interdisziplinäre Forschung of the University of Bielefeld, Germany. Thank you for your support.
1. A Creative Genius for Vision 17
2. Inflating an Artist's Sketch 35
3. The Invisible Surface that Glows 68
4. Spontaneous Morphing 102
5. The Day Color Drained Away 130
6. When the World Stopped Moving 166
7. The Feel of a Phantom 198
8. Peeking Behind the Icons 211
Notes 240
References 254
Index 299
Visual Intelligence is available at Amazon and Barnes and Noble.
Up to Hoffman's Home Page.