Everything should be made as simple as possible, but not simpler. —Albert Einstein
This book tries to explain how minds work. How can intelligence emerge from nonintelligence? To answer that, we'll show that you can build a mind from many little parts, each mindless by itself.
I'll call Society of Mind this scheme in which each mind is made of many smaller processes. These we'll call agents. Each mental agent by itself can only do some simple thing that needs no mind or thought at all. Yet when we join these agents in societies — in certain very special ways — this leads to true intelligence.
There's nothing very technical in this book. It, too, is a society — of many small ideas. Each by itself is only common sense, yet when we join enough of them we can explain the strangest mysteries of mind. One trouble is that these ideas have lots of cross-connections. My explanations rarely go in neat, straight lines from start to end. I wish I could have lined them up so that you could climb straight to the top, by mental stair-steps, one by one. Instead they're tied in tangled webs.
Perhaps the fault is actually mine, for failing to find a tidy base of neatly ordered principles. But I'm inclined to lay the blame upon the nature of the mind: much of its power seems to stem from just the messy ways its agents cross-connect. If so, that complication can't be helped; it's only what we must expect from evolution's countless tricks.
What can we do when things are hard to describe? We start by sketching out the roughest shapes to serve as scaffolds for the rest; it doesn't matter very much if some of those forms turn out partially wrong. Next, draw details to give these skeletons more lifelike flesh. Last, in the final filling-in, discard whichever first ideas no longer fit.
That's what we do in real life, with puzzles that seem very hard. It's much the same for shattered pots as for the cogs of great machines. Until you've seen some of the rest, you can't make sense of any part.
Good theories of the mind must span at least three different scales of time: slow, for the billion years in which our brains have evolved; fast, for the fleeting weeks and months of infancy and childhood; and in between, the centuries of growth of our ideas through history.
To explain the mind, we have to show how minds are built from mindless stuff, from parts that are much smaller and simpler than anything we'd consider smart. Unless we can explain the mind in terms of things that have no thoughts or feelings of their own, we'll only have gone around in a circle. But what could those simpler particles be — the agents that compose our minds? This is the subject of our book, and knowing this, let's see our task. There are many questions to answer.
Function: How do agents work? Embodiment: What are they made of? Interaction: How do they communicate? Origins: Where do the first agents come from? Heredity: Are we all born with the same agents? Learning: How do we make new agents and change old ones? Character: What are the most important kinds of agents? Authority: What happens when agents disagree? Intention: How could such networks want or wish? Competence: How can groups of agents do what separate agents cannot do? Selfness: What gives them unity or personality? Meaning: How could they understand anything? Sensibility: How could they have feelings and emotions? Awareness: How could they be conscious or self-aware?
How could a theory of the mind explain so many things, when every separate question seems too hard to answer by itself? These questions all seem difficult, indeed, when we sever each one's connections to the other ones. But once we see the mind as a society of agents, each answer will illuminate the rest.
It was never supposed [the poet Imlac said] that cogitation is inherent in matter, or that every particle is a thinking being. Yet if any part of matter be devoid of thought, what part can we suppose to think? Matter can differ from matter only in form, density, bulk, motion, and direction of motion. To which of these, however varied or combined, can consciousness be annexed? To be round or square, to be solid or fluid, to be great or little, to be moved slowly or swiftly, one way or another, are modes of material existence all equally alien from the nature of cogitation. If matter be once without thought, it can only be made to think by some new modification; but all the modifications which it can admit are equally unconnected with cogitative powers. —Samuel Johnson
How could solid-seeming brains support such ghostly things as thoughts? This question troubled many thinkers of the past. The world of thoughts and the world of things appeared to be too far apart to interact in any way. So long as thoughts seemed so utterly different from everything else, there seemed to be no place to start.
A few centuries ago it seemed equally impossible to explain Life, because living things appeared to be so different from anything else. Plants seemed to grow from nothing. Animals could move and learn. Both could reproduce themselves — while nothing else could do such things. But then that awesome gap began to close. Every living thing was found to be composed of smaller cells, and cells turned out to be composed of complex but comprehensible chemicals. Soon it was found that plants did not create any substance at all but simply extracted most of their material from gases in the air. Mysteriously pulsing hearts turned out to be no more than mechanical pumps, composed of networks of muscle cells. But it was not until the present century that John von Neumann showed theoretically how cell-machines could reproduce while, almost independently, James Watson and Francis Crick discovered how each cell actually makes copies of its own hereditary code. No longer does an educated person have to seek any special, vital force to animate each living thing.
Similarly, a century ago, we had essentially no way to start to explain how thinking works.  Somewhat later, on the mechanical side, mathematicians like Kurt Gödel and Alan Turing began to reveal the hitherto unknown range of what machines could be made to do. These two streams of thought began to merge only in the 1940s, when Warren McCulloch and Walter Pitts began to show how machines might be made to see, reason, and remember. Research in the modern science of Artificial Intelligence started only in the 1950s, stimulated by the invention of modern computers. This inspired a flood of new ideas about how machines could do what only minds had done previously.
Most people still believe that no machine could ever be conscious, or feel ambition, jealousy, humor, or have any other mental life-experience. To be sure, we are still far from being able to create machines that do all the things people do. But this only means that we need better theories about how thinking works. This book will show how the tiny machines that we'll call agents of the mind could be the long sought particles that those theories need.
You know that everything you think and do is thought and done by you. But what's a you ? What kinds of smaller entities cooperate inside your mind to do your work? To start to see how minds are like societies, try this: pick up a cup of tea!
Your GRASPING agents want to keep hold of the cup. Your BALANCING agents want to keep the tea from spilling out. Your THIRST agents want you to drink the tea. Your MOVING agents want to get the cup to your lips.
Yet none of these consume your mind as you roam about the room talking to your friends. You scarcely think at all about Balance; Balance has no concern with Grasp; Grasp has no interest in Thirst; and Thirst is not involved with your social problems. Why not? Because they can depend on one another. If each does its own little job, the really big job will get done by all of them together: drinking tea.
How many processes are going on, to keep that teacup level in your grasp? There must be at least a hundred of them, just to shape your wrist and palm and hand. Another thousand muscle systems must work to manage all the moving bones and joints that make your body walk around. And to keep everything in balance, each of those processes has to communicate with some of the others. What if you stumble and start to fall? Then many other processes quickly try to get things straight.  Others are occupied with what to do about the tea: you wouldn't want to burn your own hand, but neither would you want to scald someone else. You need ways to make quick decisions.
All this happens while you talk, and none of it appears to need much thought. But when you come to think of it, neither does your talk itself. What kinds of agents choose your words so that you can express the things you mean? How do those words get arranged into phrases and sentences, each connected to the next? What agencies inside your mind keep track of all the things you've said — and, also, whom you've said them to? How foolish it can make you feel when you repeat — unless you're sure your audience is new.
We're always doing several things at once, like planning and walking and talking, and this all seems so natural that we take it for granted. But these processes actually involve more machinery than anyone can understand all at once. So, in the next few sections of this book, we'll focus on just one ordinary activity — making things with children's building-blocks. First we'll break this process into smaller parts, and then we'll see how each of them relates to all the other parts.
In doing this, we'll try to imitate how Galileo and Newton learned so much by studying the simplest kinds of pendulums and weights, mirrors and prisms. Our study of how to build with blocks will be like focusing a microscope on the simplest objects we can find, to open up a great and unexpected universe. It is the same reason why so many biologists today devote more attention to tiny germs and viruses than to magnificent lions and tigers. For me and a whole generation of students, the world of work with children's blocks has been the prism and the pendulum for studying intelligence. In science, one can learn the most by studying what seems the least.
Imagine a child playing with blocks, and imagine that this child's mind contains a host of smaller minds. Call them mental agents. Right now, an agent called Builder is in control. Builder's specialty is making towers from blocks.
Our child likes to watch a tower grow as each new block is placed on top. But building a tower is too complicated a job for any single, simple agent, so Builder has to ask for help from several other agents:
In fact, even to find another block and place it on the tower top is too big for a job for any single agent. So Add, in turn, must call for other agents' help. Before we're done, we'll need more agents than would fit in any diagram.
Why break things into such small parts? Because minds, like towers, are made that way — except that they're composed of processes instead of blocks. And if making stacks of blocks seems insignificant — remember that you didn't always feel that way. When first you found some building toys in early childhood, you probably spent joyful weeks of learning what to do with them. If such toys now seem relatively dull, then you must ask yourself how you have changed. Before you turned to more ambitious things, it once seemed strange and wonderful to be able to build a tower or a house of blocks.
Yet, though all grown-up persons know how to do such things, no one understands how we learn to do them! And that is what will concern us here. To pile up blocks into heaps and rows: these are skills each of us learned so long ago that we can't remember learning them at all. Now they seem mere common sense — and that's what makes psychology hard. This forgetfulness, the amnesia of infancy, makes us assume that all our wonderful abilities were always there inside our minds, and we never stop to ask ourselves how they began and grew.
We found a way to make our tower builder out of parts. But Builder is really far from done. To build a simple stack of blocks, our child's agents must accomplish all these other things.
See must recognize its blocks, whatever their color, size, and place — in spite of different backgrounds, shades, and lights, and even when they're partially obscured by other things.
Then, once that's done, Move has to guide the arm and hand through complicated paths in space, yet never strike the tower's top or hit the child's face.
And think how foolish it would seem, if Find were to see, and Grasp were to grasp, a block supporting the tower top!
When we look closely at these requirements, we find a bewildering world of complicated questions. For example, how could Find determine which blocks are still available for use? It would have to understand the scene in terms of what it is trying to do. This means that we'll need theories both about what it means to understand and about how a machine could have a goal. Consider all the practical judgments that an actual Builder would have to make. It would have to decide whether there are enough blocks to accomplish its goal and whether they are strong and wide enough to support the others that will be placed on them.
What if the tower starts to sway? A real builder must guess the cause. It is because some joint inside the column isn't square enough? Is the foundation insecure, or is the tower too tall for its width? Perhaps it is only because the last block was placed too roughly.
All children learn about such things, but we rarely ever think about them in our later years. By the time we are adults we regard all of this to be simple common sense. But that deceptive pair of words conceals almost countless different skills.
Common sense is not a simple thing. Instead, it is an immense society of hard-earned practical ideas — of multitudes of life-learned rules and exceptions, dispositions and tendencies, balances and checks.
If common sense is so diverse and intricate, what makes it seem so obvious and natural? This illusion of simplicity comes from losing touch with what happened during infancy, when we formed our first abilities. As each new group of skills matures, we build more layers on top of them. As time goes on, the layers below become increasingly remote until, when we try to speak of them in later life, we find ourselves with little more to say than I don't know.
We want to explain intelligence as a combination of simpler things. This means that we must be sure to check, at every step, that none of our agents is, itself, intelligent. Otherwise, our theory would end up resembling the nineteenth-century chessplaying machine that was exposed by Edgar Allan Poe to actually conceal a human dwarf inside. Accordingly, whenever we find that an agent has to do anything complicated, we'll replace it with a subsociety of agents that do simpler things. Because of this, the reader must be prepared to feel a certain sense of loss. When we break things down to their smallest parts, they'll each seem dry as dust at first, as though some essence has been lost.
For example, we've seen how to construct a tower-building skill by making Builder from little parts like Find and Get. Now, where does its knowing-how-to-build reside when, clearly, it is not in any part — and yet those parts are all that Builder is? The answer: It is not enough to explain only what each separate agent does. We must also understand how those parts are interrelated — that is, how groups of agents can accomplish things.
Accordingly, each step in this book uses two different ways to think about agents. If you were to watch Builder work, from the outside, with no idea of how it works inside, you'd have the impression that it knows how to build towers. But if you could see Builder from the inside, you'd surely find no knowledge there.  Does Builder really know how to build towers? The answer depends on how you look at it. Let's use two different words, agent and agency, to say why Builder seems to lead a double life. As agency, it seems to know its job. As agent, it cannot know anything at all.
When you drive a car, you regard the steering wheel as an agency that you can use to change the car's direction. You don't care how it works. But when something goes wrong with the steering, and you want to understand what's happening, it's better to regard the steering wheel as just one agent in a larger agency: it turns a shaft that turns a gear to pull a rod that shifts the axle of a wheel. Of course, one doesn't always want to take this microscopic view; if you kept all those details in mind while driving, you might crash because it took too long to figure out which way to turn the wheel. Knowing how is not the same as knowing why. In this book, we'll always be switching between agents and agencies because, depending on our purposes, we'll have to use different viewpoints and kinds of descriptions.
It is the nature of the mind that makes individuals kin, and the differences in the shape, form, or manner of the material atoms out of whose intricate relationships that mind is built are trivial. —Isaac Asimov
We saw that Builder's skill could be reduced to the simpler skills of Get and Put. Then we saw how these, in turn, could be made of even simpler ones. Get merely needs to Move the hand to Grasp the block that Find just found. Put only has to Move the hand so that it puts that block upon the tower top. So it might appear that all of Builder's functions have been reduced to things that simpler parts can do.
But something important has been left out. Builder is not merely a collection of parts like Find, Get, Put, and all the rest. For Builder would not work at all unless those agents were linked to one another by a suitable network of interconnections.
Could you predict what Builder does from knowing just that left-hand list? Of course not: you must also know which agents work for which. Similarly, you couldn't predict what would happen in a human community from knowing only what each separate individual can do; you must also know how they are organized — that is, who talks to whom. And it's the same for understanding any large and complex thing. First, we must know how each separate part works. Second, we must know how each part interacts with those to which it is connected. And third we have to understand how all these local interactions combine to accomplish what that system does — as seen from the outside.
In the case of the human brain, it will take a long time to solve these three kinds of problems. First we will have to understand how brain cells work. This will be difficult because there are hundreds of different types of brain cells. Then we'll have to understand how the cells of each type interact with the other types of cells to which they connect. There could be thousands of these different kinds of interactions. Then, finally, comes the hardest part: we'll also have to understand how our billions of brain cells are organized into societies. To do this, we'll need to develop many new theories and organizational concepts. The more we can find out about how our brains evolved from those of simpler animals, the easier that task will be.
It's always best when mysteries can be explained in terms of things we know. But when we find this hard to do, we must decide whether to keep trying to make old theories work or to discard them and try new ones. I think this is partly a matter of personality. Let's call Reductionists those people who prefer to build on old ideas, and Novelists the ones who like to champion new hypotheses. Reduction- ists are usually right — at least at science's cautious core, where novelties rarely survive for long. Outside that realm, though, novelists reign, since older ideas have had more time to show their flaws.
It really is amazing how certain sciences depend upon so few kinds of explanations. The science of physics can now explain virtually everything we see, at least in principle, in terms of how a very few kinds of particles and force-fields interact. Over the past few centuries reductionism has been remarkably successful. What makes it possible to describe so much of the world in terms of so few basic rules? No one knows.
Many scientists look on chemistry and physics as ideal models of what psychology should be like. After all, the atoms in the brain are subject to the same all-inclusive physical laws that govern every other form of matter. Then can we also explain what our brains actually do entirely in terms of those same basic principles? The answer is no, simply because even if we understood how each of our billions of brain cells works separately, this would not tell us how the brain works as an agency. The laws of thought depend not only upon the properties of those brain cells, but also on how they are connected. And these connections are established not by the basic, general laws of physics, but by the particular arrangements of the millions of bits of information in our inherited genes. To be sure, general laws apply to everything. But, for that very reason, they can rarely explain anything in particular.
Does this mean that psychology must reject the laws of physics and find its own? Of course not. It is not a matter of different laws, but of additional kinds of theories and principles that operate at higher levels of organization. Our ideas of how Builder works as an agency need not, and must not, conflict with our knowledge of how Builder's lower-level agents work. Each higher level of description must add to our knowledge about lower levels, rather than replace it. We'll return to the idea of level at many places in this book.
Will psychology ever resemble any of the sciences that have successfully reduced their subjects to only a very few principles? That depends on what you mean by few. In physics, we're used to explanations in terms of perhaps a dozen basic principles. For psychology, our explanations will have to combine hundreds of smaller theories. To physicists, that number may seem too large. To humanists, it may seem too small.
We're often told that certain wholes are more than the sum of their parts. We hear this expressed with reverent words like holistic and gestalt, whose academic tones suggest that they refer to clear and definite ideas. But I suspect the actual function of such terms is to anesthetize a sense of ignorance. We say gestalt when things combine to act in ways we can't explain, holistic when we're caught off guard by unexpected happenings and realize we understand less than we thought we did. For example, consider the two sets of questions below, the first subjective and the second objective :
What makes a drawing more than just its separate lines? How is a personality more than a set of traits? In what way is a culture more than a mere collection of customs? What makes a tower more than separate blocks? Why is a chain more than its various links? How is a wall more than a set of many bricks?
Why do the objective questions seem less mysterious? Because we have good ways to answer them — in terms of how things interact. To explain how walls and towers work, we just point out how every block is held in place by its neighbors and by gravity. To explain why chain-links cannot come apart, we can demonstrate how each would get in its neighbors' way. These explanations seem almost self-evident to adults. However, they did not seem so simple when we were children, and it took each of us several years to learn how real-world objects interact — for example, to prevent any two objects from ever being in the same place. We regard such knowledge as obvious only because we cannot remember how hard it was to learn.
Why does it seem so much harder to explain our reactions to drawings, personalities, and cultural traditions? Many people assume that those subjective kinds of questions are impossible to answer because they involve our minds. But that doesn't mean they can't be answered. It only means that we must first know more about our minds.
Subjective reactions are also based on how things interact. The difference is that here we are not concerned with objects in the world outside, but with processes inside our brains.
In other words, those questions about arts, traits, and styles of life are actually quite technical. They ask us to explain what happens among the agents in our minds. But this is a subject about which we have never learned very much — and neither have our sciences. Such questions will be answered in time. But it will just prolong the wait if we keep using pseudo-explanation words like holistic and gestalt. True, sometimes giving names to things can help by leading us to focus on some mystery. It's harmful, though, when naming leads the mind to think that names alone bring meaning close.
It has been the persuasion of an immense majority of human beings that sensibility and thought [as distinguished from matter] are, in their own nature, less susceptible of division and decay, and that, when the body is resolved into its elements, the principle which animated it will remain perpetual and unchanged. However, it is probable that what we call thought is not an actual being, but no more than the relation between certain parts of that infinitely varied mass, of which the rest of the universe is composed, and which ceases to exist as soon as those parts change their position with respect to each other. —Percy Bysshe Shelley
What is Life? One dissects a body but finds no life inside. What is Mind? One dissects a brain but finds no mind therein. Are life and mind so much more than the sum of their parts that it is useless to search for them? To answer that, consider this parody of a conversation between a Holist and an ordinary Citizen.
Holist: I'll prove no box can hold a mouse. A box is made by nailing six boards together. But it's obvious that no box can hold a mouse unless it has some ‘mousetightness’ or ‘containment.’ Now, no single board contains any containment, since the mouse can just walk away from it. And if there is no containment in one board, there can't be any in six boards. So the box can have no mousetightness at all. Theoretically, then, the mouse can escape!
Citizen: Amazing. Then what does keep a mouse in a box?
Holist: Oh, simple. Even though it has no real mouse- tightness, a good box can ‘simulate’ it so well that the mouse is fooled and can't figure out how to escape.
What, then, keeps the mouse confined? Of course, it is the way a box prevents motion in all directions, because each board bars escape in a certain direction. The left side keeps the mouse from going left, the right from going right, the top keeps it from leaping out, and so on. The secret of a box is simply in how the boards are arranged to prevent motion in all directions!
That's what containing means. So it's silly to expect any separate board by itself to contain any containment, even though each contributes to the containing. It is like the cards of a straight flush in poker: only the full hand has any value at all.
The same applies to words like life and mind. It is foolish to use these words for describing the smallest components of living things because these words were invented to describe how larger assemblies interact. Like boxing-in, words like living and thinking are useful for describing phenomena that result from certain combinations of relationships. The reason box seems nonmysterious is that everyone understands how the boards of a well-made box interact to prevent motion in any direction. In fact, the word life has already lost most of its mystery — at least for modern biologists, because they understand so many of the important interactions among the chemicals in cells. But mind still holds its mystery — because we still know so little about how mental agents interact to accomplish all the things they do.
In the late 1960s Builder was embodied in the form of a computer program at the MIT Artificial Intelligence Laboratory. Both my collaborator, Seymour Papert, and I had long desired to combine a mechanical hand, a television eye, and a computer into a robot that could build with children's building-blocks. It took several years for us and our students to develop Move, See, Grasp, and hundreds of other little programs we needed to make a working Builder-agency.  The project left us wondering if even a thousand microskills would be enough to enable a child to fill a pail with sand. It was this body of experience, more than anything we'd learned about psychology, that led us to many ideas about societies of mind.
To do those first experiments, we had to build a mechanical Hand, equipped with sensors for pressure and touch at its fingertips. Then we had to interface a television camera with our computer and write programs with which that Eye could discern the edges of the building-blocks. It also had to recognize the Hand itself. When those programs didn't work so well, we added more programs that used the fingers' feeling-sense to verify that things were where they visually seemed to be. Yet other programs were needed to enable the computer to move the Hand from place to place while using the eye to see that there was nothing in its way. We also had to write higher-level programs that the robot could use for planning what to do — and still more programs to make sure that those plans were actually carried out. To make this all work reliably, we needed programs to verify at every step (again by using Eye and Hand) that what had been planned inside the mind did actually take place outside — or else to correct the mistakes that occurred.
In attempting to make our robot work, we found that many everyday problems were much more complicated than the sorts of problems, puzzles, and games adults consider hard. At every point, in that world of blocks, when we were forced to look more carefully than usual, we found an unexpected universe of complications. Consider just the seemingly simple problem of not reusing blocks already built into the tower. To a person, this seems simple common sense: Don't use an object to satisfy a new goal if that object is already involved in accomplishing a prior goal. No one knows exactly how human minds do this. Clearly we learn from experience to recognize the situations in which difficulties are likely to occur, and when we're older we learn to plan ahead to avoid such conflicts. But since we cannot be sure what will work, we must learn policies for dealing with uncertainty. Which strategies are best to try, and which will avoid the worst mistakes? Thousands and, perhaps, millions of little processes must be involved in how we anticipate, imagine, plan, predict, and prevent — and yet all this proceeds so automatically that we regard it as ordinary common sense. But if thinking is so complicated, what makes it seem so simple? At first it may seem incredible that our minds could use such intricate machinery and yet be unaware of it.
In general, we're least aware of what our minds do best.
It's mainly when our other systems start to fail that we engage the special agencies involved with what we call consciousness. Accordingly, we're more aware of simple processes that don't work well than of complex ones that work flawlessly. This means that we cannot trust our offhand judgments about which of the things we do are simple, and which require complicated machinery. Most times, each portion of the mind can only sense how quietly the other portions do their jobs.
Many people feel offended when their minds are likened to computer programs or machines. We've seen how a simple tower-building skill can be composed of smaller parts. But could anything like a real mind be made of stuff so trivial?
Ridiculous, most people say. I certainly don't feel like a machine!
 A person might reply, I think, therefore I know how the mind works. But that would be suspiciously like saying, I drive my car, therefore I know how its engine works. Knowing how to use something is not the same as knowing how it works.
But everyone knows that machines can behave only in lifeless, mechanical ways.
This objection seems more reasonable: indeed, a person ought to feel offended at being likened to any trivial machine. But it seems to me that the word machine is getting to be out of date. For centuries, words like mechanical made us think of simple devices like pulleys, levers, locomotives, and typewriters. (The word computerlike inherited a similar sense of pettiness, of doing dull arithmetic by little steps.) But we ought to recognize that we're still in an early era of machines, with virtually no idea of what they may become. What if some visitor from Mars had come a billion years ago to judge the fate of earthly life from watching clumps of cells that hadn't even learned to crawl? In the same way, we cannot grasp the range of what machines may do in the future from seeing what's on view right now.
Our first intuitions about computers came from experiences with machines of the 1940s, which contained only thousands of parts. But a human brain contains billions of cells, each one complicated by itself and connected to many thousands of others. Present-day computers represent an intermediate degree of complexity; they now have millions of parts, and people already are building billion-part computers for research on Artificial Intelligence. And yet, in spite of what is happening, we continue to use old words as though there had been no change at all. We need to adapt our attitudes to phenomena that work on scales never before conceived. The term machine no longer takes us far enough.
But rhetoric won't settle anything. Let's put these arguments aside and try instead to understand what the vast, unknown mechanisms of the brain may do. Then we'll find more self-respect in knowing what wonderful machines we are.
Most children not only like to build, they also like to knock things down. So let's imagine another agent called Wrecker, whose specialty is knocking-down. Our child loves to hear the complicated noises and watch so many things move all at once.
Suppose Wrecker gets aroused, but there's nothing in sight to smash. Then Wrecker will have to get some help — by putting Builder to work, for example. But what if, at some later time, Wrecker considers the tower to be high enough to smash, while Builder wants to make it taller still? Who could settle that dispute?
The simplest policy would be to leave that decision to Wrecker, who was responsible for activating Builder in the first place. But in a more realistic picture of a child's mind, such choices would depend on many other agencies. For example, let's assume that both Builder and Wrecker were originally activated by a higher-level agent, Play-with-Blocks. Then, a conflict might arise if Builder and Wrecker disagree about whether the tower is high enough.
What aroused Play-with-Blocks in the first place? Perhaps some even higher-level agent, Play, was active first. Then, inside Play, the agent Play-with-Blocks achieved control, in spite of two competitors, Play-with-Dolls and Play-with-Animals. But even Play itself, their mutual superior-in-chief, must have had to compete with other higher-level agencies like Eat and Sleep. For, after all, a child's play is not an isolated thing but always happens in the context of other real-life concerns. Whatever we may choose to do, there are always other things we'd also like to do.
In several sections of this book, I will assume that conflicts between agents tend to migrate upward to higher levels. For example, any prolonged conflict between Builder and Wrecker will tend to weaken their mutual superior, Play-with-Blocks. In turn, this will reduce Play-with-Blocks' ability to suppress its rivals, Play-with-Dolls and Play-with-Animals. Next, if that conflict isn't settled soon, it will weaken the agent Play at the next-higher level. Then Eat or Sleep might seize control.
To settle arguments, nations develop legal systems, corporations establish policies, and individuals may argue, fight, or compromise — or turn for help to mediators that lie outside themselves. What happens when there are conflicts inside minds?
Whenever several agents have to compete for the same resources, they are likely to get into conflicts. If those agents were left to themselves, the conflicts might persist indefinitely, and this would leave those agents paralyzed, unable to accomplish any goal. What happens then? We'll assume that those agents' supervisors, too, are under competitive pressure and likely to grow weak themselves whenever their subordinates are slow in achieving their goals, no matter whether because of conflicts between them or because of individual incompetence.
The Principle of Noncompromise: The longer an internal conflict persists among an agent's subordinates, the weaker becomes that agent's status among its own competitors. If such internal problems aren't settled soon, other agents will take control and the agents formerly involved will be dismissed.
So long as playing with blocks goes well, Play can maintain its strength and keep control. In the meantime, though, the child may also be growing hungry and sleepy, because other processes are arousing the agents Eat and Sleep. So long as Eat and Sleep are not yet strongly activated, Play can hold them both at bay. However, any conflict inside Play will weaken it and make it easier for Eat or Sleep to take over. Of course, Eat or Sleep must conquer in the end, since the longer they wait, the stronger they get.
We see this in our own experience. We all know how easy it is to fight off small distractions when things are going well. But once some trouble starts inside our work, we become increasingly impatient and irritable. Eventually we find it so hard to concentrate that the least disturbance can allow another, different, interest to take control.
Now, when any of our agencies loses the power to control what other systems do, that doesn't mean it has to cease its own internal activity. An agency that has lost control can continue to work inside itself — and thus become prepared to seize a later opportunity. However, we're normally unaware of all those other activities proceeding deep inside our minds.
Where does it stop, this process of yielding control to other agencies? Must every mind contain some topmost center of control? Not necessarily. We sometimes settle conflicts by appealing to superiors, but other conflicts never end and never cease to trouble us.
At first, our principle of noncompromise may seem too extreme. After all, good human supervisors plan ahead to avoid conflicts in the first place, and — when they can't — they try to settle quarrels locally before appealing to superiors. But we should not try to find a close analogy between the low-level agents of a single mind and the members of a human community. Those tiny mental agents simply cannot know enough to be able to negotiate with one another or to find effective ways to adjust to each other's interference. Only larger agencies could be resourceful enough to do such things. Inside an actual child, the agencies responsible for Building and Wrecking might indeed become versatile enough to negotiate by offering support for one another's goals. Please, Wrecker, wait a moment more till Builder adds just one more block: it's worth it for a louder crash!
bu•reauc′ra•cy n. the administration of government through departments and subdivisions managed by sets of officials following an inflexible routine. —Webster's Unabridged Dictionary
As an agent, Builder does no physical work but merely turns on Begin, Add, and End.  Then these divide into agents like Move and Grasp. It seems that it will never stop — this breaking-down to smaller things.
Eventually, it all must end with agents that do actual work, but there are many steps before we get to all the little muscle-motor agents that actually move the arms and hands and finger joints. Thus Builder is like a high-level executive, far removed from those subordinates who actually produce the final product.
Does this mean that Builder's administrative work is unimportant? Not at all. Those lower-level agents need to be controlled. It's much the same in human affairs. When any enterprise becomes too complex and large for one person to do, we construct organizations in which certain agents are concerned, not with the final result, but only with what some other agents do. Designing any society, be it human or mechanical, involves decisions like these:
Which agents choose which others to do what jobs? Who will decide which jobs are done at all? Who decides what efforts to expend? How will conflicts be settled?
How much of ordinary human thought has Builder' s character? The Builder we described is not much like a human supervisor. It doesn't decide which agents to assign to which jobs, because that has already been arranged. It doesn't plan its future work but simply carries out fixed steps until End says the job is done. Nor has it any repertoire of ways to deal with unexpected accidents.
Because our little mental agents are so limited, we should not try to extend very far the analogy between them and human supervisors and workers. Furthermore, as we'll shortly see, the relations between mental agents are not always strictly hierarchical. And in any case, such roles are always relative. To Builder, Add is a subordinate, but to Find, Add is a boss. As for yourself, it all depends on how you live. Which sorts of thoughts concern you most — the orders you are made to take or those you're being forced to give?
A hierarchical society is like a tree in which the agent at each branch is exclusively responsible for the agents on the twigs that branch from it. This pattern is found in every field, because dividing work into parts like that is usually the easiest way to start solving a problem. It is easy to construct and understand such organizations because each agent has only a single job to do: it needs only to look up for instructions from its supervisor, then look down to get help from its subordinates.
But hierarchies do not always work. Consider that when two agents need to use each other's skills, then neither one can be on top. Notice what happens, for example, when you ask your vision-system to decide whether the following left-side scene depicts three blocks — or only two.
The agent See could answer that if it could Move the front block out of the line of view. But, in the course of doing that, Move might have to See if there were any obstacles that might interfere with the arm's trajectory. At such a moment, Move would be working for See, and See would be working for Move, both at the same time. This would be impossible inside a simple hierarchy.
Most of the diagrams in the early parts of this book depict simple hierarchies. Later, we'll see more cross-connected rings and loops — when we are forced to consider the need for memory, which will become a constant subject of concern in this book. People often think of memory in terms of keeping records of the past, for recollecting things that happened in earlier times. But agencies also need other kinds of memory as well. See, for example, requires some sort of temporary memory in order to keep track of what next to do, when it starts one job before its previous job is done. If each of See's agents could do only one thing at a time, it would soon run out of resources and be unable to solve complicated problems. But if we have enough memory, we can arrange our agents into circular loops and thus use the same agents over and over again to do parts of several different jobs at the same time.
In any actual child's mind, the urge to Play competes with other demanding urges, such as Eat and Sleep. What happens if another agent wrests control from Play, and what happens to the agents Play controlled?
Suppose that our child is called away, no matter whether by someone else or by an internal urge like Sleep. What happens to the processes remaining active in the mind? One part of the child may still want to play, while another part wants to sleep. Perhaps the child will knock the tower down with a sudden, vengeful kick. What does it mean when children make such scenes? Is it that inner discipline breaks down to cause those savage acts? Not necessarily. Those childish acts might still make sense in other ways.
Smashing takes so little time that Wrecker, freed from Play's constraint, need persist for only one more kick to gain the satisfaction of a final crash.
Though childish violence might seem senseless by itself, it serves to communicate frustration at the loss of goal. Even if the parent scolds, that just confirms how well the message was transmitted and received. Destructive acts can serve constructive goals by leaving fewer problems to be solved. That kick may leave a mess outside, yet tidy up the child's mind.
When children smash their treasured toys, we shouldn't ask for the reason why — since no such act has a single cause. Besides, it isn't true in a human mind that, when Sleep starts, then Play must quit and all its agents have to cease. A real child can go to bed — yet still build towers in its head.
When you're in pain, it's hard to keep your interest in other things. You feel that nothing's more important than finding some way to stop the pain. That's why pain is so powerful: it makes it hard to think of anything else. Pain simplifies your point of view.
When something gives you pleasure, then, too, it's hard to think of other things. You feel that nothing's more important than finding a way to make that pleasure last. That's why pleasure is so powerful. It also simplifies your point of view.
Pain's power to distract us from our other goals is not an accident; that's how it helps us to survive. Our bodies are endowed with special nerves that detect impending injuries, and the signals from these nerves for pain make us react in special ways. Somehow, they disrupt our concerns with long-term goals — thus forcing us to focus on immediate problems, perhaps by transferring control to our lowest-level agencies. Of course, this can do more harm than good, especially when, in order to remove the source of pain, one has to make a complex plan. Unfortunately, pain interferes with making plans by undermining interest in anything that's not immediate. Too much suffering diminishes us by restricting the complexities that constitute our very selves. It must be the same for pleasure as well.
We think of pleasure and pain as opposites, since pleasure makes us draw its object near while pain impels us to reject its object. We also think of them as similar, since both make rival goals seem small by turning us from other interests. They both distract. Why do we find such similarities between antagonistic things? Sometimes two seeming opposites are merely two extremes along a single scale, or one of them is nothing but the absence of the other — as in the case of sound and silence, light and darkness, interest and unconcern. But what of opposites that are genuinely different, like pain and pleasure, fear and courage, hate and love?
In order to appear opposed, two things must serve related goals — or otherwise engage the selfsame agencies.
Thus, affection and abhorrence both involve our attitudes toward relationships; and pleasure and pain both engage constraints that simplify our mental scenes. The same goes for courage and cowardice: each does best by knowing both. When on attack, you have to press against whatever weakness you can find in your opponent's strategy. When on defense, it's much the same: you still must guess the other's plan.
We are what we pretend to be, so we must be careful about what we pretend to be. —Kurt Vonnegut
self n. 1. the identity, character, or essential qualities of any person or thing. 2. the identity, personality, individuality, etc. of a given person; one's own person as distinct from all others. —Webster's Unabridged Dictionary
We all believe that human minds contain those special entities we call selves. But no one agrees about what they are. To keep things straight, I shall write self when speaking in a general sense about an entire person and reserve Self for talking about that more mysterious sense of personal identity. Here are some of the things people say about the Self:
Self is the part of mind that's really me, or rather, it's the part of me — that is, part of my mind — that actually does the thinking and wanting and deciding and enjoying and
suffering. It's the part that's most important to me because it's that which stays the same through all experience — the identity which ties everything together. And whether you can treat it scientifically or not, I know it's there, because it's me. Perhaps it's the sort of thing that Science can't explain.
This isn't much of a definition, but I don't think it is a good idea to try to find a better one. It often does more harm than good to force definitions on things we don't understand. Besides, only in logic and mathematics do definitions ever capture concepts perfectly. The things we deal with in practical life are usually too complicated to be represented by neat, compact expressions. Especially when it comes to understanding minds, we still know so little that we can't be sure our ideas about psychology are even aimed in the right directions. In any case, one must not mistake defining things for knowing what they are. You can know what a tiger is without defining it. You may define a tiger, yet know scarcely anything about it.
Even if our old ideas about the mind are wrong, we can learn a lot by trying to understand why we believe them. Instead of asking, What are Selves? we can ask, instead, What are our ideas about Selves? — and then we can ask, What psychological functions do those ideas serve? When we do this, it shows us that we do not have one such idea, but many.
Our ideas about our Selves include beliefs about what we are. These include beliefs both about what we are capable of doing and about what we may be disposed to do. We exploit these beliefs whenever we solve problems or make plans. I'll refer to them, rather vaguely, as a person's self-images. In addition to our self-images, our ideas about ourselves also include ideas about what we'd like to be and ideas about what we ought to be. These, which I'll call a person's self-ideals, influence each person's growth from infancy, but we usually find them hard to express because they're inaccessible to consciousness.
One common image of the Self suggests that every mind contains some sort of Voyeur-Puppeteer inside — to feel and want and choose for us the things we feel, want, and choose. But if we had those kinds of Selves, what would be the use of having Minds? And, on the other hand, if Minds could do such things themselves, why have Selves? Is this concept of a Self of any real use at all? It is indeed — provided that we think of it not as a centralized and all-powerful entity, but as a society of ideas that include both our images of what the mind is and our ideals about what it ought to be.
Besides, we're often of two minds about ourselves. Sometimes we regard ourselves as single, self-coherent entities. Other times we feel decentralized or dispersed, as though we were made of many different parts with different tendencies. Contrast these views:
SINGLE-SELF VIEW. I think, I want, I feel. It's me, myself, who thinks my thoughts. It's not some nameless crowd or cloud of selfless parts.
MULTIPLE-SELF VIEW. One part of me wants this, another part wants that. I must get better control of myself.
We're never wholly satisfied with either view. We all sense feelings of disunity, conflicting motives, compulsions, internal tensions, and dissensions. We carry on negotiations in our head. We hear scary tales in which some person's mind becomes enslaved by compulsions and commands that seem to come from somewhere else. And the times we feel most reasonably unified can be just the times that others see us as the most confused.
But if there is no single, central, ruling Self inside the mind, what makes us feel so sure that one exists? What gives that myth its force and strength? A paradox: perhaps it's because there are no persons in our heads to make us do the things we want — nor even ones to make us want to want — that we construct the myth that we're inside ourselves.
And we thank Thee that darkness reminds us of light. —T.S. Eliot
A common concept of the soul is that the essence of a self lies in some spark of invisible light, a thing that cowers out of body, out of mind, and out of sight. But what might such a symbol mean? It carries a sense of anti-self-respect: that there is no significance in anyone's accomplishments.
People ask if machines can have souls. And I ask back whether souls can learn. It does not seem a fair exchange — if souls can live for endless time and yet not use that time to learn — to trade all change for changelessness. And that's exactly what we get with inborn souls that cannot grow: a destiny the same as death, an ending in a permanence incapable of any change and, hence, devoid of intellect.
Why try to frame the value of a Self in such a singularly frozen form? The art of a great painting is not in any one idea, nor in a multitude of separate tricks for placing all those pigment spots, but in the great network of relationships among its parts. Similarly, the agents, raw, that make our minds are by themselves as valueless as aimless, scattered daubs of paint. What counts is what we make of them.
 But minds are just the opposite. We start as little embryos, which then build great and wondrous selves — whose merit lies entirely within their own coherency. The value of a human self lies not in some small, precious core, but in its vast, constructed crust.
What are those old and fierce beliefs in spirits, souls, and essences? They're all insinuations that we're helpless to improve ourselves. To look for our virtues in such thoughts seems just as wrongly aimed a search as seeking art in canvas cloths by scraping off the painter's works.
How do we control our minds? Ideally, we first choose what we want to do, then make ourselves do it. But that's harder than it sounds: we spend our lives in search of schemes for self-control. We celebrate when we succeed, and when we fail, we're angry with ourselves for not behaving as we wanted to — and then we try to scold or shame or bribe ourselves to change our ways. But wait! How could a self be angry with itself? Who would be mad at whom? Consider an example from everyday life.
I was trying to concentrate on a certain problem but was getting bored and sleepy. Then I imagined that one of my competitors, Professor Challenger, was about to solve the same problem. An angry wish to frustrate Challenger then kept me working on the problem for a while. The strange thing was, this problem was not of the sort that ever interested Challenger.
What makes us use such roundabout techniques to influence ourselves? Why be so indirect, inventing misrepresentations, fantasies, and outright lies? Why can't we simply tell ourselves to do the things we want to do?
To understand how something works, one has to know its purposes. Once, no one understood the heart. But as soon as it was seen that hearts move blood, a lot of other things made sense: those things that looked like pipes and valves were really pipes and valves indeed — and anxious, pounding, pulsing hearts were recognized as simple pumps. New speculations could then be formed: was this to give our tissues drink or food? Was it to keep our bodies warm or cool? For sending messages from place to place? In fact, all those hypotheses were correct, and when that surge of functional ideas led to the guess that blood can carry air as well, more puzzle parts fell into place.
To understand what we call the Self, we first must see what Selves are for. One function of the Self is to keep us from changing too rapidly. Each person must make some long-range plans in order to balance single-purposeness against attempts to do everything at once. But it is not enough simply to instruct an agency to start to carry out our plans. We also have to find some ways to constrain the changes we might later make — to prevent ourselves from turning those plan-agents off again! If we changed our minds too recklessly, we could never know what we might want next. We'd never get much done because we could never depend on ourselves.
Those ordinary views are wrong that hold that Selves are magic, self-indulgent luxuries that enable our minds to break the bonds of natural cause and law. Instead, those Selves are practical necessities. The myths that say that Selves embody special kinds of liberty are merely masquerades. Part of their function is to hide from us the nature of our self-ideals — the chains we forge to keep ourselves from wrecking all the plans we make.
Let's look more closely at that episode of Professor Challenger. Apparently, what happened was that my agency for Work exploited Anger to stop Sleep. But why should Work use such a devious trick?
To see why we have to be so indirect, consider some alternatives. If Work could simply turn off Sleep, we'd quickly wear our bodies out. If Work could simply switch Anger on, we'd be fighting all the time. Directness is too dangerous.
Extinction would be swift indeed for species that could simply switch off hunger or pain. Instead, there must be checks and balances. We'd never get through one full day if any agency could seize and hold control over all the rest. This must be why our agencies, in order to exploit each other's skills, have to discover such roundabout pathways. All direct connections must have been removed in the course of our evolution.
This must be one reason why we use fantasies: to provide the missing paths. You may not be able to make yourself angry simply by deciding to be angry, but you can still imagine objects or situations that make you angry. In the scenario about Professor Challenger, my agency Work exploited a particular memory to arouse my Anger's tendency to counter Sleep. This is typical of the tricks we use for self-control.
Most of our self-control methods proceed unconsciously, but we sometimes resort to conscious schemes in which we offer rewards to ourselves: If I can get this project done, I'll have more time for other things. However, it is not such a simple thing to be able to bribe yourself. To do it successfully, you have to discover which mental incentives will actually work on yourself. This means that you — or rather, your agencies — have to learn something about one another's dispositions. In this respect the schemes we use to influence ourselves don't seem to differ much from those we use to exploit other people — and, similarly, they often fail. When we try to induce ourselves to work by offering ourselves rewards, we don't always keep our bargains; we then proceed to raise the price or even to deceive ourselves, much as one person may try to conceal an unattractive aspect of a bargain from another person.
Human self-control is no simple skill, but an ever-growing world of expertise that reaches into everything we do. Why is it that, in the end, so few of our self-incentive tricks work well? Because, as we have seen, directness is too dangerous. If self-control were easy to obtain, we'd end up accomplishing nothing at all.
Those who really seek the path to Enlightenment dicate terms to their mind. Then they proceed with strong determination. —Buddha
The episode of Professor Challenger showed just one way we can control ourselves: by exploiting an emotional aversion in order to accomplish an intellectual purpose. Consider all the other kinds of tricks we use to try to force ourselves to work when we're tired or distracted.
WILLPOWER: Tell yourself, Don't give in to that, or, Keep on trying.
Such self-injunctions can work at first — but finally they always fail, as though some engine in the mind runs out of fuel. Another style of self-control involves more physical activity:
ACTIVITY: Move around. Exercise. Inhale. Shout.
Certain physical acts are peculiarly effective, especially the facial expressions involved in social communication: they affect the sender as much as the recipient.
EXPRESSION: Set jaw. Stiffen upper lip. Furrow brow.
Another kind of stimulating act is moving to a stimulating place. And we often perform actions that directly change the brain's chemical environment.
CHEMISTRY: Take coffee, amphetamines, or other brain-affecting drugs.
Then there are actions in the mind with which we set up thoughts and fantasies that move our own emotions, arousing hopes and fears through self-directed offers, bribes, and even threats.
EMOTION: If I win, there's much to gain, but more to lose if I fail!
Perhaps most powerful of all are those actions that promise gain or loss of the regard of certain special persons.
ATTACHMENT: Imagine admiration if you succeed — or disapproval if you fail — especially from those to whom you are attached.
So many schemes for self-control! How do we choose which ones to use? There isn't any easy way. Self-discipline takes years to learn; it grows inside us stage by stage.
We often become involved in projects that we can't complete. It is easy to solve small problems because we can treat them as though they were detached from all our other goals. But it is different for projects that span larger portions of our lives, like learning a trade, raising a child, or writing a book. We cannot simply decide or choose to accomplish an enterprise that makes a large demand for time, because it will inevitably conflict with other interests and ambitions. Then we'll be forced to ask questions like these:
What must I give up for this? What will I learn from it? Will it bring power and influence? Will I remain interested in it? Will other people help me with it? Will they still like me?
Perhaps the most difficult question of all is, How will adopting this goal change me? Just wanting to own a large, expensive house, for instance, can lead to elaborate thoughts like these:
That means I'd have to save for years and not get other things I'd like. I doubt that I could bear it. True, I could reform myself, and try to be more thrifty and deliberate. But that's just not the sort of person I am.
Until such doubts are set aside, all the plans we make will be subject to the danger that we may change our mind. So how can any long-range plan succeed? The easiest path to self-control is doing only what one is already disposed to do.
Many of the schemes we use for self-control are the same as those we learn to use for influencing other people. We make ourselves behave by exploiting our own fears and desires, offering ourselves rewards, or threatening the loss of what we love. But when short-range tricks won't keep us to our projects for long enough, we may need some way to make changes that won't let us change ourselves back again. I suspect that, in order to commit ourselves to our largest, most ambitious plans, we learn to exploit agencies that operate on larger spans of time.
Which are our slowest-changing agencies of all? Later we'll see that these must include the silent, hidden agencies that shape what we call character. These are the systems that are concerned not merely with the things we want, but with what we want ourselves to be — that is, the ideals we set for ourselves.
We usually reserve the word ideals to refer to how we think we ought to conduct our ethical affairs. But I'll use the term in a broader sense, to include the standards we maintain — consciously or otherwise — for how we ought to think about ordinary matters.
We're always involved with goals of varying spans and scales. What happens when a transient inclination clashes with a long-term self-ideal? What happens, for that matter, when our ideals disagree among themselves, as when there is an inconsistency between the things we want to do and those we feel we ought to do? These disparities give rise to feelings of discomfort, guilt, and shame. To lessen such disturbances, we must either change the things we do — or change the ways we feel. Which should we try to modify — our immediate wants or our ideals? Such conflicts must be settled by the multilayered agencies that are formed in the early years of the growth of our personalities.
In childhood, our agencies acquire various types of goals. Then we grow in overlapping waves, in which our older agencies affect the making of the new. This way, the older agencies can influence how our later ones will behave. Outside the individual, similar processes go on in every human community; we find children taking after persons other than themselves by absorbing values from their parents, families, and peers, even from the heroes and villains of mythology.
Without enduring self-ideals, our lives would lack coherence. As individuals, we'd never be able to trust ourselves to carry out our personal plans. In a social group, no one person would be able to trust the others. A working society must evolve mechanisms that stabilize ideals — and many of the social principles that each of us regards as personal are really long-term memories in which our cultures store what they have learned across the centuries.
Whenever we can, we like to explain things in terms of simple cause and effect. We explained the case of Professor Challenger by assuming that my wish to Work came first, then Work exploited Anger's aptitude for fighting Sleep. But in real life the causal relations between feelings and thoughts are rarely so simple. My desire to work and my annoyance with Challenger were probably so intermingled, all along, that it is inappropriate to ask which came first, Anger or Work. Most likely, both agencies exploited one another simultaneously, thus combining both into a single fiendish synthesis that accomplished two goals at once; Work thus got to do its work — and, thereby, injured Challenger! (In an academic rivalry, a technical accomplishment can hurt more than a fist.) Two goals can support each other.
A causes B John wanted to go home because he felt tired of work. B causes A John felt tired of work because he wanted to go home.
There need be no first cause since John could start out with both distaste for work and inclination to go home. Then a loop of circular causality ensues, in which each goal gains support from the other until their combined urge becomes irresistible. We're always enmeshed in causal loops. Suppose you had borrowed past your means and later had to borrow more in order to pay the interest on your loan. If you were asked what the difficulty was, it would not be enough to say simply, Because I have to pay the interest, or to say only, Because I have to pay the principal. Neither alone is the actual cause, and you'd have to explain that you're caught in a loop.
We often speak of straightening things out when we're involved in situations that seem too complicated. It seems to me that this metaphor reflects how hard it is to find one's way through a maze that has complicated loops in it. In such a situation, we always try to find a path through it by seeking causal explanations that go in only one direction. There's a good reason for doing this.
There are countless different types of networks that contain loops. But all networks that contain no loops are basically the
same: each has the form of a simple chain.
Because of this, we can apply the very same types of reasoning to everything we can represent in terms of chains of causes and effects. Whenever we accomplish that, we can proceed from start to end without any need for a novel thought; that's what we mean by straightening out. But frequently, to construct such a path, we have to ignore important interactions and dependencies that run in other directions.
When we reflect on anything for long enough, we're likely to end up with what we sometimes call basic questions — ones we can see no way at all to answer. For we have no perfect way to answer even this question: How can one tell when a question has been properly answered?
What caused the universe, and why? How can you tell which beliefs are true? What is the purpose of life? How can you tell what is good?
These questions seem different on the surface, but all of them share one quality that makes them impossible to answer: all of them are circular! You can never find a final cause, since you must always ask one question more: What caused that cause? You can never find any ultimate goal, since you're always obliged to ask, Then what purpose does that serve? Whenever you find out why something is good — or is true — you still have to ask what makes that reason good and true. No matter what you discover, at every step, these kinds of questions will always remain, because you have to challenge every answer with, Why should I accept that answer? Such circularities can only waste our time by forcing us to repeat, over and over and over again, What good is Good? and, What god made God?
When children keep on asking, Why? we adults learn to deal with this by simply saying, Just because! This may seem obstinate, but it's also a form of self-control. What stops adults from dwelling on such questions endlessly? The answer is that every culture finds special ways to deal with these questions. One way is to brand them with shame and taboo; another way is to cloak them in awe or mystery; both methods make those questions undiscussable. Consensus is the simplest way — as with those social styles and trends wherein we each accept as true whatever all the others do. I think I once heard W. H. Auden say, We are all here on earth to help others. What I can't figure out is what the others are here for.
All human cultures evolve institutions of law, religion, and philosophy, and these institutions both adopt specific answers to circular questions and establish authority-schemes to indoctrinate people with those beliefs. One might complain that such establishments substitute dogma for reason and truth. But in exchange, they spare whole populations from wasting time in fruitless reason loops. Minds can lead more productive lives when working on problems that can be solved.
But when thinking keeps returning to its source, it doesn't always mean something's wrong. For circular thinking can lead to growth when it results, at each return, in deeper and more powerful ideas. Then, because we can communicate, such systems of ideas may even find the means to cross the boundaries of selfish selves — and thus take root in other minds. This way, a language, science, or philosophy can transcend the limitation of each single mind's mortality. Now, we cannot know that any individual is destined for some paradise. Yet certain religions are oddly right; they manage to achieve their goal of offering an afterlife — if only to their own strange souls.
When people have no answers to important questions, they often give some anyway.
| What controls the brain? | The Mind. |
| What controls the mind? | The Self. |
| What controls the Self? | Itself. |
To help us think about how our minds are connected to the outer world, our culture teaches schemes like this:
This diagram depicts our sensory machinery as sending information to the brain, wherein it is projected on some inner mental movie screen. Then, inside that ghostly theater, a lurking Self observes the scene and then considers what to do. Finally, that Self may act — somehow reversing all those steps — to influence the real world by sending various signals back through yet another family of remote-control accessories.
This concept simply doesn't work. It cannot help for you to think that inside yourself lies someone else who does your work. This notion of homunculus — a little person inside each self — leads only to a paradox since, then, that inner Self requires yet another movie screen inside itself, on which to project what it has seen! And then, to watch that play-within-a-play, we'd need yet another Self-inside-a-Self — to do the thinking for the last. And then this would all repeat again, as each new Self requires yet another one to do its job!
The idea of a single, central Self doesn't explain anything. This is because a thing with no parts provides nothing that we can use as pieces of explanation!
Then why do we so often embrace the strange idea that what we do is done by Someone Else — that is, our Self? Because so much of what our minds do is hidden from the parts of us that are involved with consciousness.
Why do we accept that paradoxical image of a central Self inside the self? Because it serves us well in many spheres of practical life. Here are some reasons to regard a person as a single thing.
The Physical World: Our bodies act like other objects that take up space. Because of that, we must base our plans and decisions on having a single body. Two people cannot fit where there is room for only one — nor can a person walk through walls or stay aloft without support.
Personal Privacy: When Mary tells Jack something, she must remember to whom it was told, and she must not assume that every other person knows it, too. Also, without the concept of an individual, we could have no sense of responsibility. Mental Activity: We often find it hard to think two different thoughts at once particularly when they're similar, because we get confused when the same agencies are asked to do different jobs at the same time.
Why do our mental processes so often seem to us to flow in streams of consciousness ? Perhaps because, in order to keep control, we have to simplify how we represent what's happening. Then, when that complicated mental scene is straightened out, it seems as though a single pipeline of ideas were flowing through the mind.
These are all compelling reasons why it helps to see ourselves as singletons. Still, each of us must also learn not only that different people have their own identities, but that the same person can entertain different beliefs, plans, and dispositions at the same time.
For finding good ideas about psychology, the single-agent image has become a grave impediment. To comprehend the human mind is surely one of the hardest tasks any mind can face. The legend of the single Self can only divert us from the target of that inquiry.
Why do we like so many things that seem to us to have no earthly use? We often speak of this with mixtures of defensiveness and pride.
Why do we take refuge in such vague, defiant declarations? There's no accounting for it sounds like a guilty child who's been told to keep accounts. And I just like it sounds like a person who is hiding reasons too unworthy to admit. However, we often do have sound practical reasons for making choices that have no reasons by themselves but have effects on larger scales.
Recognizability: The legs of a chair work equally well if made square or round. Then why do we tend to choose our furniture according to systematic styles or fashions? Because familiar styles make it easier for us to recognize and classify the things we see. Uniformity: If every object in a room were interesting in itself, our furniture might occupy our minds too much. By adopting uniform styles, we protect ourselves from distractions. Predictability: It makes no difference whether a single car drives on the left or on the right. But it makes all the difference when there are many cars! Societies need rules that make no sense for individuals.
It can save a lot of mental work if one makes each arbitrary choice the way one did before. The more difficult the decision, the more this policy can save. The following observation by my associate, Edward Fredkin, seems important enough to deserve a name:
Fredkin's Paradox: The more equally attractive two alternatives seem, the harder it can be to choose between them — no matter that, to the same degree, the choice can only matter less.
No wonder we often can't account for taste — if it depends on hidden rules that we use when ordinary reasons cancel out! I do not mean to say that fashion, style, and art are all the same — only that they often share this strategy of using forms that lie beneath the surface of our thoughts. When should we quit reasoning and take recourse in rules of style? Only when we're fairly sure that further thought will just waste time. Perhaps that's why we often feel such a sense of being free from practicality when we make aesthetic choices. Such decisions might seem more constrained if we were aware of how they're made. And what about those fleeting hints of guilt we sometimes feel for just liking art? Perhaps they're how our minds remind themselves not to abandon thought too recklessly.
Isn't it remarkable that words can portray human individuals? You might suppose this should be impossible, considering how much there is to say. Then what permits a writer to depict such seemingly real personalities? It is because we all agree on so many things that are left unsaid. For example, we assume that all the characters are possessed of what we call commonsense knowledge, and we also agree on many generalities about what we call human nature.
Hostility evokes defensiveness. Frustration arouses aggression.
We also recognize that individuals have particular qualities and traits of character.
Jane is tidy. Mary's timid. Grace is smart. That's not the sort of thing Charles does. It's not his style.
Why should traits like these exist? Humanists are prone to boast about how hard it is to grasp the measure of a mind. But let's ask instead, What makes personalities so easy to portray? Why, for example, should any person tend toward a general quality of being neat, rather than simply being tidy about some things and messy about others? Why should our personalities show such coherencies? How could it be that a system assembled from a million agencies can be described by short and simple strings of words? Here are some possible reasons.
Selectivity: First we should face the fact that our images of other minds are often falsely clear. We tend to think of another person's personality in terms of that which we can describe — and tend to set aside the rest as though it simply weren't there. Style: To escape the effort of making decisions we consider unimportant, we tend to develop policies that become so systematic that they can be discerned from the outside and characterized as personal traits.
Predictability: Because it is hard to maintain friendship without trust, we try to conform to the expectations of our friends. Then, to the extent that we frame our images of our associates in terms of traits, we find ourselves teaching ourselves to behave in accord with those same descriptions. Self-Reliance: Thus, over time, imagined traits can make themselves actual! For even to carry out our own plans, we must be able to predict what we ourselves are likely to do — and that will become easier the more we simplify ourselves.
It's nice to be able to trust our friends, but we need to be able to trust ourselves. How can that be possible when we can't be sure what's in our own heads? One way to accomplish this is by thinking of ourselves in terms of traits — and then proceeding to train ourselves to behave according to those self-images. Still, a personality is merely the surface of a person. What we call traits are only the regularities we manage to perceive. We never really know ourselves because there are so many other processes and policies that never show themselves directly in our behavior but work behind the scenes.
What do we signify by words like me, myself, and I ? What does a story mean that starts with In my childhood ? What is that strange possession you, which stays the same throughout your life? Are you the same person you were before you learned to read? You scarcely can imagine, now, how words looked then. Just try to look at these words without reading them:
We find it almost impossible to separate the appearances of things from what they've come to mean to us. But if we cannot recollect how things appeared to us before we learned to link new meanings to those things, what makes us think we can recollect how we ourselves appeared to us in previous times? What would you say if someone asked questions like these:
Are you the same person now that you once were, before you learned to talk? Of course I am. Why, who else could I be? Do you mean that you haven't changed at all? Of course not. I only mean I'm the same person — the same in some ways, different in others — but still the same me. But how can you be the same as the person you were before you had even learned to remember things? Can you even imagine what that was like? Perhaps I can't — yet still there must have been some continuity. Even if I can't remember it, I surely was that person, too.
We all experience that sense of changelessness in spite of change, not only for the past but also for the future, too! Consider how you are generous to future self at present self's expense. Today, you put some money in the bank in order that sometime later you can take it out. Whenever did that future self do anything so good for you? Is you the body of those memories whose meanings change only slowly? Is it the never-ending side effects of all your previous experience? Or is it just whichever of your agents change the least as time and life proceed?
In real life, you often have to deal with things you don't completely understand. You drive a car, not knowing how its engine works. You ride as passenger in someone else's car, not knowing how that driver works.  Isn't it amazing that we can think, not knowing what it means to think? Isn't it remarkable that we can get ideas, yet not explain what ideas are?
In every normal person's mind there seem to be some processes that we call consciousness. We usually regard them as enabling us to know what's happening inside our minds. But this reputation of self-awareness is not so well deserved, because our conscious thoughts reveal to us so little of what gives rise to them.
Consider how a driver guides the immense momentum of a motorcar, not knowing how its engine works or how its steering wheel directs it to the left or right. Yet when one comes to think of it, we drive our bodies in much the same way. So far as conscious thought is concerned, you turn yourself to walk in a certain direction in much the way you steer a car; you are aware only of some general intention, and all the rest takes care of itself. To change your direction of motion is actually quite complicated. If you simply took a larger or smaller step on one side, the way you would turn a rowboat, you would fall toward the outside of the turn. Instead, you start to turn by making yourself fall toward the inside — and then use centrifugal force to right yourself on the next step. This incredible process involves a huge society of muscles, bones, and joints, all controlled by hundreds of interacting programs that even specialists don't yet understand. Yet all you think is, Turn that way,
and your wish is automatically fulfilled.
We give the name signals to acts whose consequences are not inherent in their own character but have merely been assigned to them. When you accelerate your car by pressing on the gas pedal, this is not what does the work; it is merely a signal to make the engine push the car. Similarly, rotating the steering wheel is merely a signal that makes the steering mechanism turn the car. The car's designer could easily have assigned the pedal to steer the car or made the steering wheel control its speed. But practical designers try to exploit the use of signals that already have acquired some significance.
Our conscious thoughts use signal-signs to steer the engines in our minds, controlling countless processes of which we're never much aware. Not understanding how it's done, we learn to gain our ends by sending signals to those great machines, much as the sorcerers of older times used rituals to cast their spells.
How do we ever understand anything? Almost always, I think, by using one or another kind of analogy — that is, by representing each new thing as though it resembles something we already know. Whenever a new thing's internal workings are too strange or complicated to deal with directly, we represent whatever parts of it we can in terms of more familiar signs. This way, we make each novelty seem similar to some more ordinary thing. It really is a great discovery, the use of signals, symbols, words, and names. They let our minds transform the strange into the commonplace.
Suppose an alien architect has invented a radically new way to go from one room to another. This invention serves the normal functions of a door, but it has a form and mechanism so far outside our experience that to see it, we would never recognize it as a door, nor guess how to use it. All its physical details are wrong. It is not what we normally expect a door to be — a hinged, swinging, wooden slab set into a wall. No matter: just superimpose on its exterior some decoration, symbol, icon, token, word, or sign that can remind us of its use. Clothe it in a rectangular shape, or add to it a push-plate lettered EXIT in red and white, and every visitor from the planet Earth will know, without a conscious thought, just what that pseudoportal's purpose is, and use it as though it were a door.
At first it may seem mere trickery, to assign the symbol for a door to an invention that is not really a door. But we're always in that same predicament. There are no doors inside our minds, only connections among our signs. To overstate the case a bit, what we call consciousness consists of little more than menu lists that flash, from time to time, on mental screen displays that other systems use. It is very much like the way the players of computer games use symbols to invoke the processes inside their complicated game machines without the slightest understanding of how they work.
And when you come to think about it, it scarcely could be otherwise! Consider what would happen if we actually could confront the trillion-wire networks in our brains. Scientists have peered at tiny fragments of those structures for many years, yet failed to comprehend what they do. Fortunately, for the purposes of everyday life, it is enough for our words or signals to evoke some useful happenings within the mind. Who cares how they work, so long as they work! Consider how you can scarcely see a hammer except as something to hit with, or see a ball except as something to throw and catch. Why do we see things, less as they are, and more in view of how they can be used? It is because our minds did not evolve to serve as instruments for science or philosophy, but to solve practical problems of nutrition, defense, procreation, and the like. We tend to think of knowledge as good in itself, but knowledge is useful only when we can exploit it to help us reach our goals.
How do you discover things about the world? Just look and see! It seems simple — but it's not. Each casual glance employs a billion brain cells to represent the present scene and to summarize its differences from records of other experiences. Your agencies formulate little bits of theories about what happens in the world and then make you do small experiments to confirm or reformulate those conjectures. It only seems simple because you're unaware of what is happening.
How do you discover things about your mind? You use a similar technique. You make up little bits of theories about how you think, then test them with tiny experiments. The trouble is that thought-experiments don't often lead to the sorts of clear, crisp findings that scientists seek. Ask yourself what happens when you try to imagine a round square — or when you try to be happy and sad at the same time. Why is it so hard to describe the results of such experiments or draw useful conclusions from them? It is because we get confused. Our thoughts about our mind-experiments are mind-experiments themselves — and therefore interfere with one another.
Thinking affects our thoughts.
People who program computers encounter similar problems when new programs malfunction because of unexpected interactions among their parts. To find out what's happening, programmers have developed special programs for debugging other programs. But just as in thought-experiments, there is a danger that the program being watched might change the one that's watching it. To prevent this, all modern computers are equipped with special interruption machinery that detects any other program's attempt to alter a debugging program; when this happens, the culprit is frozen in its tracks so that the debugging program can examine it. To do this, the interruption machinery must be supplied with a private memory bank that can store enough information to make it possible, later, to restart the frozen program as though nothing had happened.
Are brains equipped to do similar things? It was easy to build self-examination systems into computers that did only one thing at a time, but it would be much harder to do in a system that, like the brain, engages many processes at once. The problem is that if you were to freeze only one process without stopping the others, it would change the situation you're trying to examine. However, if you were to stop all those processes all at once, you couldn't experiment on how they interact.
Later, we'll see that consciousness is connected with our most immediate memories. This means that there are limits on what consciousness can tell us about itself — because it can't do perfect self-experiments. That would require keeping perfect records of what happens inside one's memory machinery. But any such machinery must get confused by self-experiments that try to find out how it works — since such experiments must change the very records they are trying to inspect! We cannot handle interruptions perfectly. This doesn't mean that consciousness cannot be understood, in principle. It only means that to study it, we'll have to use the less direct methods of science, because we cannot simply look and see.
There is one way for a mind to watch itself and still keep track of what's happening. Divide the brain into two parts, A and B. Connect the A-brain's inputs and outputs to the real world — so it can sense what happens there. But don't connect the B-brain to the outer world at all; instead, connect it so that the A-brain is the B-brain's world!
Now A can see and act upon what happens in the outside world — while B can see and influence what happens inside A. What uses could there be for such a B? Here are some A-activities that B might learn to recognize and influence.
A seems disordered and confused. Inhibit that activity. A appears to be repeating itself. Make A stop. Do something else. A does something B considers good. Make A remember this. A is occupied with too much detail. Make A take a higher-level view A is not being specific enough. Focus A on lower-level details.
This two-part arrangement could be a step toward having a more reflective mind-society. The B-brain could do experiments with the A-brain, just as the A-brain can experiment with the body or with the objects and people surrounding it. And just as A can attempt to predict and control what happens in the outer world, B can try to predict and control what A will do. For example, the B-brain could supervise how the A-brain learns, either by making changes in A directly or by influencing A's own learning processes.
Even though B may have no concept of what A's activities mean in relation to the outer world, it is still possible for B to be useful to A.
This is because a B-brain could learn to play a role somewhat like that of a counselor, psychologist, or management consultant, who can assess a client's mental strategy without having to understand all the details of that client's profession. Without having any idea of what A's goals are, B might be able to learn to tell when A is not accomplishing them but only going around in circles or wandering, confused because certain A-agents are repeating the same things over and over again. Then B might try some simple remedies, like suppressing some of those A-agents. To be sure, this could also result in B's activities becoming nuisances to A. For example, if A had the goal of adding up a long column of numbers, B might start to interfere with this because, from B's point of view, A appears to have become trapped in a repetitive loop. This could cause a person accustomed to more variety to find it difficult to concentrate on such a task and complain of being bored.
To the extent that the B-brain knows what is happening in A, the entire system could be considered to be partly self-aware. However, if we connect A and B to watch each other too closely, then anything could happen, and the entire system might become unstable. In any case,
there is no reason to stop with only two levels; we could connect a C-brain to watch the B-brain, and so on.
No supervisor can know everything that all its agents do. There's simply never time enough. Each bureaucrat sees but a fraction of what happens underneath its place in the pyramid of information flow. The best subordinates are those that work most quietly. Indeed, that's why we build administrative pyramids for jobs we don't know how to do or don't have time to do ourselves. It is also why so many of our thoughts must hide beyond our consciousness.
Good scientists never try to learn too much at once. Instead, they select particular aspects of a situation, observe carefully, and make records. Experimental records are frozen phenomena. They let us take all the time we need to make our theories. But how could we do the same thing inside the mind? We'd need some kind of memory in which to keep such records safe.
We'll see how this could work when we come to the chapters on memory. We'll conjecture that your brain contains a host of agents called K-lines, which you can use to make records of what some of your brain-agents are doing at a certain moment. Later, when you activate the same K-lines, this restores those agents to their previous states. This makes you remember part of your previous mental state, by making those parts of your mind do just what they did before. Then, the other parts of your mind will react as though the same events were happening again! Of course, such memories will always be incomplete, since nothing could have capacity enough to record every detail of its own state. (Otherwise, it would have to be larger than itself. ) Since we can't remember everything, each individual mind faces the same problem that scientists always face: they have no foolproof way to know, before the fact, what are the most important things to notice and record.
Using the mind to examine itself is like science in another way. Just as physicists cannot see the atoms they talk about, psychologists can't watch the processes they try to examine. We only know such things through their effects. But the problem is worse where the mind is concerned, since scientists can read each other's notes, but different parts of the mind can't read each other's memories.
We've now seen several reasons why we cannot simply watch our minds by sitting still and waiting till our vision clears. The only course left for us is to study the mind the way scientists do when something is too large or small to see — by building theories based on evidence. Make a guess; test it with a shrewd experiment; collect one's thoughts and guess again. When introspection seems to work, it's not because we've found a magic way to see inside ourselves. Instead, it means that we've done some well-designed experiment.
What do you think you're thinking now? You might reply, Why, just the thoughts I'm thinking now! And that makes sense, in ordinary life, where now means at this moment in time. But the meaning of now is far less clear for an agent inside a society.
It takes some time for changes in one part of a mind to affect the other parts. There's always some delay.
For example, suppose you meet your friend Jack. Your agencies for Voices and Faces may recognize Jack's voice and face, and both send messages to an agency Names, which may recall Jack's name. But Voices may also send a word-message to Quotes, a language-based agency that has a way to remember phrases Jack has said before, while Faces may also send a message to Places, an agency concerned with space, which might recall some earlier place in which Jack's face was seen.
Now suppose we could ask both Places and Quotes which had happened first, seeing Jack or hearing his voice? We'd get two different answers! Places will first detect the face — while Quotes will first detect the voice. The seeming order of events depends upon which message reached each agent first — so the seeming sequence of events differs from one agent to another. Each agent will react in its own, slightly different way — because it has been affected by a slightly different causal history, which spreads like a wave into the past.
It is simply impossible, in general, for any agent P to know for certain what another agent Q is doing at precisely the same time. The best that P can do is send a query straight to Q and hope that Q can get a truthful message back before other agents change Q's state — or change its message along the way. No portion of a mind can ever know everything that is happening at the same time in all the other agencies. Because of this, each agency must have at least a slightly different sense both of what has happened in the past — and of what is happening now. Each different agent of the mind lives in a slightly different world of time.
Our everyday ideas about the progression of mental time are wrong: they leave no room for the fact that every agent has a different causal history. To be sure, those different pasts are intermixed over longer spans of time, and every agent is eventually influenced by what has happened in the common, remote history of its society. But that's not what one means by now. The problem is with the connections between the moment-to-moment activities of largely separate agencies.
When a pin drops, you might say, I just heard a pin drop. But no one says, I hear a pin dropping. Our speaking agencies know from experience that the physical episode of pin dropping will be over before you can even start to speak. But you would say, I am in love, rather than I was just in love, because your speaking agencies know that the agencies involved with personal attachments work at a slower pace, with states that may persist for months or years. And, in between, when someone asks, What sorts of feelings have you now? we often find our half-formed answers wrong before they can be expressed, as other feelings intervene. What seems only a moment to one agency may seem like an era to another.
Our memories are only indirectly linked to physical time. We have no absolute sense of when a memorable event actually happened. At best, we can only know some temporal relations between it and certain other events. You might be able to recall that X and Y occurred on different days but be unable to determine which of those days came earlier. And many memories seem not to be linked to intervals of time at all — like knowing that four comes after three, or that I am myself.
The slower an agency operates — that is, the longer the intervals between each change of state — the more external signals can arrive inside those intervals. Does this mean that the outside world will appear to move faster to a slow agency than to a faster agency? Does life seem swift to tortoises, but tedious to hummingbirds?
Just as we walk without thinking, we think without thinking! We don't know how our muscles make us walk — nor do we know much more about the agencies that do our mental work. When you have a hard problem to solve, you think about it for a time. Then, perhaps, the answer seems to come all at once, and you say, Aha, I've got it. I'll do such and such. But if someone were to ask how you found the solution, you could rarely say more than things like the following:
If we could really sense the workings of our minds, we wouldn't act so often in accord with motives we don't suspect. We wouldn't have such varied and conflicting theories for psychology. And when we're asked how people get their good ideas, we wouldn't be reduced to metaphors about ruminating, and digesting, conceiving and giving birth to concepts — as though our thoughts were anywhere but in the head. If we could see inside our minds, we'd surely have more useful things to say.
Many people seem absolutely certain that no computer could ever be sentient, conscious, self-willed, or in any other way aware of itself. But what makes everyone so sure that they themselves possess those admirable qualities? It's true that if we're sure of anything at all, it is that I'm aware — hence I'm aware. Yet what do such convictions really mean? If self-awareness means to know what's happening inside one's mind, no realist could maintain for long that people have much insight, in the literal sense of seeing-in. Indeed, the evidence that we are self-aware — that is, that we have any special aptitude for finding out what's happening inside ourselves — is very weak indeed. It is true that certain people have a special excellence at assessing the attitudes and motivations of other persons (and, more rarely, of themselves). But this does not justify the belief that how we learn things about people, including ourselves, is fundamentally different from how we learn about other things. Most of the understandings we call insights are merely variants of our other ways to figure out what's happening.
We'll take the view that nothing can have meaning by itself, but only in relation to whatever other meanings we already know. One might complain that this has the quality of the old question, Which came first, the chicken or the egg? If each thing one knows depends on other things one knows, isn't that like castles built on air? What keeps them from all falling down, if none are tied to solid ground?
Well, first, there's nothing basically wrong with the idea of a society in which each part lends meaning to the other parts. Some sets of thoughts are much like twisted ropes or woven cloths in which each strand holds others both together and apart. Consider all the music tunes you know. Among them you can surely find two tunes of which you like each one the more because of how it's similar to or different from the other one. Besides, no human mind remains entirely afloat. Later we'll see how our conceptions of space and time can be based entirely on networks of relationships, yet can still reflect the structure of reality.
If every mind builds somewhat different things inside itself, how can any mind communicate with a different mind? In the end, surely, communication is a matter of degree but it is not always lamentable when different minds don't understand each other perfectly. For then, provided some communication remains, we can share the richness of each other's thoughts. What good would other people be if we were all identical? In any case, the situation is the same inside your mind — since even you yourself can never know precisely what you mean! How useless any thought would be if, afterward, your mind returned to the selfsame state. But that never happens, because every time we think about a certain thing, our thoughts go off in different ways.
The secret of what anything means to us depends on how we've connected it to all the other things we know. That's why it's almost always wrong to seek the real meaning of anything. A thing with just one meaning has scarcely any meaning at all.
An idea with a single sense can lead you along only one track. Then, if anything goes wrong, it just gets stuck — a thought that sits there in your mind with nowhere to go. That's why, when someone learns something by rote — that is, with no sensible connections — we say that they don't really understand. Rich meaning-networks, however, give you many different ways to go: if you can't solve a problem one way, you can try another. True, too many indiscriminate connections will turn a mind to mush. But well-connected meaning-structures let you turn ideas around in your mind, to consider alternatives and envision things from many perspectives until you find one that works. And that's what we mean by thinking!
There is no singularly real world of thought; each mind evolves its own internal universe. The worlds of thought that we appear to like the best are those where goals and actions seem to mesh in regions large enough to spend our lives in — and thus become a Buddhist, or Republican, or poet, or topologist. Some mental starting points grow into great, coherent continents. In certain parts of mathematics, science, and philosophy, a relatively few but clear ideas may lead into an endless realm of complex yet consistent new structures. Yet even in mathematics, a handful of seemingly innocent rules can lead to complications far beyond our grasp. Thus we feel we understand perfectly the rules of addition and multiplication — yet when we mix them together, we encounter problems about prime numbers that have remained unsolved for centuries.
Minds also make up pleasant worlds of practical affairs — which work because we make them work, by putting things in order there. In the physical realm, we keep our books and clothes in self-made shelves and cabinets — thus building artificial boundaries to keep our things from interacting very much. Similarly, in mental realms, we make up countless artificial schemes to force things to seem orderly, by specifying legal codes, grammar rules and traffic laws. When growing up in such a world, it all seems right and natural — and only scholars and historians recall the mass of precedents and failed experiments it took to make it work so well. These natural worlds are actually more complex than the technical worlds of philosophy. They're far too vast to comprehend — except where we impose on them the rules we make.
There is also a different and more sinister way to make the world seem orderly, in which the mind has merely found a way to simplify itself. This is what we must suspect whenever some idea seems to explain too much. Perhaps no problem was actually solved at all; instead, the mind has merely found some secondary pathway in the brain, through which one can mechanically dislodge each doubt and difference from its rightful place! This may be what happens in some of those experiences that leave a person with a sense of revelation — in a state in which no doubts remain, or with a vision of astounding clarity — yet unable to recount any details. Some accident of mental stress has temporarily suppressed the capacity to question, doubt, or probe. One remembers that no questions went unanswered but forgets that none were asked! One can acquire certainty only by amputating inquiry.
When victims of these incidents become compelled to recapture them, their lives and personalities are sometimes permanently changed. Then others, seeing the radiance in their eyes and hearing of the glory to be found, are drawn to follow them. But to offer hospitality to paradox is like leaning toward a precipice. You can find out what it is like by falling in, but you may not be able to fall out again. Once contradiction finds a home, few minds can spurn the sense-destroying force of slogans such as all is one.
Suppose that while you walked and talked, you could watch the signals that traverse your brain. Would they make any sense to you? Many people have done experiments to make such signals audible and visible, by using biofeedback devices. This often helps a person to learn to control various muscles and glands that are not usually under conscious control. But it never leads to comprehending how their hidden circuits work.
Scientists encounter similar problems when they use electronic instruments to tap into brain signals. This has led to a good deal of knowledge about how nervous systems work — but those insights and understandings never came from observation by itself. One cannot use data without having at least the beginnings of some theory or hypothesis. Even if we could directly sense all the interior details of mental life, it wouldn't tell us how to understand them. It might even make that enterprise more difficult, by overwhelming our capacity to interpret what we see. The causes and functions of what we observe are not themselves things we can observe.
Where do we get the ideas we need? Most of our concepts come from the communities in which we're raised. Even the ideas we get for ourselves come from communities — this time, the ones inside our heads. Brains don't manufacture thoughts in the direct ways that muscles exert forces or ovaries make estrogens; instead, to get a good idea, one must engage huge organizations of submachines that do a vast variety of jobs. Each human cranium contains hundreds of kinds of computers, developed over hundreds of millions of years of evolution each with a somewhat different architecture. Each specialized agency must learn to call on other specialists that can serve its purposes. Certain sections of the brain distinguish the sounds of voices from other sorts of sounds; other specialized agencies distinguish the sights of faces from other types of objects. No one knows how many different such organs lie in our brains. But it is almost certain that they all employ somewhat different types of programming and forms of representation; they share no common language code.
If a mind whose parts use different languages and modes of thought attempted to look inside itself, few of those agencies would be able to comprehend one another. It is hard enough for people who speak different human languages to communicate, and the signals used by different portions of the mind are surely even less similar. If agent P asked any question of an unrelated agent Q, how could Q sense what was asked, or P understand its reply? Most pairs of agents can't communicate at all.
If agents can't communicate, how is it that people can — in spite of having such different backgrounds, thoughts, and purposes? The answer is that we overestimate how much we actually communicate. Instead, despite those seemingly important differences, much of what we do is based on common knowledge and experience. So even though we can scarcely speak at all about what happens in our lower-level mental processes, we can exploit their common heritage. Although we can't express what we mean, we can often cite various examples to indicate how to connect structures we're sure must already exist inside the listener's mind. In short, we can often indicate which sorts of thoughts to think, even though we can't express how they operate.
The words and symbols we use to summarize our higher-level goals and plans are not the same as the signals used to control lower-level ones. So when our higher-level agencies attempt to probe into the fine details of the lower-level submachines that they exploit, they cannot understand what's happening. This must be why our language-agencies cannot express such things as how we balance on our bicycles, distinguish pictures from real things, or fetch our facts from memory. We find it particularly hard to use our language skills to talk about the parts of the mind that learned such skills as balancing, seeing, and remembering, before we started to learn to speak.
Meaning itself is relative to size and scale: it makes sense to talk about a meaning only in a system large enough to have many meanings. For smaller systems, that concept seems vacant and superfluous. For example, Builder's agents require no sense of meaning to do their work; Add merely has to turn on Get and Put. Then Get and Put do not need any subtle sense of what those turn-on signals mean — because they're wired up to do only what they're wired up to do. In general, the smaller an agency is, the harder it will be for other agencies to comprehend its tiny “language&rdquo.
The smaller two languages are, the harder it will be to translate between them. This is not because there are too many
meanings, but because there are too few. The fewer things an agent does, the less likely that what another agent does will correspond to any of those things. And if two agents have nothing in common, no translation is conceivable.
In the more familiar difficulty of translating between human languages, each word has many meanings, and the main problem is to narrow them down to something they share. But in the case of communication between unrelated agents, narrowing down cannot help if the agents have nothing in common from the start.
To “know oneself&rdquo more perfectly might seem to promise something powerful and good. But there are fallacies concealed behind that happy thought. No doubt, a mind that wants to change itself could benefit from knowing how it works. But such knowledge might as easily encourage us to wreck ourselves — if we had ways to poke our clumsy mental fingers into the tricky circuits of the mind's machinery. Could this be why our brains force us to play those games of mental hide and seek?
Just see how prone we are to risk experiments that change ourselves; how fatally we're drawn to drugs, to meditation, music, even conversation — all powerful addictions that can change our very personalities. Just see how everyone is entranced by any promise to transgress the bounds of normal pleasure and reward.
In ordinary life, our pleasure systems help us learn — and, therefore, to behave ourselves — by forcing checks and balances on us. Why, for example, do we become bored when doing the same thing over and over, even if that activity was pleasant at first? This appears to be one property of our pleasure systems; without enough variety, they tend to satiate. Every learning machine must have some such protective scheme, since otherwise it could get trapped into endlessly repeating the same activity. We are fortunate to be equipped with mechanisms that keep us from wasting too much time, and it is fortunate, too, that we find it hard to suppress such mechanisms.
If we could deliberately seize control of our pleasure systems, we could reproduce the pleasure of success without the need for any actual accomplishment. And that would be the end of everything.
What prevents such meddling? Our minds are bound by many self-constraints. For example, we find it hard to determine what's happening inside the mind. Later, when we talk about infant development, we'll see that even if our inner eyes could see what's there, we'd find it singularly hard to change the agents we might want most to change — the ones that, in our infancy, helped shape our longest-lasting self-ideals.
These agents are hard to change because of their special evolutionary origin.  Few of us would survive if, left to random chance, our most adventurous impulses could tamper freely with the basis of our personalities. Why would that be such a bad thing to do? Because an ordinary change of mind can be reversed if it leads to a bad result. But when you change your self-ideals — then nothing is left to turn you back.
Sigmund Freud theorized that each person's growth is governed by unconscious needs to please, placate, oppose, or terminate our images of parental authority. If we recognized the influence of those old images, however, we might consider them too infantile or too unworthy to tolerate and seek to replace them with something better. But then what would we substitute for them — once we divested ourselves of all those ties to instinct and society? We'd each end up as instruments of even more capricious sorts of self-invented goals.
It's mainly when our systems fail that consciousness becomes engaged. For example, we walk and talk without much sense of how we actually do those things. But a person with an injured leg may, for the first time, begin to formulate theories about how walking works ( To turn to the left, I'll have to push myself that way, ) and then perhaps consider which muscles might accomplish that goal. When we recognize that we're confused, we begin to reflect on how our minds solve problems and engage the little we know about our strategies of thought. Then we find ourselves saying things like this:
Now I must get organized. Why can't I concentrate on the important questions and not get distracted by those other nonessential details?
Paradoxically, it is smart to realize that one is confused — as opposed to being confused without knowing it. For that stimulates us to apply our intellect to altering or repairing the defective process. Yet we dislike and disparage the sense of confusion, not appreciating the quality of this recognition.
However, once your B-brains make you start to ask yourself What was I really attempting to do? you can exploit that as an opportunity to change your goals or change how you describe your situation. That way, you can escape the distress of feeling trapped because there seem to be no adequate alternatives. The conscious experience of confusion can resemble pain; perhaps this is because of how they both impel us to discover ways to escape from a predicament. The difference is that confusion is directed against a person's own failing state of mind, whereas pain reflects exterior disturbances. In either case, internal processes must be demolished and rebuilt.
Both confusion and pain have injurious effects when they lead us to abandon goals on larger scales than appropriate: The entire subject makes me feel ill. Perhaps I should abandon the whole project, occupation, or relationship. But even such dispiriting thoughts can serve as probes for finding other agencies that might be engaged for help.
Many people insist on having some definition of intelligence.
CRITIC: How can we be sure that things like plants and stones, or storms and streams, are not intelligent in ways that we have not yet conceived?
It doesn't seem a good idea to use the same word for different things, unless one has in mind important ways in which they are the same. Plants and streams don't seem very good at solving the kinds of problems we regard as needing intelligence.
CRITIC: What's so special about solving problems? And why don't you define intelligence precisely, so that we can agree on what we're discussing?
That isn't a good idea, either. An author's job is using words the ways other people do, not telling others how to use them. In the few places the word intelligence appears in this book, it merely means what people usually mean — the ability to solve hard problems.
CRITIC: Then you should define what you mean by a hard problem. We know it took a lot of human intelligence to build the pyramids — yet little coral reef animals build impressive structures on even larger scales. So don't you have to consider them intelligent? Isn't it hard to build gigantic coral reefs?
Yes, but it is only an illusion that animals can solve those problems! No individual bird discovers a way to fly. Instead, each bird exploits a solution that evolved from countless reptile years of evolution. Similarly, although a person might find it very hard to design an oriole's nest or a beaver's dam, no oriole or beaver ever figures out such things at all. Those animals don't solve such problems themselves; they only exploit procedures available within their complicated gene-built brains.
CRITIC: Then wouldn't you be forced to say that evolution itself must be intelligent, since it solved those problems of flying and building reefs and nests?
No, because people also use the word intelligence to emphasize swiftness and efficiency. Evolution's time rate is so slow that we don't see it as intelligent, even though it finally produces wonderful things we ourselves cannot yet make. Anyway, it isn't wise to treat an old, vague word like intelligence as though it must define any definite thing. Instead of trying to say what such a word means, it is better simply to try to explain how we use it.
Our minds contain processes that enable us to solve problems we consider difficult. Intelligence is our name for whichever of those processes we don't yet understand.
Some people dislike this definition because its meaning is doomed to keep changing as we learn more about psychology. But in my view that's exactly how it ought to be, because the very concept of intelligence is like a stage magician's trick. Like the concept of the unexplored regions of Africa, it disappears as soon as we discover it.
 They send us bills and checks for zero dollars and zero cents. They don't mind working in endless loops, repeating the same thing a billion times. Their total lack of common sense is another reason people think that no machine could have a mind.
It is interesting to note that some of the earliest computer programs excelled at what people consider to be expert skills. A 1956 program solved hard problems in mathematical logic, and a 1961 program solved college-level problems in calculus. Yet not till the 1970s could we construct robot programs that could see and move well enough to arrange children's building-blocks into simple towers and playhouses. Why could we make programs do grown-up things before we could make them do childish things? The answer may seem paradoxical: much of expert adult thinking is actually simpler than what is involved when ordinary children play! Why is it easier to program what experts do than what children do?
What people vaguely call common sense is actually more intricate than most of the technical expertise we admire. Neither that expert program for logic nor the one for calculus embodied more than a hundred or so facts — and most of them were rather similar to one another. Yet these were enough to solve college-level problems. In contrast, think of all the different kinds of things a child must know merely to build a house of blocks — a process that involves knowledge of shapes and colors, space and time, support and balance, and an ability to keep track of what one is doing.
To be considered an expert, one needs a large amount of knowledge of only a relatively few varieties. In contrast, an ordinary person's common sense involves a much larger variety of different types of knowledge — and this requires more complicated management systems.
There is a simple reason why it is easier to acquire specialized knowledge than commonsense knowledge. Each type of knowledge needs some form of representation and a body of skills adapted to using that style of representation. Once that investment has been made, it is relatively easy for a specialist to accumulate further knowledge, provided the additional expertise is uniform enough to suit the same style of representation. A lawyer, doctor, architect, or composer who has learned to deal with a range of cases in some particular field finds it relatively easy to acquire more knowledge of a similar character. Think how much longer it would take a single person to learn to deal competently with a few diseases and several kinds of law cases and a small variety of architectural blueprints and a few orchestral scores. The greater variety of representations would make it much harder to acquire the same amount of knowledge. For each new domain, our novice would have to learn another type of representation and new skills for using it. It would be like learning many different languages, each with its own grammar, lexicon, and idioms. When seen this way, what children do seems all the more remarkable, since so many of their actions are based upon their own inventions and discoveries.
Many people reason that machines do only what they're programmed to do — and hence can never be creative or original. The trouble is that this argument presumes what it purports to show: that you can't program a machine to be creative! In fact, it is surprisingly easy to program a computer so that it will proceed to do more different things than any programmer could imagine in advance. This is possible because of what we'll call the puzzle principle.
Puzzle Principle: We can program a computer to solve any problem by trial and error, without knowing how to solve it in advance, provided only that we have a way to recognize when the problem is solved.
By trial and error we mean programming the machine systematically to generate all possible structures within some universe of possibilities. For example, suppose you wished to have a robot machine that could build a bridge across a stream. The most efficient program for this would simply execute a specific procedure, planned out in advance, to precisely place some boards and nails. Of course, you couldn't write such a program unless you already knew how to build a bridge. But consider the alternative below, which is sometimes called the generate and test method. It consists of writing a two-part program.
Generate. The first process simply produces, one after another, every possible arrangement of the boards and nails. At first, you might expect such a program to be hard to write. But it turns out to be surprisingly easy, once you appreciate that there is no requirement for each arrangement to make any sense whatsoever! Test. The second part of the process examines each arrangement to see whether the problem has been solved. If the goal were to build a dam, the test is simply whether it holds back the stream. If the goal were to build a bridge, the test is simply whether it spans the stream.
This possibility makes us reexamine all our old ideas about intelligence and creativity, since it means that, in principle, at least, we can make machines solve any problems whose solutions we can recognize. This is rarely practical, however. Consider that there must be a thousand ways to attach two boards, a million ways to connect three of them, and a billion ways to nail four boards together. It would take inconceivably long before the puzzle principle produced a workable bridge. But it does help, philosophically, to replace our feeling of mystery about creativity by more specific and concrete questions about the efficiency of processes. The main problem with our bridge-building machine is the lack of connection between its generator and its test. Without some notion of progress toward a goal, it is hard to do better than mindless chance.
In principle, we can use the generate and test method — that is, trial and error — to solve any problem whose solution we can recognize. But in practice, it can take too long for even the most powerful computer to test enough possible solutions.  Here is one way to improve upon blind trial-and-error search.
The Progress Principle: Any process of exhaustive search can be greatly reduced if we possess some way to detect when progress has been made. Then we can trace a path toward a solution, just as a person can climb an unfamiliar hill in the dark — by feeling around, at every step, to find the direction of steepest ascent.
Many easy problems can be solved this way, but for a hard problem, it may be almost as difficult to recognize progress as to solve the problem itself. Without a larger overview, that hill climber may get stuck forever on some minor peak and never find the mountaintop. There is no foolproof way to avoid this.