Children take many years to learn effective ways to use their pronomes and isonomes. The youngest ones can neither rearrange their representations of physical scenes nor make the kinds of inference we're discussing here. To think like adults, we must develop and learn to use memory-controlling processes that manipulate several sets of pronome values at the same time. Just such a process was concealed in our simple script for Put the apple in the pail — which first appears to be merely a matter of assigning apple to the Origin and pail to the Destination. But you can't Put something until you Get it, so this must actually involve two Trans-frame operations. The first is for moving your hand to the apple, and the second is for moving the apple to the pail. During the transition, your pronomes have to change their roles since the apple's location is the Destination of the first Trans, but then becomes the Origin of the second Trans. No matter that this seems too obvious to state; some mental process has to switch that pronome's role.
By learning to manipulate our isonomes, we become able to combine mental representations into structures that resemble bridges, chains, and towers. Our language-agencies learn to express these in the form of compound sentences, by using conjunctive grammar words like and, because, or or. But language is not the only realm in which we learn to conceptualize — that is, to treat our mental processes almost as though they were object-things. After you solve a hard problem, you may find yourself representing the steps you took as if they were the parts of a physical structure. Doing this can enable you to reassemble them into other forms that achieve the same results with much more speed and much less thought.
Language lets us treat our thoughts as though they were much like ordinary things. Suppose you meet someone who is trying to solve a problem. You ask what's happening. I'm thinking, you are told. I can see that, you say, but what are you thinking about? Well, I was looking for a way to solve this problem, and I think I've just found one. We speak as though ideas resemble building-blocks that one can find and grasp!
Why do we thing-ify our thoughts? One reason is that this enables us to reapply the wonderful machines our brains contain for understanding worldly things. Another thing it does is help us organize our expeditions in the mental world, much as we find our ways through space. Consider how the strategies we use to find ideas resemble the strategies we use for finding real things: Look in the places they used to be or where they're usually found — but don't keep looking again and again in the same place. Indeed, for many centuries our memory-training arts have been dominated by two techniques. One is based on similarities of sounds, exploiting the capacities of our language-agencies to make connections between words. The other method is based on imagining the items we want to remember as placed in some familiar space, such as a road or room one knows particularly well. This way, we can apply our thing-location skills to keeping track of our ideas.
Our ability to treat ideas as though they were objects goes together with our abilities to reuse our brain-machinery over and over again. Whenever an agency becomes overburdened by a large and complicated structure, we may be able to treat that structure as a simple, single unit by thing-ifying — or, as we usually say, conceptualizing — it. Then, once we replace a larger structure by representing it with a compact symbol-sign, that overloaded agency may be able to continue its work. This way, we can build grand structures of ideas — much as we can build great towers from smaller parts.
I suspect that, as they're represented in the mind, there's little difference between a physical object and an idea. Worldly things are useful to us because they are substantial — that is, because their properties are relatively permanent. Now we don't usually think of ideas as substantial, because they don't have the usual properties of worldly things — such as color, shape, and weight. Yet good ideas must also have substantiality, albeit of a different sort:
No conception or idea could have much use unless it could remain unchanged — and stay in some kind of mental place — for long enough for us to find it when we need it. Nor could we ever achieve a goal unless it could persist for long enough. In short, no mind can work without some stable states or memories.
This may sound as though I'm speaking metaphorically, since a mental place is not exactly like a worldly place. But then, when you think of a place you know, that thought itself is not a worldly place, but only a linkage of memories and processes inside your mind. This wonderful capacity — to think about thoughts as though they were things — is also what enables us to contemplate the products of our thoughts. Without that ability to reflect, we would have no general intelligence — however large our repertoire of special-purpose skills might grow. Of course this same capacity enables us to think such empty thoughts as This statement is about itself, which is true but useless, or This statement is not about itself, which is false and useless, or This statement is false, which is downright paradoxical. Yet the benefit of being able to conceptualize is surely worth the risk that we may sometimes be nonsensical.
For virtually every change we see, we tend to seek some cause. And when we find no cause on the scene, we'll postulate that one exists, even though we might be wrong. We do this so consistently that I wouldn't be surprised to find that brains have built-in tendencies to try to represent all situations in certain special ways:
THINGS. Whatever we may see or touch, we represent the scene in terms of separate object-things. We do the same for representing processes and mental states. In languages, these object-symbols tend to correspond to nouns. DIFFERENCES. Whenever we discern a change or just compare two different things, we represent this as a difference thing. In languages, these often correspond to verbs.
CAUSES. Whenever we conceive of an action, change, or difference, we try to assign a cause to it — that is, some other person, process, or thing that we can hold to be responsible for it. In languages, causes often take the forms of things. CLAUSES. Whatever structures we conceive are dealt with like single things. In languages, this corresponds to treating an entire phrase as though it were a single word.
In English, almost every sentence form demands some sort of Actor noun — and I think this reflects the need to find a motive or a cause. Consider how we place that it in Soon it will begin to rain. We're always chopping complex situations into artificially clear-cut chunks which we perceive as separate things. Then we notice various differences and relationships among those parts and assign them to various parts of speech. We string our words into clauses and our clauses into chains, often interrupting one by inserting fragments of others inside it, yet proceeding as though there were no interruptions at all. It has been alleged that the construction of such structures is unique to the grammar-machinery of language, but I suspect that languages evolved those forms because of mechanisms deeper in the grain of how we think. For example, when we talked about visual ambiguity, we saw that our vision-systems are highly proficient at representing structures that interrupt one another. This suggests that both our visual and linguistic abilities to deal with interruptions could be based on similar methods with which we manage what is represented in our short-term memories.
In any case, our brains appear to make us seek to represent dependencies. Whatever happens, where or when, we're prone to wonder who or what's responsible. This leads us to discover explanations that we might not otherwise imagine, and that helps us predict and control not only what happens in the world, but also what happens in our minds. But what if those same tendencies should lead us to imagine things and causes that do not exist? Then we'll invent false gods and superstitions and see their hand in every chance coincidence. Indeed, perhaps that strange word I — as used in I just had a good idea — reflects the selfsame tendency. If you're compelled to find some cause that causes everything you do — why, then, that something needs a name. You call it me.
What enables us to tolerate an interruption and then return to our previous thoughts? This must engage the agents that control our short-term memories. It is important also to recognize that many interruptions come not only from outside, but also from inside the mind. For example, all but the simplest discourses make interruptions in the trains of thought they start. Consider this sentence:
The thief who took the moon moved it to Paris.
We can regard this as expressing one thought that is interrupted by another. The principal intention of the speaker is to express this Trans-frame:
The thief moved the moon (from ?) to Paris. Actor Trans Object Origin Destination The speaker, realizing that the listener may not know who the thief was, interrupts the main sentence with a relative clause — who took the moon — to further describe that Actor thief. As it happens, this interrupting clause also has the form of a Trans-frame — so now the language-agency must deal with two such frames at once.
Who took the moon (from ?) (to?) Actor Trans Object Origin Destination English tends to use certain wh words, like which and who, to interrupt a listener's language-agency and cause its short-term memories to temporarily store away some of their present pronome assignments. This provides the language-agency with more capacity to understand the interrupting phrase. In the case of the moon sentence, the word who instructs the listener to prepare to elaborate the description of the Actor thief. Once this is done, the language-agency can re-member its previous state in the process of understanding the main sentence. We can often tell when to use an interruption process even though the initial wh word is missing; however, this doesn't always work so well:
The cotton clothing is made of is grown in the south.
This sentence is confusing because the reader tends to treat the word cotton in cotton clothing as an adjective that modifies clothing, when the writer meant it as a noun. The same sentence is easier to understand when set in a larger context:
Where do people grow the cotton that is used to make clothing? --- The cotton clothing is made of is grown in the south.
The first sentence activates the noun sense of cotton and asks a question about that subject. Now a question is really a sort of command: it makes the reader focus attention on a certain subject. Here, it prepares the reader to add more structure to the representation of the cotton noun, so there is less need for an explicit interruption signal. Still, it is very curious how rarely we bother to use any signal at all for marking the end of an interrupting phrase. We never say a word that means un-who. Evidently, we're usually ready to assume that the interrupting phrase is complete.
We sometimes think of words like who or it as pronouns — that is, signals that represent or substitute for other nouns or phrases. But as we've seen, pronouns don't refer to words so much as to partial states that are active in the listener's mind. In order to refer to such an activity, the listener must assign it to some short-term memory- unit — that is, to some pronome. However, communication will fail unless the listener can correctly guess which pronome the speaker wishes to assign to that activity. This can be a problem when there is more than one available choice. For example, consider the pronoun it in the following sentence:
The thief who took the moon moved it to Paris.
How does the listener understand that it must mean the moon? English grammar constrains the choice: it cannot be assigned to the thief because it cannot refer to a person at all. (We apologize to all the small children we have called it in this book.) But grammar alone can't determine the choice, since it could also mean the sun, as it does in this little dialogue:
Good grief; what's happened to the sun? Oh, that! The thief who took the moon moved it to Paris.
The way it works is not so much grammatical as psychological. The expression moved it causes the listener's language-agencies to seek a pronome that represents something that could have been moved. This could be either the sun or the moon. But the previous question, What happened to the sun? has already prepared the listener to expect to hear about an action whose Object pronome represents the sun, just as our earlier question about cotton made the listener anticipate an answer concerning that topic. Furthermore, the new phrase, The thief . . . moved it, fulfills this expectation by activating a Trans- frame whose Actor and Action pronomes already have assignments; this frame requires only an Object to be complete. So the word it is perfectly suited to fill the role of sun in that unassigned Object slot.
What does expectation mean? At each point in a dialogue, both parties are already involved with various concerns and desires. These establish contexts in which each new word, description, or representation, however ambiguous, gets merged with whichever short-term memory best matches it. Why do we make such assignments so quickly, instead of waiting until all the ambiguities are resolved? That is a practical matter. Our language-agencies must dispose of each phrase as soon as possible, so that they can apply their full capacities to deal with what comes afterward. If something in the conversation does not match anything that came before, the listener must activate a new memory-unit. This tends to slow the process down, because it consumes our limited supplies of short-term memory and makes subsequent matching more difficult. If the listener cannot make suitable assignments quickly enough, the conversation will seem incoherent and communication will break down.
Eloquent speakers avoid this by designing each new expression to be easily attached to structures already active in the listener; otherwise the listener is entitled to complain that the language isn't clear. A speaker can also indicate which subjects have not been mentioned yet,
to spare the listener from struggling to make a nonexistent match; we use expressions like by the way to tell the listener not to attach what comes next to any presently active pronome. To do such things, the speaker must anticipate some of what is happening inside the listener's mind. The next section describes a way of doing this — by using the speaker's own mind as a model and assuming that the listener will be similar.
How easily people can communicate. We listen and speak without the slightest sense of what's involved! One of us expresses an idea, the other understands it, and neither thinks anything complicated has happened; it seems as natural to talk as it is to walk. Yet both simplicities are illusions. To walk, you must engage a vast array of agencies to move your body down the street. To talk, you must engage a vast array of agencies to build new structures in another person's mind. But how do you know just what to say to affect that other person's agencies?
Let's suppose that Mary wants to tell Jack something. This means there is a certain structure p somewhere inside the network of Mary's agencies — and that Mary's language-agency must construct a similar structure inside Jack's mind. To do this, Mary will need to speak words that will activate appropriate activities inside Jack's agencies, then correctly link them together. How can she do that? Here is what we'll call the re-duplication theory of how we formulate what we say:
Mary proceeds, step by step, to construct a new version of p — call it q — inside her own mind. In doing this, she will apply various memory-control operations to activate certain isonomes and polynemes. As Mary performs each internal operation, her speech-agency selects certain corresponding verbal expressions — and these cause similar operations to occur inside Jack. As a result, Jack builds a structure similar to q. To be able to do that, Mary must have learned at least one expressive technique that corresponds to each frequently used mental operation. And Jack must have learned to recognize those expressive techniques — we'll call them grammar-tactics — and to use them to activate some corresponding isonomes and polynemes.
To build her new version of p, Mary could employ a goal-achieving scheme: she keeps comparing p with the latest version of q, and whenever she senses a significant difference, she applies some operation to q that removes or reduces the difference. For example, if Mary notices that p has an Origin pronome where q lacks one, her memory-control system will focus on p's Origin. In this case, if p itself is a motion frame, the usual speech-tactic is to use the word from. Next she must describe the substructure attached to p's Origin pronome. If this were a simple polyneme like Boston, Mary's speech-agency could simply pronounce the corresponding word. But if that pronome is assigned to some more complicated structure, such as an entire frame, Mary's language-agency must interrupt itself to copy that. This is expressed, as we have seen, by using words like who or which. In any case, Mary continues this difference- duplication process until she senses no significant discrepancies between q and p. Of course, what Mary finds significant depends on what she wants to say.
This re-duplication theory of speech describes only the first stages of how we use language. In later stages, the mental operations we use to construct q are not always immediately applied to pronouncing words. Instead, we learn techniques for storing sequences of grammar-tactics temporarily; this makes it possible to modify and rearrange our words and sentences before we say them. Learning these arts takes a long time: most children need a decade or more to complete their language-systems and many keep learning, throughout their lives, to sense new sorts of discrepancies and discover ways to express them.
There is a wonderful capacity that comes along with the ability to express ideas. Whatever we may want to say, we probably won't say exactly that. But in exchange, there is a chance of saying something else that is both good and new! After all, the thing we want to say — the structure p we're trying to describe — is not always a definite, fixed structure that our language-agents can easily read and copy. If p exists at all, it's likely to be a rapidly changing network involving several agencies. If so, then the language-agency may only be able to make guesses and hypotheses about p and try to confirm or refute them by performing experiments. Even if p were well defined in the first place, this very process is liable to change it, so that the final version q won't be the same as the original structure p. Sometimes we call this process thinking in words.
In other words, whether or not what you meant to say actually existed before you spoke, your language-agencies are likely either to reformulate what did exist or create something new and different from anything you had before. Whenever you try to express with words any complicated mental state, you're forced to oversimplify — and that can cause both loss and gain. On the losing side, no word description of a mental state can ever be complete; some nuances are always lost. But in exchange, when you're forced to separate the essences from accidents, you gain the opportunity to make reformulations. For example, when stuck on a problem, you may say to yourself things like Now, let's see — just what was I really trying to accomplish? Then, since your language-agency knows so little about the actual state of those other agencies it must answer such questions by making theories about them, and these may well leave you in a state that is simpler, clearer, and better suited to solving your problem.
When we try to explain what we think we know, we're likely to end up with something new. All teachers know how often we understand something for the first time only after trying to explain it to someone else. Our abilities to make language descriptions can engage all our other abilities to think and to solve problems. If speaking involves thinking, then one must ask, How much of ordinary thought involves the use of words? Surely many of our most effective thinking methods scarcely engage our language-agencies at all. Perhaps we turn to words only when other methods fail. But then the use of language can open entirely new worlds of thought. This is because once we can represent things in terms of strings of words, it becomes possible to use them in a boundless variety of ways to change and rearrange what happens in our other agencies. Of course, we never realize we're doing this; instead we refer to such activities by names like paraphrase or change of emphasis, as though we weren't changing what we're trying to describe. The crucial thing is that during the moments in which those word-strings are detached from their meanings, they are no longer subject to the constraints and limitations of other agencies, and the language-systems can do what they want with them. Then we can transmit, from one person's brain to another, the strings of words our grammar-tactics produce, and every individual can gain access to the most successful formulations that others can articulate. This is what we call culture — the conceptual treasures our communities accumulate through history.
Much of ordinary thought is based on recognizing differences. This is because it is generally useless to do anything that has no discernible effect. To ask if something is significant is virtually to ask, What difference does it make? Indeed, whenever we talk about cause and effect we're referring to imaginary links that connect the differences we sense. What, indeed, are goals themselves, but ways in which we represent the kinds of changes we might like to make?
It is interesting how many familiar mental activities can be represented in terms of the differences between situations. Suppose you have in mind two situations A and Z, and D is your description of the difference between them. Suppose also that you are thinking of applying a certain procedure P to the first situation, A. There are several kinds of thinking you might do.
PREDICTING. To the extent that you can predict how various P's will affect A, you can avoid the expense and risk of actually performing those actions.
EXPECTING. If you expect P to produce Z but it actually produces Y, then you can try to explain what went wrong in terms of difference between Y and Z. EXPLAINING. If actions like P usually lead to differences of type D, then when you observe such a D, you can suspect that it was caused by something like P.
WANTING. If you are in situation A, but wish to be in a situation more like Z, it may help to remember ways to remove or reduce differences like D. ESCAPING, ATTACKING, and DEFENDING. If P causes a disturbing type of difference D, we can try to improve matters by finding some action that counteracts or opposes P. ABSTRACTING. In many forms of thought, the differences we notice between objects at each level become the objects of our higher-level thoughts.
Not only are differences important by themselves; more often than we realize, we think about differences between differences. For example, the height of a physical object is really a difference between the locations of its top and its bottom. And this means that the higher-level agents in our Societies-of-More must actually deal with differences between differences. For example, the agent Taller has to react to the difference between two heights — but as we've just seen, a height is already a difference between two locations!
The ability to consider differences between differences is important because it lies at the heart of our abilities to solve new problems. This is because these second-order-differences are what we use to remind ourselves of other problems we already know how to solve. Sometimes this is called reasoning by analogy and is considered to be an exotic or unusual way to solve problems. But in my view, it's our most ordinary way of doing things.
It is important for us to be able to notice differences. But this seemingly innocent requirement poses a problem whose importance has never been recognized in psychology. To see the difficulty, let's return to the subject of mental rearrangements. Let's first assume that the problem is to compare two room-arrangement descriptions represented in two different agencies: agency A represents a room that contains a couch and a chair; agency Z represents the same room, but with the couch and chair exchanged.
Now if both agencies are to represent furniture arrangements in ways that some third agency D can compare, then the difference- detecting agency D must receive two sets of inputs that match almost perfectly. Otherwise, every other, irrelevant difference between the outputs of A and Z would appear to D to be differences in those rooms — and D would perceive so many spurious differences that the real ones would be indiscernible!
The Duplication Problem. The states of two different agencies cannot be compared unless those agencies themselves are virtually identical.
But this is only the tip of the iceberg, for it is not enough that the descriptions to be compared emerge from two almost identical agencies. Those agencies must, in turn, receive inputs of near identical character. And for that to come about, each of their subagencies must also fulfill that same constraint. The only way to meet all these conditions is for both agencies — and all the subagencies upon which they depend — to be identical. Unless we find another way, we'll need an endless host of duplicated brains!
This duplication problem comes up all the time. What happens when you hear that Mary bought John's house? Must you have separate agencies to keep both Mary and John in your mind at once? Even that would not suffice, for unless both person-representing agencies had similar connections to all your other agencies, those two representations of persons would not have similar implications. The same kind of problem must arise when you compare your present situation with some recollection or experience — that is, when you compare how you react to the two different partial states of mind. But to compare those two reactions, what kind of simultaneous machinery would be needed to maintain both momentary personalities? How could a single mind hold room for two — one person old, the other new?
Fortunately, there is a way to get around the duplication problem entirely. Let's take a cue from how a perfume makes a strong impression first, but then appears to fade away, or how, when you put your hand in water that is very hot or very cold, the sensation is intense at first — but will soon disappear almost entirely. As we say, we get used to these sensations. Why? Because our senses react mainly to how things change in time. This is true even for the sensors in our eyes — though normally we're unaware of it because our eyes are always moving imperceptibly. Most of the sensory agents that inform our brains about the world are sensitive only to various sorts of time changes — and that, surely, is also true of most of the agents inside the brain.
Any agent that is sensitive to changes in time can also be used to detect differences. For whenever we expose such an agent, first to a situation A and then to a situation B, any output from
that agent will signify some difference between A and B.
This suggests a way to solve the duplication problem. Since most agents can be made to serve as difference-agents, we can compare two descriptions simply by presenting them to the same agency at different times. This is easily done if that agency is equipped with a pair of high-speed, temporary K-line memories. Then we need only load the two descriptions into those memories and compare them by activating first one and then the other.
Store the first description in pronome p. Store the second description in pronome q. Activate p and q in rapid succession. Then any changes in the agents' outputs represent differences between A and B!
We can use this trick to implement the scheme we described for escaping from a topless-arch. Suppose that p describes the present situation and q describes a box that permits no escape. Each Move agent is designed to detect the appearance of a wall. If we simply blink
from the present situation to the box frame, one of these agents will announce the appearance of any box wall that was not already apparent in the present situation. Thus, automatically, this scheme will find all the directions that are not closed off. If the outputs of the Move agents were connected to cause you to move in the corresponding direction, this agency would lead you to escape!
The method of time blinking can also be used to simplify our difference-engine scheme for composing verbal expressions, since now the speaker can maintain both p and q inside the selfsame agency. If not for this, each speaker would need what would amount to a duplicate society of mind in order to simulate the listener's state. Although the method of time blinking is powerful and efficient, it has some limitations; for example, it cannot directly recognize relations among more than two things at a time. I suspect that people share this limitation, too — and this may be why we have relatively few language-forms, like between and middle, for expressing three-way comparisons and relationships.
Let's return just one time more to all the things that More can mean. Each usage has a different sense — more powerful; more meaningful — and each such meaning must be learned. In other words, each use of More involves a connection with an agent for some adjective. But More must also engage some systematic use of isonomes, since all the different meanings share a certain common character.
When we hear the word more, we become disposed to make comparisons.
This suggests that More engages both an accumulation of different meanings and also some systematic, isonomelike effect. Indeed, More could exploit our time-blinking mechanism, which already uses isonomes to make comparisons. To do that, More would have to activate a memory-control process that blinks whichever pronomes have been assigned to the things to be compared. Then their differences would be computed automatically.
More needs two additional ingredients. We'd never ask, all by itself, Which one is more, an apple or a pear? — because our general- purpose comparison script would generate difference-descriptions in too many agencies. We also need to know which kind of difference is of concern at the moment. So we scarcely ever say more by itself but usually attach some modifier — more red, say, or more expensive. Of course, if our focus of concern is already clear from the context — for example, if it is clear that we want to know whether apples or pears are more expensive — then we need not express this explicitly.
Finally, it is one thing to find a difference, but another to know whether to call it more or less. It may seem self-evident that Taller corresponds to more, and Thinner corresponds to less — yet that is something we once had to learn. This is the other ingredient of More: we need another polyneme to say which sorts of differences should be considered positive. In English we sometimes encode such preferences as choices between pairs of adjectives like large and
small, but we do not have pairs of words for concepts such as triangular or red, presumably because we do not think of them as having natural opposites. Instead, we can use word-pairs like more red and less triangular. We can even modify the words themselves; we often say redder or rounder — but for some reason we never say triangularer.
How does one answer a question like Which is bigger, a large mouse or a small elephant? We can't compare two descriptions until we engage enough knowledge to construct suitable representations of them. One way to compare mouse and elephant would be to envision another entity of intermediate size. A suitcase would be suitable for this, since it could hold the largest mouse but not the smallest elephant. How do you find such standards for comparison? That can take considerable time, during which you have to search your memories for structures that can serve as links for longer chains of comparisons. As life proceeds, each person's concept of More grows more and more elaborate. When it comes to notions like more similar, more interesting, or more difficult, there seems no limit to the complexity of what a word like more can represent.
It is not unusual for an adult to learn a second language with nearly perfect mastery of grammar and vocabulary. But once past adolescence, most people never manage to imitate the new language's pronunciation perfectly, no matter how long and hard they work at it. In other words, they speak with foreign accents. Even when another speaker tries to help with Say it like this, not that, the learner is unable to learn what changes to make. Most people who change countries in their later teens never learn to speak the way the natives do.
Why do adults find it so hard to learn how to pronounce new word sounds? Some like to say that this reflects a general decline in the learning capacities of older people, but that appears to be a myth. Instead, I suspect this particular disability is caused, more or less directly, by a genetically programmed mechanism that disables our ability to learn to make new connections in or between the agents we use to represent speech sounds. There is evidence that our brains use different machinery for recognizing language sounds than for recognizing other sorts of sounds, particularly for the little speech- sound units that language scientists call phonemes. Most human languages use less than a hundred phonemes.
Why should we be able to learn many different speech sounds before the age of puberty but find it so much harder to learn new ones afterward? I suspect that this link to puberty is no coincidence. Instead, one or more of the genetically controlled mechanisms that brings on sexual maturity also acts to reduce the capacities of these particular agencies to learn to recognize and make new sounds! But why did this peculiar disability evolve? What evolutionary survival advantage would favor individuals whose genes reduce, after that age, this particular ability to learn? Consider this hypothesis:
The onset of the childbearing age is the biological moment when a person's social role changes from learner to teacher. The evolutionary purpose of suppressing speech-sound learning may simply serve to prevent the parent from learning
the child's speech — thus making the child learn the adult's speech instead!
Wouldn't parents want to teach the children their language anyway? Not necessarily. In the short run, a parent is usually more concerned with communication than with instruction. Accordingly, if we found it easier to imitate our children's sounds, that's what we'd do. But if parents were inclined and able to learn to speak the ways their children do, those children would lose both incentive and opportunity to learn to speak like adults, and — if every child acquired a different set of language sounds — no common, public language would ever have evolved in the first place! If this is right, puberty-linked genes for suppressing speech-sound learning may have formed fairly early in the evolution of human languages. No one knows when that occurred, but if biologists could find and date the genes for this, we could obtain a clue about the time of language's unknown origin, perhaps within the last half million years. sh\
When we enter a room, we seem to see the entire scene at a glance. But, really, it takes time to see — to apprehend all the details and see if they confirm our expectations and beliefs. Our first impressions often have to be revised. Still, one wonders how so many kinds of visual cues can lead so quickly to consistent views. What could explain the blinding speed of sight?
The secret is that sight is intertwined with memory. When face to face with someone you've just met, you seem to react almost instantly — but not as much to what you see as to what that sight reminds you of. The moment you sense the presence of a person, a whole world of assumptions is aroused that are usually true about people in general. At the same time, certain superficial cues remind you of particular people you've already met. Unconsciously, then, you will assume that this stranger must also resemble them, not only in appearance but in other traits as well. No amount of self-discipline can keep those superficial similarities from provoking assumptions that may then affect your judgments and decisions. When we disapprove of this, we complain about stereotypes — and when we sympathize with it, we speak of sensitivity and empathy.
It's much the same with language, too. If someone said, It's raining frogs, your mind would swiftly fill with thoughts about the origins of those frogs, about what happens to them when they hit the ground, about what could have caused that peculiar plague, and about whether or not the announcer had gone mad. Yet the stimulus for all of this is just three words. How do our minds conceive such complex scenes from such sparse cues? The additional details must come from memories and reasoning.
Most older theories in psychology could not account for how a mind could do such things — because, I think, those theories were based on ideas about chunks of memory that were either much too small or much too large. Some of those theories tried to explain appearances only in terms of low-level cues, while other theories tried to deal with entire scenes at once. None of those theories ever got very far. The next few sections describe what seems to be a useful compromise; at least it has led to some better results in some projects concerned with Artificial Intelligence. Our idea is that each perceptual experience activates some structures that we'll call frames — structures we've acquired in the course of previous experience. We all remember millions of frames, each representing some stereotyped situation like meeting a certain kind of person, being in a certain kind of room, or attending a certain kind of party.
A frame is a sort of skeleton, somewhat like an application form with many blanks or slots to be filled. We'll call these blanks its terminals; we use them as connection points to which we can attach other kinds of information. For example, a frame that represents a chair might have some terminals to represent a seat, a back, and legs, while a frame to represent a person would have some terminals for a body and head and arms and legs. To represent a particular chair or person, we simply fill in the terminals of the corresponding frame with structures that represent, in more detail, particular features of the back, seat, and legs of that particular person or chair. As we'll see, virtually any kind of agent can be attached to a frame-terminal. It can be a K-line, polyneme, isonome, memory-control script, or, best of all, another frame.
In principle, we could use frames without attaching their terminals to anything. Normally, though, the terminals come with other agents already attached — and these are what we called default assignments when we first talked about level-bands. If one of your person-frames is active, and you actually see some arms and legs, their descriptions will be assigned to the proper terminals. However, if certain parts cannot be seen, perhaps because they're out of view, the missing information will be filled in by default. We use default assumptions all the time: that's how, when you see someone wearing shoes, you know that there are feet in them. From where do those assumptions come? I'll argue that
Default assumptions fill our frames to represent what's typical.
 You do this not only with language, but with vision, too. For example, when someone is seated across the table from you, you may be unable to see any part of that person's chair. Still, this situation will probably activate a sitting-frame. But a sitting-frame will surely have a terminal for what to sit upon, and that will be assigned, by default, to some stereotypical chair. Then, though there is no chair in sight, a chair-frame will be supplied by default. Default assignments are of huge significance because they help us represent our previous experience. We use them for reasoning, recognizing, generalizing, predicting what may happen next, and knowing what we ought to try when expectations aren't met. Our frames affect our every thought and everything we do.
Frames are drawn from past experience and rarely fit new situations perfectly. We therefore have to learn how to adapt our frames to each particular experience. What if a given situation closely matches several different frames at once? Some such conflicts could be resolved by the locking-in negotiations we described earlier; then only the frames that manage to suppress their competitors can influence one's other agencies. But the other frames could lurk offstage, awaiting opportunities to intervene.
In order to be more concrete, let's make a little theory of how a frame might actually work. Consider, for example, a Trans-frame that is filled in to represent this sentence:
Jack drove from Boston to New York on the turnpike with Mary.
Whenever this particular frame is active, if you wonder about the Destination of that trip, you'll almost instantly think of New York. This suggests that the polyneme for New York must be aroused by the coincidence of two mental events, namely, the arousal of this particular travel-frame and the arousal of the pronome for Destination. Now how could a brain-agent recognize such a coincidence? Simple: we need only assume that the polyneme for New York is attached to an AND-agent with two inputs; one of them represents the arousal of the travel-frame itself, and the other represents the arousal of the Destination pronome. Accordingly, each terminal of our frame could simply be an AND-agent with two inputs.
According to this simple scheme, a frame could consist of little more than a collection of AND-agents, one for each of the frame's pronome terminals! Then the entire frame for the New York trip would look like this:
When a frame-agent is activated — either by seeing, hearing, or imagining something — this supplies each of those AND-agents with one of these two inputs. The second input is provided by some pronome which can thereby activate whichever agent or frame is presently assigned to that terminal. If several pronomes are active at the same time, all the corresponding agents will be activated, too. When the frame above is active, the pronome for Origin will activate the K-line for Boston, and the pronome for Vehicle will activate the K-line for car.
How could such a frame be made to learn which polynemes should fill its terminals? We could begin with each terminal initially connected to a virgin K-line; then each terminal will represent whatever the corresponding K-line learns. Notice that to build frames this way, we need only connect AND-agents to K-lines that can in turn be constructed from little more than simple AND-type agents. One of the great surprises of modern computer science was the discovery that so much can be done with so few kinds of ingredients.
When someone says, John threw a ball, you probably unconsciously assume a certain set of features and qualities of the ball, like color, size, and weight. These are your assumptions by default, the kind we talked about when we first introduced the idea of level-bands. Your assumptions about that ball might be derived from some ball you owned long ago — or, possibly, your newest one. It is our theory that such optional details are usually attached too weakly to hold against the sharp insistence of reality, so that other stimuli will find them easy to detach or otherwise adapt. This is why default assumptions make weak images, and why we aren't too amazed when they turn out wrong. It is no surprise that frames share so many properties of K-lines, since the terminals of frames themselves will lie in level-bands near the K-lines whose fringes represent our expectations and default assumptions.
But why use default assumptions at all, instead of simply seeing what's really there? Because unless we make assumptions, the world would simply make no sense. It would be as useless to perceive how things actually look as it would be to watch the random dots on untuned television screens. What really matters is being able to see what things look like. This is why our brains need special machinery for representing what we see in terms of distinct objects. The very idea of an object embodies making many assumptions that go without saying — for example, that it has substance and boundaries, that it existed before we saw it, and that it will remain afterward — in short, that it will act like other typical objects. Thus, though we never see every side of an object at once, we always assume that its unseen sides exist. I suspect that the larger part of what we know — or think we know — is represented by default assumptions, because there is so little that we know with perfect certainty.
We use default assumptions in personal relations, too. Why do so many people give such credence to astrology, to classify friends by the months of their births? Perhaps it seems a forward step, to class all persons into just twelve types — to those who once supposed that there were less. And how does the writer's craft evoke such lifelike characters? It's ridiculous to think that people could be well portrayed in so few words. Instead, our story writers use phrases that activate great networks of assumptions that already lie in the minds of their readers. It takes great skill to create those illusions — to activate unknown processes in unknown readers' minds and to shape them to one's purposes. Indeed, in doing so, a writer can make things clearer than reality. For although words are merely catalysts for starting mental processes, so, too, are real things: we can't sense what they really are, only what they remind us of. As Proust went on to say:
Each reader reads only what is already inside himself. A book is only a sort of optical instrument which the writer offers to let the reader discover in himself what he would not have found without the aid of the book.
Even when you were very young, if someone had told you that most Snarks are green — and, also, that every Boojum is a Snark — you would have been able to conclude that most Boojums are green. What could have led you to that conclusion? Presumably, you answer questions about the properties of Boojums by attaching your polyneme for Snark to whichever of your memory-units currently represents a Boojum. Accordingly, you assume that the color property of a Boojum is green by using your usual way of recalling the properties of things you know — activating their polynemes to set your various agencies into the corresponding states. In other words, we do this kind of reasoning by manipulating our memories to replace particular things by typical things. I mention all this because it is often assumed that adults are better than children are at what is often called abstract or logical reasoning. This idea is unfair both to adult and child because logical thinking is so much simpler — and less effective — than common-sense thinking. Actually, what appears to be a matter of logic is usually not logical at all and frequently turns out to be wrong. In the case above, you would have been wrong because Boojums are albino Snarks.
The situation is different when you happen to know more about a particular example. For example, suppose you first had learned that penguins cannot fly and then learned that penguins are a kind of bird. When you discover that, should you replace all of your penguin properties with those of your generic bird? Clearly not, since then you'd lose your hard-earned penguin facts. To deal with this effectively, children must develop complex skills, not merely to replace one representation with another, but to compare two representations and then move around inside them, making different changes at different levels. These intricate skills involve the use of isonomes that control the level-band of the activities inside our agencies.
In any case, to reason well, our memory-control agencies must learn to move our memories around as though those memories were building-blocks. Conceivably, those agencies have to learn such skills before we can learn to build with blocks in the outside world of object-things. Unfortunately, we know very little about how such processes work. Indeed, we're virtually unaware that they even exist, because these kinds of commonsense inferences and assumptions come to mind without the slightest conscious effort or activity. Perhaps this unawareness is a consequence of the speed with which those skills employ the very same short-term memory-units that might otherwise be used to record those agents' own recent activities.
When you think about an object in a certain place, many different processes go on inside your mind. Some of your agencies know the visual direction in which that object lies, others can direct your hand to reach toward it, and yet other agencies anticipate how it would feel if it touched your skin. It is one thing to know that a block has flat sides and right angles, another to be able to recognize a block by sight, and yet another to be able to shape your hand to grasp that shape or recognize it from feeling it in your grasp. How do so many different agencies communicate about places and shapes?
No one yet knows how shapes and places are represented in the brain. The agencies that do such things have been evolving since animals first began to move. Some of those agencies must be involved with postures of the arm and hand, others must represent what we discover from the images inside our eyes, and yet others must represent the relations between our bodies and the objects that surround us.
How can we use so many different kinds of information at once? In the following sections I'll propose a new hypothesis to deal with this: that many agencies inside our brains use frames whose terminals are controlled by interaction-square arrays. Only now we'll use those square-arrays not to represent the interactions of different causes, but to describe the relations between closely related locations. For example, thinking of the appearance of a certain place or object would involve arousing a squarelike family of frames, each of which in turn represents a detailed view of the corresponding portion of that scene. If we actually use such processes, this could explain some psychological phenomena.
If you were walking through a circular tube, you could scarcely keep from thinking in terms of bottom and top and sides — however vaguely their boundaries are defined. Without a way to represent the scene in terms of familiar parts, you'd have no well-established thinking skills to apply to it.
The diagram is meant to suggest that we represent directions and places by attaching them to a special set of pronome-like agents that we shall call direction-nemes. Later we'll see how these might be involved in surprisingly many realms of thought.
Whenever we see a thing so utterly new that it resembles nothing we've ever seen before, this means that none of our prelearned frames will fit it very well. But this rarely happens to adults. For example, we have each accumulated enough room-frames to represent most rooms we're likely to see, such as kitchens, bedrooms, offices, factories, and concert halls; one of these will usually match whichever place we happen to be. Besides, we can almost always use a less specific frame that fits most any room at all — a frame with terminals that correspond to nothing more than ceiling, floor, and walls. Then each of those six surfaces could be represented, in turn, by a subframe that has terminals for several vaguely defined regions. To be specific, let's employ our direction-neme idea and divide each surface — for ceiling, floor, and each of the walls — into zones that correspond to the nine regions of an interaction-square. A typical wall might then be represented in this fashion:
In spite of its simplicity, we can represent quite a lot of information with this scheme. It provides enough structure to recall later that there was a window toward the left, some shelves high to the right side of the wall, and a table to the right. If this does not seem precise enough, the fact is that we usually don't remember things so exactly, except when they attract special attention; normally it would be enough to know only roughly where that television was, and we could assume by default that it was supported by the tabletop. It takes surprisingly few such observations to enable us to tell, later, whether much has been changed.
Given more time, one could keep noticing more details and include them by attaching additional subframes. This would overcome the limitations of starting with so few terminals. For example, one might notice that the window is closer to the shelf than to the television set, and closer to the ceiling line than to either the shelf or the television set. And if the outline of the desk and television set reminds you of a goatlike animal, your representation can include that fact.
Suppose you had first assumed this to be a living room but later recognized the table to be a kitchen table? Must you undo all the work you've done, to activate a different kitchen-frame and start all over again? No, because later we'll see a convenient way to switch over to another frame — while still retaining what was learned so far. The trick will be to make all our frames for different rooms share the same terminals, so that when we interchange those frames, the information stored in them remains.
Now that we've seen how picture-frames could represent memories of spatial arrangements, let's ask how we actually build such frames. We'll use the same technique that we used to build Trans-frames, except for one small change. To make a picture-frame, we'll simply replace the pronomes of our Trans-frame scheme by a set of nine direction-nemes! The diagram below also includes an agent to serve for turning on the frame itself.
To apply the picture-frames idea to how our vision-systems work, imagine that you're looking at some real-world scene. Your eyes move in various directions, controlled in some way by direction-nemes. Now suppose that every time you move your eyes, the same direction-nemes also activate the K-lines attached to the corresponding terminals of a certain vision-frame. Suppose, also, that those K-lines are ready to form new memories. Then each time you look in a different direction, your vision-system will describe what you see — and the corresponding K-line will record what you see when you look in that direction!
Now suppose that the same frame is activated at some later date — but this time by means of memory and not from looking at some scene. Then, as any of your agencies conceives of looking in a certain direction, the thought itself will involve the activation of the corresponding direction-neme; then, before you have a chance to think of anything else, the corresponding K-line will be aroused. This creates a most remarkable effect:
Whichever way your mind's eye looks, you'll seem to see the corresponding aspect of the scene. You will experience an almost perfect simulus of being there!
How real could such a recollection seem? In principle, it could even seem as real as vision itself — since it could make you seem to sense not only how an object looks, but also how it tastes and feels. Shortly we'll see how this could yield not merely the sense of seeing a scene, but also the sense of being able to move around inside it.
How do frames become activated? This amounts to asking how we recognize familiar situations or things. There is no limit to how complicated such a question can become, since there are no natural boundary lines between recognizing, remembering, and all the rest of how we think. For questions like this, with no place to start, we have to construct some boundary lines from our own imagination.
We'll simply assume that every frame is activated by some set of recognizers. We can regard a recognizer as a type of agent that, in a sense, is the opposite of a K-line — since instead of arousing a certain state of mind, it has to recognize when a certain state of mind occurs. Accordingly, the recognizers of a frame are very much like the terminals of a frame, except that the connections to the terminals are reversed.
This suggests that not only frames but agencies in general might be organized in the form of agents sandwiched between recognizers and memorizers.
This sketch of how our agencies are organized is oversimplified. Each agent, be it a frame, a K-line, or whatever, must have some machinery for learning when it should become active — and that may require more than simply recognizing the presence of certain features. For example, to recognize an object as a car, it isn't enough that it include some assortment of parts like body, wheels, and license plate; the frame must also recognize that those parts are in suitable relationships — that the wheels be properly attached to the body, for example. Workers in the field of Artificial Intelligence have experimented with a variety of ways to make frame-recognizers, but the field is still in its infancy. The recognizers of our higher-level agencies might have to include mechanisms as complex as difference-engines in order to match their relational descriptions to actual situations.
Each of the drawings below can be seen in at least two different ways.
The drawing on the left might represent either a single candlestick or two people facing each other. The drawing on the right looks like a cube — but first it looks like a cube as seen from above and then, suddenly, it looks like a cube as seen from below. Why does each drawing seem to change its character from time to time? Why can't we see both forms at once? Because, it seems, our agencies can tolerate just one interpretation at a time.
We must ask certain questions here. First, what enables us to see those pictures as composed of the features we call by names like edges, lines, corners, and areas? Our vision-systems seem virtually compelled to group the outputs of our sensors into entities like these. Next, what enables us to see those features as grouped together to form larger objects? Apparently, our vision-systems again are virtually compelled to represent each of those features, be it a corner, edge, or area, as belonging to one and only one larger object at a time. I won't discuss those questions in this book except to suggest a general hypothesis:
Our vision-systems are born equipped, on each of several different levels, with some sort of locking-in machinery that at every moment permits each part, at each level, to be assigned to one and only one whole at the next level.
We should also ask, how do we recognize those objects as examples of familiar things like faces, cubes, or candlesticks? And again we'll make the similar hypothesis that our memory-frame machinery also uses locking-in machinery that permits each object to be attached only to one frame at a time. The end result is that in every region of the picture, the frames must compete with one another to account for each feature.
When we first discussed how Builder works, we assumed that it employed a vision-agent, See, to locate the various blocks it needs. However, we never discussed how See itself might work. A person simply looks and sees — but that's more complicated than it seems. For instance, even a simple cube looks different from each point of view, since as you move, the images it makes inside your eye keep changing in both shape and size.
How strange and dangerous moving would be if every step made everything seem wholly new! But that's not how it seems to us. When we move to the right, so that A becomes invisible, we remember what we learned when we saw it, and it still seems part of what we're seeing now. How can this be? Here is a theory of why things seem to stay the same, even when what we see of them keeps changing as we move around.
Frame-Arrays. When we move, our vision-systems switch among a family of different frames that all use the same terminals.
I'll use the term frame-arrays for these groups of frames that share the same terminals. When you represent a thing's details with a frame-array, you can continue to move around yet keep in mind all that you've observed from those different viewpoints, even though you've never seen them all at once. This gives us the wonderful ability to conceive of all of an object's different views as aspects of a single thing.
I do not mean to suggest that every time you see a new object you build a brand-new frame-array for it. First, you try to match what you see to the frame-arrays in the memories you have accumulated and refined over periods of many years. How do frame-arrays originate? I would assume that this underlying pattern — of families of frames that all share common terminals — is built into the architectures of major sections of the brain. But although that pattern is built in, developing the skills for using it involves each child in more than a decade of predestined learning.
What makes objects seem to stay in place no matter how the viewer moves? To common sense this is no mystery: it's simply that we're seeing all the time and keeping contact with the world. However, I suspect that if we had to start seeing all over again from every moment to the next, we'd scarcely be able to see at all. This is because our higher-level agents don't see the outputs of the sensors in our eyes at all. Instead, they watch the states of middle-level agencies that don't change state so frequently. What keeps those inner models of the world from changing all the time? This is the function of our frame-arrays: to store what we learn about the world at terminals that stay unchanged when we move our heads and bodies around. This explains a wonderful pseudoparadox: objects in the world seem to change only when the pictures they project into our eyes don't change — that is, don't change according to our expectations. For example, when you walk past a circular dish, your frame-arrays expect that circle to turn into an ellipse. When that actually happens,
the shape continues to look circular. However, should that expected change fail to occur, the shape will seem to change of its own accord.
How, then, do we automatically compensate for changes of view? The system could work just as we described in section 24.8: by using the same direction-nemes both to control our own motions and to select frames from our frame-arrays. For example, you might use several frames to represent an image of a cube, arranged in a network like this:
When you activate your move east direction-neme in order to make your body move in that direction, the same signal will also make this frame-array replace the middle frame with the one to its left. This will compensate for your change of viewpoint and determine what you expect to see — and the scene will appear to remain stationary! Michael Crichton has suggested that when you move inside such a space, you must unconsciously be registering the distortions of the shape, the moving walls and corners. Only you don't interpret these as changes in the room itself, but use them as more accurate cues to orient yourself in the space.
You can bypass this entire system by gently pushing the side of your eye with your finger; then the world will indeed appear to move, because your frame-arrays do not receive the corresponding direction signals!
Imagine what these frame-arrays can do! They let us visualize imaginary scenes, such as what might happen when we move, because the frames for what we can expect to see are filled in automatically. Not only that, but by using other processes to fill in all those terminals, we can imagine scenes and views of things we've never seen before. Still, many people find it hard to consider the thought that mental images could be based on anything as crude as frame-arrays. The world of our experience seems so perfectly continuous. Could such smooth thoughts emerge from sudden frame-to-frame jumps? If the mind kept jerking from one frame to another, wouldn't what we experience seem equally abrupt? Yet we rarely feel our minds change frames, any more than we perceive a visual scene as composed of disconnected spots of light. Why do we have the sense that things proceed in smooth, continuous ways? Is it because, as some mystics think, our minds are part of some flowing stream? I think it's just the opposite: our sense of constant, steady change emerges from the parts of mind that manage to insulate themselves against the continuous flow of time!
In other words, our sense of smooth progression from one mental state to another emerges not from the nature of that progression itself, but from the descriptions we use to represent it. Nothing can seem jerky except what is represented as jerky. Paradoxically, our sense of continuity comes from our marvelous insensitivity to most kinds of changes rather than from any genuine perceptiveness. Existence seems continuous to us not because we continually experience what is happening in the present, but because we hold to our memories of how things were in the recent past. Without those short-term memories, all would seem entirely new at every instant, and we would have no sense at all of continuity or, for that matter, of existence.
One might suppose that it would be wonderful to possess a faculty of continual awareness. But such an affliction would be worse than useless, because the more frequently our higher-level agencies change their representations of reality, the harder it is for them to find significance in what they sense. The power of consciousness comes not from ceaseless change of state, but from having enough stability to discern significant changes in our surroundings. To notice change requires the ability to resist it. In order to sense what persists through time, one must be able to examine and compare descriptions from the recent past. We notice change in spite of change, not because of it.
Our sense of constant contact with the world is not a genuine experience; instead, it is a form of immanence illusion. We have the sense of actuality when every question asked of our visual-systems is answered so swiftly that it seems as though those answers were already there. And that's what frame-arrays provide us with: once any frame fills its terminals, the terminals of the other frames in its array are also filled. When every change of view engages frames whose terminals are already filled, albeit only by default, then sight seems instantaneous.
 You'd be as shocked as though the world had changed before your eyes because so many of your expectations were not met. When we look about a familiar place, we know roughly what to expect. But what does expect mean?
Whenever we become familiar with some particular environment like an office, home, or outdoor place, we represent it with a frame-array whose terminals have already been filled. Then, for each direction of motion inside that environment, our vision-systems activate the corresponding frames of that array. We also activate the corresponding frames even when we merely consider or imagine a certain body motion — and this amounts to knowing what to expect. In general, each frame of a spatial frame-array is controlled by some direction-neme. However, in surroundings that are either especially familiar or whose relationships we do not understand, we may learn to use more specific stimuli instead of using direction-nemes to switch the frames. For example, when you approach a familiar door, the frame for the room that you expect to find behind that door might be activated, not by your direction of motion, but by your recognition of that particular door. This could explain how a person can reside in the same home for decades, yet never learn which of its rooms share common walls.
In any case, all this is oversimplified. Many of our frame-arrays must require more than nine direction views; they need machinery to modify the sizes and shapes of their objects; they must be adapted to three dimensions; and they must be able to represent what happens at intermediate moments during motion from one view to another. Furthermore, the control of frame selection cannot depend on a single, simple set of direction-nemes, for we must also compensate for the motions of our eyes, neck, body, and legs. Indeed, a major portion of our brain-machinery is involved with such calculations and corrections, and it takes a long time to learn to use all that machinery. The psychologist Piaget found that it takes ten years or more for children to refine their abilities to imagine how the same scene will appear from different viewpoints.
This was the basis of Hogarth's complaint. The artist felt that many painters and sculptors never learned enough about spatial transformations.  Accordingly, Hogarth worked out ways to train people to better predict how viewpoints change appearances.
[He who undertakes the acquisition of] perfect ideas of the distances, bearings, and oppositions of several material points and lines in even the most irregular figures, will gradually arrive at the knack of recalling them into his mind when the objects themselves are not before him — and will be of infinite service to those who invent and draw from fancy, as well as to enable those to be more correct who draw from the life.
I first conceived the idea of frames in the early 1970s, while working on making a robot that could see, and I described the theory in a 1974 essay entitled A Framework for Representing Knowledge. The essay influenced the next decade of research on Artificial Intelligence, despite the fact that most readers complained that its explanations were too vague. In retrospect, it seems those explanations were at just the right level-bands of detail to meet the needs of that time, which is why the essay had the effect it did. If the theory had been any vaguer, it would have been ignored, but if it had been described in more detail, other scientists might have tested it, instead of contributing their own ideas. Then they might have found my proposals were inadequate. Instead, many versions were suggested by other people, and frame-based programming became popular.
Two students in particular, Scott Fahlman and Ira Goldstein, claimed to understand what I had meant — and then explained many details I hadn't imagined at all. Another student, Terry Winograd, worked on making a robot that understood a certain class of English-language sentences; this led to important theories about the relation between grammar and its effect upon a listener. Then, since that robot's task was building towers of children's blocks, Winograd also worked out many details of how to make a Builder. You can see how his theories affected this book. Yet another student, Eugene Charniak, worked on the problem of how young children understand the stories they read. He spent at least a solid year thinking about one such story, which had to do with bringing a kite to a birthday party. Shortly, you'll see the influence Charniak had on this book.
All along, I had felt that the frame idea itself was rather obvious and perhaps implicit in the earlier work of psychologists like Bartlett. I considered the more important concept in the 1974 essay to be the idea of a frame-system — renamed frame-array in this book. I was surprised that the frame idea became popular while the frame-array idea did not. The neme concept emerged in 1977 (under the term C-lines ); the K-line idea crystallized in 1979. The concept of pronomes was in my unconscious mind for several years but did not crystallize until, while writing this book, I realized how to reformulate several of Roger Schank's early ideas into the form of Trans-frames. The scheme proposed in this book, in which the frame-terminals are controlled by bundles of nemes or isonomes, did not emerge until a full decade after the original concept of a frame-array.
Many questions remain about how frames might work. For example, it should be possible to recognize several different things at once by using different frames in parallel. But how can we see many faces in a crowd at once, or bricks in a wall, or chairs in a room? Do we make many copies of the same frame? I suspect that's impractical. Instead, perhaps we match each frame only to one example at a time — and simply assume that the same frame also applies to every other visible object that shares some characteristic features with the object under attention.
What happens when a child reads a story that begins like this?
Mary was invited to Jack's party. She wondered if he would like a kite.
If you asked what that kite was for, most people would answer that it must be a birthday present for Jack. How amazing it is that every normal person can make such complicated inferences so rapidly — considering that the idea of a gift was never mentioned at all! Could any machine do such remarkable things? Consider all the other assumptions and conclusions that almost everyone will make:
The party is a birthday party. Jack and Mary are children. She is Mary. He is Jack.
She is considering giving Jack a kite. She wonders if he would like the kite.
We call these understandings common sense. They're made so swiftly that they're often ready in our minds before a sentence is complete! But how is this done? In order to realize that the kite is a present, one has to use such knowledge as that parties involve presents, that presents for children are usually toys, and that kites are appropriate toys to be given as presents. None of this is mentioned in the story itself. How do we bring together all that scattered knowledge so quickly? Here's what I think must happen. Somehow the words Mary was invited to Jack's party arouses a party- invitation frame in the reader's mind — and attached to the terminals of that frame are certain memories of various concerns. Who is the host? Who will attend? What present should I bring? What clothing shall I wear? Each of those concerns, in turn, is represented by a frame to whose terminals are already attached, as default assignments, the most usual solutions to that particular kind of problem.
Such knowledge comes from previous experience. I was raised in a culture in which an invitation to a party carries the obligation to arrive well dressed and to bring a birthday present. Accordingly, when I read or hear that Mary was invited to a party, I attribute to Mary the same sorts of subjective reactions and concerns that I would have in such a situation. Therefore, although the story never mentions clothes or gifts at all, to expect their possible involvement seems only simple common sense. But though it is common, it is not simple. The next few sections speculate about how story understanding works.
We'll now see how frames can help to explain how we understand that children's tale. How do we know that the kite is a present for Jack — when neither sentence mentioned this?
Mary was invited to Jack's party. She wondered if he would like a kite.
After the first sentence activates a party-invitation frame, the reader's mind remains engaged with that frame's concerns — including the question of what type of birthday gift to bring. If this concern is represented by some subframe, what are the concerns of that subframe? That present must be something that will please the party host. Toy would be a good default for it, since that's the most usual kind of gift for a child.
Since Jack is a he and a kite is a toy, these two frames will merge perfectly — provided that the reader's frame for boy assumes that Jack is likely to enjoy kites. Then our two sentences combine perfectly to fill the present frame's terminals, and our problem is solved!
What makes a story comprehensible? What gives it coherency? The secret lies in how each phrase and sentence stirs frames into activity or helps already active ones to fill their terminals. When the first sentence of our story mentions a party, various frames are excited — and these are still active in the reader's mind when the next sentence is read. The ground is prepared for understanding the second sentence because so many agents are already ready to recognize possible references to presents, clothes, and other matters that might be related to birthday parties.
We've barely started to see what minds must do to comprehend the simplest children's tales. Let's look again at the beginning of our party story.
Mary was invited to Jack's party.
How marvelous that sentence is! How much it says in just six words! Two characters are introduced and quickly cast in clear-cut roles. We learn that there will be a party soon, with Jack as the host and Mary a guest — provided she accepts the invitation. We also learn that this setting is established in the past.
Those six short words tell even more. We can expect the story to focus on Mary's activities rather than Jack's — because Mary is the first word that attracts our attention. But to accomplish that, the narrator had to use a clever grammar-tactic. Normally, an English-language sentence begins with a phrase that describes the Actor responsible for some action, and we usually represent this with a simple Trans-frame.
JACK INVIT — ed MARY Donor action verb Recipient In this active verb form of sentence-frame, the verb is sandwiched between two nouns; the first describes a Donor and the second describes a Recipient. However, if our storyteller actually used the active form of sentence-frame, it would tend to mislead the listener into expecting Jack to be the central character of the story — if only because he is mentioned first. Fortunately, English grammar provides an alternative sentence-frame in which the Recipient is mentioned first — and which never mentions the Donor at all!
MARY was INVIT — ed Recipient was verb — ed.
How does the understanding listener detect this passive verb sentence-frame? Some language-agent has to notice the way the verb is sandwiched between was and -ed. As soon as this special subframe is recognized, the language-agency will reassign the first noun, Mary, not to the Donor terminal, but to the Recipient terminal — and thus Mary is represented as receiving the invitation. Why don't we need to say who the donor is? Because in this case the listener can assume it by default. Specifically, the expression Jack's party evokes a party-invitation frame, and in such situations it is typical for the host — or the host's parents — to invite the party guests. By thus arousing familiar frames, we can say a great deal in a very few words.
Dictionary definitions never say enough. Every child knows that a party is more than just a gathering assembled to celebrate someone's birthday. But no brief definition can describe the complicated customs, rules, and regulations that typical communities prescribe for such ceremonies. When I was a child, a birthday party could be expected to include at least the elements of the following script:
ARRIVAL. Greeting. Be well dressed. GIFT. Give birthday present to host or guest of honor.
GAMES. Activities like blindfold competitions. DECOR. Balloons. Favors. Crepe-paper decorations. PARTY-MEAL. Hotdogs, candies, ice-cream, etc. CAKE. With candles to represent the host's age. CEREMONY. Host tries to extinguish candles with single breath (to make a wish). SONG. All guests sing birthday song and eat cake.
This is merely an outline, for every item leads to other conditions and requirements. The birthday present has to please the host, of course, but there are other strong constraints on it as well. It ought to be brand new, of good quality, and it should not be ostentatiously extravagant. It ought to be suitably party-wrapped — that is, covered with a certain kind of color-printed wrapping paper and tied with colored ribbon. There are also constraints on other items in the script. The birthday cake should be covered with a sweet sugar frosting. In my childhood, the ice cream usually consisted of three colored stripes of different flavors: vanilla, strawberry, and chocolate. Because I did not like the strawberry flavor, my personal party script included the extra steps of finding another child willing to make a trade.
To all their young participants, such parties unfold exactly as a party should, with all these queer complexities. We take our social customs for granted, as though they were natural phenomena. Few guests or hosts will ever wonder why their parties have those explicit forms or ask about their origins. As far as any child can tell, that's just how parties ought to go; they always did and always will. And so it is with almost everything we know.
We take it for granted that anyone can understand a story. But every kind of narrative demands some listening skills. Even the best storytellers find it hard to entertain children, who are prone to interrupt with questions that make perfect sense by themselves but drift away from the story's theme. Where does Mary live? Does she have a dog? To listen well, a child must acquire potent forms of self-control.
The storyteller, too, must work to fix the focus of the listener's mind. If you were speaking about something else and suddenly, completely out of context, remarked, Mary was invited to Jack's party, an unprepared listener might wonder, Mary who? and look to see if you were addressing someone else. But you can first prepare the listener by saying, Would you like to hear a story? or simply, Once upon a time . . . What is the function of such a phrase? It has a very specific effect: to set the listener into a normal and familiar state of expecting to hear a certain type of narrative — a story. In the English tradition, stories typically begin by specifying the time — if only vaguely, by saying long ago. I'm told that in Japan most stories start with saying where as well — if only by some empty phrase like in a certain time and place. The biblical book of Job begins with, There was a man in the land of Uz . . .
Most stories start with just enough to set the scene. Then they introduce some characters, with hints about their principal concerns. Next, the storyteller gives some clues about some main event or problem to be solved. From that point on, the listener has a general idea of what comes next: there will be more development of the problem; then it will be resolved, somehow; and then the story will end, perhaps by giving some practical or moral advice. In any case, those magic story-starting words arouse, in knowing listeners' minds, great hosts of expectation-frames to help the listeners anticipate which terminals to fill.
Beyond arousing all these specific expectations, once upon a time plays one more crucial role: it says that what comes after it is fictional or, in any case, far too remote to activate much personal concern. Instead, it tells the listener to disregard the normal sympathies one should feel when real persons meet the monstrous destinies so usual in children's tales: to be turned into toads, imprisoned in stones, or devoured by terrible dragon beasts.
Part of what a sentence means depends upon its separate words, and part depends on how those words are arranged.
Round squares steal honestly. Honestly steal squares round.
What makes these seem so different in character, when both use the very same words? I'll argue that this is because your language-agency, immediately upon hearing the first word-string, knows exactly what to do with it because it fits a well-established sentence-frame. The second string fits no familiar form at all. But how do we fit those sentence-frames? We'll come to that presently, but for the moment, let's simply assume that our young listener has somehow come to classify words into various types, like nouns, adjectives, verbs, and adverbs.  Then our first string of words has this form:
Adjective Noun Verb Adverb Now we'll suppose our listener has learned a specific recognition- frame that is activated on hearing this string of particular types of words. This frame then executes a special process script that makes the following assignments to the terminals of a Trans-frame. The neme for steal is assigned to the Trans-frame's Action terminal, while the neme for squares is attached to the Actor terminal. The frame then activates scripts that modify the action steal by applying to it the neme for honestly and modify the object squares by applying to it the neme for round. Up to this point, everything works smoothly: the language-agency has found a use for every word. We have special names for the strings of words that we process with such fluency: we call them phrases or sentences.
A word-string seems grammatical if all its words fit quickly and easily into frames that connect suitably to one another.
However, at this point some serious conflicts start to appear within some other agencies because of certain incompatibilities. The frame for steal requires its Actor to be animate. A square can't steal, because it's not alive! Besides, the frame for steal expects an act that's reprehensible, and that clashes with the modifier for honestly. If that weren't bad enough, our agency for describing shape can't tolerate the polynemes for round and square when both are activated at the same time. It doesn't matter that our sentence is grammatical: so much turmoil is set up that most of its meaning cancels out and we regard it as nonsense. But it is important to recognize that the distinction between sense and nonsense is only partly a matter of grammar, for consider what happens when you hear these three words:
thief -- -- careless -- -- prison --
Although these do not establish any single well-formed grammar- frame, they activate some word-sense nemes that skip past all our grammar-forms to fit a familiar story-frame, a moral tale about a thief who's caught and reaps a just reward. Ungrammatical expressions can frequently be meaningful when they lead to clear and stable mental states. Grammar is the servant of language, not the master.
At various points in their development, most children seem suddenly to comprehend new kinds of sentences. Thus, once they learn to deal with single adjectives, some children quickly learn to deal with longer strings like these:
Dogs bark. Big dogs bark. Big shaggy dogs bark. Big black shaggy dogs bark.
If this were done by using word-string sentence-frames, it would require a separate frame for each different number of adjectives. Another scheme would not use any frames at all but have the language-agency convert each adjective, as it arrives, into some corresponding neme. And yet another scheme to handle this (still popular among some grammar theorists) would have each successive adjective arouse a new subframe inside the previous one. However, when we look more closely at how people use adjectives, we find that these strings are not simple at all. Compare the two phrases below:
The wooden three heavy brown big first boxes . . . The first three big brown heavy wooden boxes . . .
Our language-agents scarcely know what to do with that first string of words because it doesn't fit the patterns we normally use for describing things. This suggests that we use framelike structures for describing nouns as well as verbs — that is, for describing things as well as actions. To fill the terminals of those frames, we expect their ingredients to arrive in a more or less definite order. We find it hard to understand a group of English adjectives unless they are arranged roughly as shown below.
Whenever a language community can agree on forms like these, expression becomes easier. Then every individual can learn, once and for all, where to put — and where to find — the answers to questions most frequently asked. In English one learns to say green box, while in French one says box green. It doesn't matter which order is used — as long as everyone agrees to do it the same way. But what are the questions most frequently asked — the ones we build into our language-forms? The answer to this is likely to be somewhat circular, since the language culture in which we're raised will probably affect the kinds of questions that will seem most natural to ask. Still, there could be useful clues in features that are common to many different languages.
Many scientists have asked, indeed, why so many human languages use similar structures such as nouns, adjectives, verbs, clauses, and sentences. It is likely that some of these reflect what is genetically built into our language-agencies. But it seems to me even more likely that most of these nearly universal language-forms scarcely depend on language at all — but reflect how descriptions are formed in other agencies. The most common forms of phrases could arise not so much from the architecture of the language-agencies as from the machinery used by other agencies for representing objects, actions, differences, and purposes — as suggested in section 22.7 — and from how those other agencies manipulate their memories. In short, the ways we think must have a strong and universal influence on how we speak — if only through its influence on the sorts of things we'll want to say.
We've seen how a four-word sentence such as Round squares steal honestly could be made to fit a certain four-terminal frame. But what about a sentence like The thief who took the moon moved it to Paris ? It would be dreadful if we had to learn a new and special ten-word frame for each particular type of ten-word string! Clearly we don't do any such thing. Instead, we use the pronoun who to make the listener find and fill a second frame. This suggests a multistage theory. In the earliest stages of learning to speak, we simply fill the terminals of word-string frames with nemes for words. Then, later, we learn to fill those terminals with other filled-in language-frames. For example, we can describe our moon sentence as based on a top-level Trans-frame for move whose Actor terminal contains a second Trans-frame for took :
Using frames this way simplifies the job of learning to speak by reducing the number of different kinds of frames we have to learn. But it makes language learning harder, too, because we have to learn to work with several frames at once.
How do we know which terminals to fill with which words? It isn't so hard to deal with red, round, thin-peeled fruit, since each such property involves a different agency. But that won't work for Mary loves Jack, since Jack loves Mary has the very same words, and only their order indicates their different roles. Each child must learn how the order of words affects which terminal each phrase should fill. As it happens, English applies the same policy both to Mary loves Jack and to our moon sentence:
Assign the Actor pronome to the phrase before the verb. Assign the Object pronome to the phrase after the verb.
The policies for assigning phrases to pronomes vary from one language to another. The word order for Actor and Object is less constrained in Latin than in English, because in Latin those roles can be specified by altering the nouns themselves. In both languages we often indicate which words should be assigned to other pronome roles by using specific prepositions like for, by, and with. In many cases, different verb types use the same prepositions to indicate the use of different pronomes. At first such usages may seem to be arbitrary, but they frequently encode important systematic metaphors; in section 21.2 we saw how from and to are used to make analogies between space and time. How did our language- forms evolve? We have no record of their earliest forms,
but they surely were affected at every stage by the kinds of questions and problems that seemed important at the time. The features of present-day languages may still contain some clues about our ancestors' concerns.
Some language scholars seem to think that what we do in language is unique, in the filling of frames with other frames to open up a universe of complicated structure-forms. But consider how frequently we do similarly complex things in understanding visual scenes. The language-agency must be able to interrupt itself, while handling one phrase, to work on parts of another phrase, and this involves some complex short-term memory skills. But in vision, too, there must be similar processes involved in breaking scenes apart and representing them as composed of objects and relationships. The picture below suggests how similar such processes may be. In language, the problem is to recognize that the two words took and out both belong to the same verb phrase, although they are separated in time. In vision, the problem is to recognize the two regions of a tabletop as being parts of the same object, although they are separated in space.
Notice also that we cannot see the tops of the blocks that serve as legs — and yet we do not have the slightest doubt about where they end. Similarly, the ends of language phrases are frequently unmarked — yet again we can tell where they end. In The thief who took the moon moved it to Paris, the word who marks the beginning of a new frame, but there is no special word to indicate the ending of that phrase. Why don't we wrongly assign the moon to the Actor of the spurious phrase, The moon moved it to Paris? It is because we first heard . . . who took the moon, which caused the moon to get attached to the Object pronome of the Trans-frame for took — so now it's not available to serve as Actor in the frame for moved. The thief is still available to play that role. I don't mean to suggest we can never assign the same phrase to two different roles, only that good speakers choose their forms so that this doesn't happen by accident.
Did our capacity to deal with phraselike structures evolve first in language or in vision? Among our ancestors, vision greatly antedates language, so if these abilities are related, our language-agencies themselves more likely evolved from variants of genes that first evolved in shaping the architecture of our vision-systems. Today we have no way to verify such a conjecture, but future geneticists may become able to trace the ancestry of many such relationships by examining the genes that generate the corresponding brain structures.
The vocabulary of a language — the words themselves — is the product of a project that spans the history of a culture and can involve millions of person years of work. Every sense of every word records some intellectual discovery that now outlives the myriad other, less distinguished thoughts that never earned a name.
Each person invents some new ideas, but most of these will die when their owners do, except for those that make their way into the culture's lexicon. Still, from that ever-growing reservoir we each inherit many thousands of powerful ideas that all our predecessors found. Yet it is no paradox to say that even as we inherit those ideas from our culture, we each must reinvent them for ourselves. We cannot learn meanings only by memorizing definitions: we must also understand them. Each situation in which a word is used must suggest some mixture of materials already in the mind of a listener, who then, alone, must attempt to assemble these ingredients into something that will work in consonance with other things already learned. Definitions sometimes help — but still one must separate the essences from the accidents of the context, link together structures and functions, and build connections to the other things one knows.
A word can only serve to indicate that someone else may have a valuable idea — that is, some useful structure to be built inside the mind. Each new word only plants a seed: to make it grow, a listener's mind must find a way to build inside itself some structure that appears to work like the one in the mind from which it was learned.
Along with the words, we also have to learn the grammar-tactics for using them. Most children start by using only one or two words at a time. Then, over the next two or three years, they learn to speak in sentences. It usually takes a full decade to learn most of the conventions of adult speech, but we often see relatively sudden advances over concentrated periods of time. How do children learn such complicated skills so quickly? Some language theorists have suggested that children learn to use grammar so readily that our brains must be born with built-in grammar-machinery. However, we've seen that our visual-systems solve many similar problems in even earlier years — and we've also seen that when they learn to play with spoons and pails, children must learn yet other languagelike skills for managing the Origins, Destinations, Recipients, and Instruments of their actions. Thus, many sections of our brains appear to demonstrate capacities for rearranging pronome roles even before we learn to speak. If so, perhaps we ought not to wonder so much about how children learn to speak so readily. Instead, we ought to wonder why it takes so long, when they already do so many similar things inside their heads.
How do we choose the words we speak, and how do we understand what others say? Earlier, I suggested that in the course of learning language we accumulate various processes and tactics that enable us to partially reproduce our own mental operations in other speakers. These processes affect our choices of words, the forms we select for phrases and sentences, and the styles in which we frame our narratives. There have been many attempts to study how children learn language, but psychologists do not yet have coherent theories about the underlying processes. For example, we do not yet even know whether we learn each bit of grammar only once — or whether we have to learn it twice, for speaking and for understanding what other people say.
We know so little about such matters that we can scarcely even speculate about the nature of those early language-learning steps. Perhaps the process starts with some agents that can enable a child to make various vocal sounds in response to specific internal states. These agents then become involved with built-in predestined learning processes that lead to limited abilities to imitate other sounds the child hears by using feedback from its ears. Later stages might then engage new layers of agents that connect word-sound agents to whichever polynemes are most frequently attached to certain pronomes in the language-agency. Once a suitable variety of such processes are established, more layers of frame- and memory- controlling agents could learn to support more complex language skills.
Let's try to imagine what kind of process could produce a language phrase that expresses a description of an object. Suppose, for example, that you want to draw attention to a certain very big box. To imagine such a thing in the first place, you might first have to activate your polyneme for box and then arouse some other isonomes and polynemes that modify the state of your Size agency. To express very big box might thus require grammar-tactics that express three mental operations:
--- box expresses the arousal of the box polyneme; --- big expresses a process that selects the Size agency;
--- very expresses an isonome that adjusts the sensitivities of agents in whichever agency was selected.
I do not mean to suggest that a child's earliest three-word noun phrases must be based upon such complicated processes; more likely they begin with simpler sequence scripts. Eventually, though, more complex systems intervene to replace the simple scripts by intricate kinds of frame-arrays that enable the child to make more complex rearrangements of what becomes attached to its expression-frames. Then, as the language-agency acquires more isonome-controlling skills, the child can learn to use pronouns like it or she to express other structures that are already attached to suitable pronomes. Also, as we develop skills for building chains and trees from other frames, the language-agency can learn to use corresponding grammar-tactics to express those chains — stringing together phrases and sentences with conjunction words like and
and but. Similarly, as we improve our methods for controlling memories and managing interruptions, we can learn to combine those skills with clause-interrupting forms like who and which. There seems scarcely any limit to the complexity of our social inventions for expressing mental processes, and it takes most children many years to master all the language arts their ancestors evolved.
Every discourse works on several scales. Each word you hear can change your state in a way that depends upon all the structures you have built while listening to the words that came before. Most of those structures are themselves mere transient things, which persist for only a few moments before you rearrange some of their parts and perhaps discard the rest entirely. Thus, a car might first appear as the subject of a sentence, then become a mere vehicle or instrument in the next sentence; finally, the whole scenario might be used merely to modify a personal trait of some actor in a larger scene. As a discourse proceeds, details on each scale become absorbed into larger-scale representation networks whose outlines become increasingly remote from the individual words that were used to construct them.
It would be wonderful to have a compact, self-contained theory that explains all our language-forms. But that ideal cannot be realized because words are merely the external signs of very complex processes, and there is no clear boundary between language and all the rest of what we call thinking. To be sure, the boundaries of words themselves are relatively clear, and when they have multiple meanings, our grammar-tactics can often help us to assign the proper senses to various terminals and other structures. These tactics include all sorts of inflections, prepositions, word orderings, and signals that indicate how to include one phrase inside another. We also combine words into larger expressions that range in vagueness of boundaries from compact clichés like hot dog to diffuse signals that are scarcely linked to specific words at all; these include our hard-to-describe nuances of phrasing, rhythm, intonation, and shifts of style and flow.
We're normally quite unaware of how our grammar-tactics constrain us in our choices of words. We're often somewhat more aware of other language-tactics we use to guide our listeners' minds — to change the focus from one theme to another, to adjust the levels of detail, to shift between foreground and setting. We learn to use phrases like by the way to change the topic of concern, to say for example to shift to a finer level of detail, to say but to modify an expectation or to interrupt the usual flow, or to say in any case or in spite of that to indicate the end of an interruption or elaboration.
But even all this is only a small part of language. To understand what people say, we also exploit our vast stores of common knowledge, not only about how specific words are related to the subjects of concern, but also about how to express and discuss those subjects. Every human community evolves a great array of discourse-forms to shape its stories, explanations, conversations, discussions, and styles of argument. Just as we learn grammar-forms for fitting words to sentence-frames, we also build up stocks of plots to organize our story-tales, and standard personalities to fill the roles of their protagonists — and every child must learn these forms.
Our reader must be anxious to know what finally became of Mary and that kite. Here is more of that story.
Mary was invited to Jack's party. She wondered if he would like a kite. Jane said, Jack already has a kite. He will make you take it back.
What does the pronoun it mean here? Clearly Jane is speaking not of the kite that already belongs to Jack, but of the new kite that Mary is thinking of giving to him. But what leads the listener to assume that this is what the storyteller meant? There are many issues here besides the question of which kite is involved. How do we know it refers to a kite at all? Does take it back mean take it back from Jack or to return it to the store? For the sake of simplicity, let's put aside the other possibilities and assume that it must mean a kite. But in order to decide which kite is meant, we still must understand the larger phrase take it back. This phrase must refer to some structure already in the listener's mind; the narrator expects the listener to find the appropriate structure by activating an appropriate fragment of commonsense knowledge about giving and receiving birthday presents. But since every listener knows so many things, what sorts of processes could activate the appropriate knowledge without taking too much time? In 1974 Eugene Charniak, a graduate student at MIT, asked how each phrase of this story works to prepare the reader to comprehend the subsequent phrases. He suggested that whenever we hear about a particular event, specific recognition-agents are thereby aroused. These then proceed actively to watch and wait for other related types of events. (Because these recognition-agents lurk silently, to intervene only in certain circumstances, they are sometimes called demons. ) For example, whenever a story contains the slightest hint that someone may have purchased a gift, specific demons might be aroused that watch for events like these:
If there is evidence that the recipient rejects the gift, look for signs of it being returned. If you see evidence of a gift being returned, look for signs that the recipient rejected it.
Charniak's thesis raised many questions. How easy should it be to activate demons? How long should they then remain active? If too few demons are aroused, we'll be slow to understand what's happening. But if too many become active, we'll get confused by false alarms. There are no simple solutions to these problems, and what we call understanding is a huge accumulation of skills. You might understand certain parts of a story by using separate, isolated demons; you might comprehend other aspects of that same story by using larger-scale processes that try to match the sequence of events to various remembered scripts; yet other understandings might depend upon which agents are aroused by various micronemes. How much of the fascination in telling a story, or in listening to one, comes from the manipulations of our demons' expectations?
It would be wonderful never to make mistakes. One way would be to always have such perfect thoughts that none of them is ever wrong. But such perfection can't be reached. Instead we try, as best we can, to recognize our bad ideas before they do much harm. We can thus imagine two poles of self-improvement. On one side we try to stretch the range of the ideas we generate: this leads to more ideas, but also to more mistakes. On the other side, we try to learn not to repeat mistakes we've made before. All communities evolve some prohibitions and taboos to tell their members what they shouldn't do. That, too, must happen in our minds: we accumulate memories to tell ourselves what we shouldn't think.
But how could we make an agent to prevent us from doing something that, in the past, has led to bad or ineffectual results? Ideally, that agent would keep us from even thinking that bad idea again. But that seems almost paradoxical, like telling someone, Don't think about a monkey! Yet there is a way to accomplish this. To see how it works,
imagine the sequence of mental states that led to a certain mistake:
We could prevent the undesired action from taking place by introducing an agent that recognizes the state which, in the past, preceded the undesired action.
Suppressor-agents wait until you get a certain bad idea. Then they prevent your taking the corresponding action, and make you wait until you think of some alternative. If a suppressor could speak, it would say, Stop thinking that!
Suppressors could indeed prevent us from repeating actions that we've learned are bad. But it is inefficient to wait until we actually reach undesirable states, then have to backtrack. It would be more efficient to anticipate such lines of thought so that we never reach those states at all. In the next section we'll see how to do this by using agents called censors.
Censor-agents need not wait until a certain bad idea occurs; instead, they intercept the states of mind that usually precede that thought. If a censor could speak, it would say, Don't even begin to think that!
Though censors were conceived of long ago by Sigmund Freud, they're scarcely mentioned in present-day psychology. I suspect that this is a serious oversight and that censors play fundamental roles in how we learn and how we think. Perhaps the trouble is that our censors work too well. For, naturally, it is easier for psychologists to study only what someone does — instead of what someone doesn't do.
To see what suppressors and censors have to do, we must consider not only the mental states that actually occur, but others that might occur under slightly different circumstances.
Suppressors work by interceding to prevent actions just before they would be performed. This leads to a certain loss of time, because nothing can be done until acceptable alternatives can be found. Censors avoid this waste of time by interceding earlier. Instead of waiting until an action is about to occur, and then shutting it off, a censor operates earlier, when there still remains time to select alternatives. Then, instead of blocking the course of thought, the censor can merely deflect it into an acceptable direction. Accordingly, no time is lost.
Clearly, censors can be more efficient than suppressors, but we have to pay a price for this. The farther back we go in time, the larger the variety of ways to reach each unwanted state of mind. Accordingly, to prevent a particular mental state from occurring, an early-acting censor must learn to recognize all the states of mind that might precede it. Thus, each censor may, in time, require a substantial memory bank. For all we know, each person accumulates millions of censor memories, to avoid the thought-patterns found to be ineffectual or harmful.
Why not move farther back in time, to deflect those undesired actions even earlier? Then intercepting agents could have even larger effects with smaller efforts and, by selecting good paths early enough, we could solve complex problems without making any mistakes at all. Unfortunately, this cannot be accomplished only by using censors. This is because as we extend a censor's range back into time, the amount of inhibitory memory that would be needed (in order to prevent turns in every possible wrong direction) would grow exponentially. To solve a complex problem, it is not enough to know what might go wrong. One also needs some positive plan.
As I mentioned before, it is easier to notice what your mind does than to notice what it doesn't do, and this means that we can't use introspection to perceive the work of these inhibitory agencies. I suspect that this effect has seriously distorted our conceptions of psychology and that once we recognize the importance of censors and other forms of negative recognizers, we'll find that they constitute large portions of our minds.
Sometimes, though, our censors and suppressors must themselves be suppressed. In order to sketch out long-range plans, for example, we must adopt a style of thought that clears the mind of trivia and sets minor obstacles aside. But that could be very hard to do if too many censors remained on the scene; they'd make us shy away from strategies that aren't guaranteed to work, and tear apart our sketchy plans before we can start to accomplish them.
We spend our lives at learning things, yet always find exceptions and mistakes. Certainty seems always out of reach. This means that we have to take some risks to keep from being paralyzed by cowardice. But to keep from having accidents, we must accumulate two complementary types of knowledge:
We search for islands of consistency within which ordinary reasoning seems safe. We work also to find and mark the unsafe boundaries of those domains.
In civilized communities, appointed guardians post signs to warn about sharp turns, thin ice, and animals that bite. And so do our philosophers, when they report to us their paradoxical discoveries — those tales of the Liar who admits to lying and the Barber who shaves all the people who do not shave themselves. These valuable lessons teach us which thoughts we shouldn't think; they are the intellectual counterparts to Freud's emotion censors. It is interesting how frequently we find paradoxical nonsense to be funny, and when we come to the section on jokes, we'll see why this is so. When we look closely, we find that most jokes are concerned with taboos, injuries, and other ways of coming to harm — and logical absurdities can also lead to harm.
We tell our children not to cross the road unless they are sure no car is coming. But what do we mean by sure ? No one can ever really prove that no car is coming, since there is no way to rule out the possibility that some mad scientist has found a way to make cars invisible. In ordinary life we have to deal with usual instead of true. All we can really ask a child to do is look both ways before you cross. In the real world, it makes no sense to ask for absolute certainty.
Unfortunately there are no simple, foolproof ways to get around the inconsistencies of common sense. Accordingly, we each must learn specific ways to keep from various mistakes. Why can't we do that logically? The answer is that perfect logic rarely works. One difficulty is finding foolproof rules for reasoning. But the more serious problem is that of finding foolproof bases for our arguments. It is virtually impossible to state any facts about the real world that actually are always true. We observed this when we discussed Birds can Fly. This statement applies to typical birds, but not to birds imprisoned in small cages, chained with leg irons, or under the influence of high-gravity fields. Similarly, when you're told, Rover is a dog, you'll assume that Rover has a tail, since your frame for a typical dog has a terminal for a tail. But should you learn that Rover lacks a tail, your mind won't self-destruct; instead, you'll change your Rover-frame — but still expect most other dogs to keep their tails.
Exceptions are a fact of life because few facts are always true. Logic fails because it tries to find exceptions to this rule.
Why do jokes have such peculiar psychological effects? In 1905, Sigmund Freud published a book explaining that we form censors in our minds as barriers against forbidden thoughts. Most jokes, he said, are stories designed to fool the censors. A joke's power comes from a description that fits two different frames at once. The first meaning must be transparent and innocent, while the second meaning is disguised and reprehensible. The censors recognize only the innocent meaning because they are too simple-minded to penetrate the forbidden meaning's disguise. Then, once that first interpretation is firmly planted in the mind, a final turn of word or phrase suddenly replaces it with the other one. The censored thought has been slipped through; a prohibited wish has been enjoyed.
Freud suggested that children construct censors in response to prohibitions by their parents or peers. This explains why so many jokes involve taboos concerning cruelty, sexuality, and other subjects that human communities typically link to guilt, disgust, or shame. But it troubled Freud that this theory did not account for the nonsense jokes people seem to enjoy so much. The trouble was that these seemed unrelated to social prohibitions. He could not explain why people find humor in the idea of a knife that has lost both its blade and its handle.
Freud considered several explanations to account for pointless nonsense jokes but concluded that none of those theories was good enough. One theory was that people tell nonsense jokes for the pleasure of arousing the expectation of a real joke and then frustrating the listener. Another theory was that senselessness reflects a wish to return to carefree childhood, when one was permitted to think without any compulsion to be logical, and to put words together without sense, for the simpler pleasures of rhythm or rhyme. Freud put it this way:
Little by little the child is forbidden this enjoyment, till there remain only significant combinations of words. But attempts still emerge to disregard restrictions which were learned.
In yet a third theory, Freud conjectured that humor is a way to ward off suffering — as when, in desperate situations, we make jokes as though the world were nothing but a game. Freud suggested that this is when the superego tries to comfort the childlike ego by rejecting all reality; but he was uneasy about this idea because such kindliness conflicted with his image of the superego's usual stern, strict character.
Despite Freud's complicated doubts, I'll argue that he was right all along. Once we recognize that ordinary thinking, too, requires censors to suppress ineffectual mental processes, then all the different-seeming forms of jokes will seem more similar. Absurd results of reasoning must be tabooed as thoroughly as social mistakes and inanities!
People often wonder if a computer could ever have a sense of humor. This question seems natural to those who think of humor as a pleasant but unnecessary luxury. But I'll argue quite the opposite — that humor has a practical and possibly essential function in how we learn.
When we learn in a serious context, the result is to change connections among ordinary agents. But when we learn in a humorous context, the principal result is to change the connections that involve our censors and suppressors.
In other words, my theory is that humor is involved with how our censors learn; it is mainly involved with negative thinking, though people rarely realize this. Why use such a distinct and peculiar medium as humor for this purpose? Because we must make a sharp distinction between our positive, action-oriented memories and the negative, inhibitory memories embodied in our censors.
Positive memory-agents must learn which mental states are desirable. Negative memory-agents must learn which mental states are undesirable.
Because these two types of learning required different processes, it was natural to evolve social signals to communicate that distinction. When people do things that we regard as good, we speak to them in encouraging tones — and this switches on their positive learning machinery.  I suspect that scolding and laughing have somewhat different effects: scolding tends to produce suppressors, but laughing tends to produce censors. Accordingly, the effect of derisive humor is somewhat more likely to disrupt our present activity. This is because the process of constructing a censor deprives us of the use of our temporary memories, which must be frozen to maintain the records of our recent states of mind.
Suppressors merely need to learn which mental states are undesirable. Censors must remember and learn which mental states were undesirable.
To see why humor is so often concerned with prohibition, consider that our most productive forms of thought are just the ones most subject to mistakes. We can make fewer errors by confining ourselves to cautious, logical reasoning, but we'll also discover fewer new ideas. More can be gained by using metaphors and analogies, even though they are often defective and misleading. I think this is why so many jokes are based on recognizing inappropriate comparisons. Why, by the way, do we so rarely recognize the negative character of humor itself? Perhaps it has a funny side effect: while shutting off those censored thoughts, our censors also shut off thoughts about themselves — and make themselves invisible.
This solves Freud's problem about nonsense jokes. The taboos that grow within social communities can be learned only from other people. But when it comes to intellectual mistakes, a child needs no helpful friend to scold it when a tower falls, when it puts a spoon in its ear, or thinks a thought that sets its mind into a fruitless and confusing loop. In other words, we can detect many of our own intellectual failures all by ourselves. Freud's theory of jokes was based on the idea that censors suppress thoughts that would be considered naughty by those to whom we are attached. He must simply have overlooked the fact that ineffectual reasoning is equally naughty — and therefore equally funny — in the sense that it, too, ought to be suppressed. There is no need for our censors to distinguish between social incompetence and intellectual stupidity.
What would a Martian visitor think to see a human being laugh? It must look truly horrible: the sight of furious gestures, flailing limbs, and thorax heaving in frenzied contortions. The air is torn with dreadful sounds as though, all at once, that person wheezes, barks, and chokes to death. The face contorts in grimaces that mix smiles and yawns with snarls and frowns. What could cause such a frightful seizure? Our theory suggests a simple answer:
The function of laughing is to disrupt another person's reasoning!
To see and hear a person laugh creates such chaos in the mind that you can't proceed along your present train of thought. Derision makes you feel ridiculous; it prevents you from being serious. What happens then? Our theory has a second part:
Laughter focuses attention on the present state of mind!
Laughter seems to freeze one's present state of mind in its tracks and hold it up to ridicule. All further reasoning is disrupted, and only the joke-thought remains in sharp focus. What is the function of this petrifying effect?
By preventing you from taking seriously your present thought, and thus proceeding to develop it, laughter gives you time to build a censor against that state of mind.
In order to construct or improve a censor, you must retain your records of the recent states of mind that made you think the censored thought. This takes some time, during which your short-term memories are fully occupied — and that will naturally disrupt whichever other processes might try to change those memories.
How could all this have evolved? Like smiling, laughter has a curious ambiguity, combining elements of affection and conciliation with elements of rejection and aggression. Perhaps all these ancestral means of social communication became fused to compose a single, absolutely irresistible way to make another person cease an activity regarded as objectionable or ridiculous. If so, it is no accident that so many jokes mix elements of pleasure, cruelty, sexuality, aggression, and absurdity. Humor must have grown along with our abilities to criticize ourselves, starting with simple internal suppressors that evolved into more sophisticated censors. Perhaps they then split off into B-brain layers that became increasingly able to predict and manipulate what the older A-brains were about to do. At this point, our ancestors must have started to experience what humanists call conscience. For the first time, animals could start to reflect upon their own mental activities and evaluate their purposes, plans, and goals. This endowed us with great new mental powers but, at the same time, exposed us to new and different kinds of conceptual mistakes and inefficiencies.
Our humor-agencies become internalized in adult life as we learn to produce the same effects entirely inside our own minds. We no longer need the ridicule of those other people, once we can make ourselves ashamed by laughing silently at our own mistakes.
Some readers might object that the censor-learning theory of jokes is too narrow to be an explanation of humor in general. What of all the other roles that humor plays in occasions of enjoyment and companionship? Our answer is the same as usual: we can't expect any single, simple theory to explain adult psychology. To ask how humor works in a grown-up person is to ask how everything works in a grown-up person, since humor gets involved with so many other things. I didn't mean to suggest that every aspect of humor is involved in making censors learn. When humor evolved, as when any other mechanism develops in biology, it must have been built upon other mechanisms that already existed, and embodied mixtures of those other functions. Just as the voice is used for many social purposes, the mechanisms involved in humor are also used for other effects that are less involved with memory. In later life the effect of functional autonomy can make it hard to recognize the original function not only of humor, but of many other aspects of adult psychology. To understand how feelings work, we need to understand both their evolutionary and their individual histories.
We've seen how important it is for us to learn about mistakes. To keep from making old mistakes ourselves, we learn about them from our families and friends. But a peculiar problem arises when we tell another person that something is wrong, for if this is interpreted as an expression of disapproval and rejection, it can evoke a sense of pain and loss — and lead to withdrawal and avoidance. Accordingly, to point out mistakes to someone whose loyalty and love we want to keep, we must adopt some pleasant or conciliatory form. Thus humor has evolved its graciously disarming ways to do its basically distasteful job! You don't want the recipient to kill the messenger who brings bad news — especially when you're the messenger.
Many people seem genuinely surprised when shown that humor is so concerned with unpleasant, painful, and disgusting subjects. In a certain sense, there's really nothing humorous about most jokes — except, perhaps, in the skill and subtlety with which their dreadful content is disguised; frequently, the thought itself is little more than See what happened to somebody else; now, aren't you glad it wasn't you? In this sense most jokes are not actually frivolous at all but reflect the most serious of concerns. Why, by the way, are jokes usually less funny when heard again? Because the censors learn some more each time and prepare to act more quickly and effectively.