FEATURES

Seeing at the Speed of Sound

Lipreading, which makes one sense do the work of another, is a skill daunting to describe. Rachel Kolb, '12, deaf since birth, shares its mysteries.

March/April 2013

Reading time min

Seeing at the Speed of Sound

Illustration: Julia Breckenreid

I am sitting in my office during a summer internship. Absorbed by my computer screen, I do not notice when my manager enters the room, much less when he starts talking. Only when a sudden hand taps my shoulder do I jump. He is gazing expectantly at me.

"I'm sorry, I didn't hear you come in," I say.

"Oh, right." His expression changes: to surprise, and then to caution. He proceeds to say something that looks like, "Would you graawl blub blub vhoom mwarr hreet twizzolt, please?" I haven't the faintest idea what he said. I have no excuse, for I was looking straight at him. But despite my attention, something went wrong. He spoke too fast; my eyes lost focus.

"Um, could you repeat that, please?" I ask.

His eyebrows raise, but he nods and says it again. I sit up straighter, attempt to concentrate, but again it reaches my eyes as a garbled mess.

"It's fine," he answers. "I'll send you an email."

Well, at least I understood that part, I think as he walks out.

Lipreading, on which I rely for most social interaction, is an inherently tenuous mode of communication. It's essentially a skill of trying to grasp with one sense the information that was intended for another. When I watch people's lips, I am trying to learn something about sound when the eyes were not meant to hear.

Spoken words occur in my blind spot, a vacancy of my perception. But if I watch a certain way, I can bring them into enough focus to guess what they are. The brain, crafty as it is, fills in the missing information from my store of knowledge.

Want an example?

---- the ---- before --------- when ------------- the house

not --- cre --------------------- even ---- m------

Do you recognize the opening of "The Night Before Christmas"? Perhaps so, because in American culture the poem is familiar enough for one to fill in the blanks through memory. Filling in the blanks is the essence of lipreading, but the ability to decipher often depends on factors outside of my control.

It is my first week as a freshman at Stanford, and I feel lost. Instead of coasting through routine interactions with people familiar to me, I have thrown myself into a place where almost nothing is predictable. I sit down at a table of strangers. One of them, I realize, is the guy from the room next to mine. "What's your name?" I ask him.

He answers, but I frown.

"Could you say that again?" I say.

He does, but I still do not understand. The name starts with a B, and ends with a Y, but it is not a name I have seen before. Bobby, Barry, Buddy—none of them match what I saw on his face.

My neighbor, sensing my struggle, mumbles, "Just call me Ben."

Later that day I find out his name is Benamy.

Even the most skilled lipreaders in English, I have read, can discern an average of 30 percent of what is being said. I believe this figure to be true. There are people with whom I catch almost every word—people I know well, or who take care to speak at a reasonable rate, or whose faces are just easier on the eyes (for lack of a better phrase). But there are also people whom I cannot understand at all. On average, 30 percent is a reasonable number.

But 30 percent is also rather unreasonable. How does one have a meaningful conversation at 30 percent? It is like functioning at 30 percent of normal oxygen, or eating 30 percent of recommended calories—possible to subsist, but difficult to feel at your best and all but impossible to excel. Often I stick with contained discussion topics because they maximize the number of words I will understand. They make the conversation feel safe. "How are you?" "How's school?" "Did you have a nice night?" Because I can anticipate that the other person will say "Fine, how are you?" or "Good," I am at lower risk for communication failure.

My companions could be discussing any topic in the universe: the particulate nature of matter, the child who keeps wetting the bed, the villa in Nice that they visited last summer. And, because the human mind is naturally erratic in conversation, ever distractible, ever spontaneous, this is just what will end up happening. How am I to predict the unpredictable? The infinity of the universe, and of man's mind, strikes me as immensely beautiful—but also very frightening.

I don't like superficial remarks and predictable rejoinders, but staying in shallow waters is better than sinking. So long as I preserve my footing, I keep up the appearance of being able to converse—to other people and, more important, to myself.

"You know, you could be a spy," David, who lives in my dorm, tells me as we are sitting at brunch.

"Why do you say that?" I ask.

"Because"—he leans in excitedly—"because you could look through binoculars and lipread and understand everything people are saying!"

"Oh." I smile and cross my arms.

"Could you understand those people over there?" David points to a couple at another table. "Maybe," I say, without trying. I dare not explain that they're too far away.

The term "lipreading" implies that the skill is, in a sense, exactly like reading—in which the words on the page are clear and perfectly legible. "Can you read my lips?" strangers ask when they meet me. (Never mind that the question is inherently illogical: If I couldn't lipread, how on earth could I answer?) As they ask it, I can see the other, unspoken questions reeling in their heads—What if she can't? What will I do then? Mime?

When I answer that, yes, I can lipread, they relax. Then they prattle on as if all preconditions are off. Because I can "read" their lips, I must therefore be able to "read" everything they say. After all, it would be absurd for me to protest that I can sometimes read the words in a book, but sometimes not. Either you can read, or you can't. (Likewise, either you can hear perfectly—meaning hear and understand everything—or you can't hear at all. Forget hearing aids and microphones and other assistive devices.)

"How did you learn to lipread?" is another common query. I do not have a satisfactory answer. The truth is, I can't explain it. No more than I could explain how I learned to walk, or than anyone else could explain how she learned to hear and understand language. "Practice," I usually answer. Since I entered a mainstreamed public school in first grade, there have been no other deaf people occupying center stage in my life. My world is primarily a hearing one, and I learned to deal with this reality at a very young age. There was no reason to sign with anyone besides close friends and family, no reason to expect anyone to communicate on my terms. Surrounded by hearing people all the time, my only option has been to adapt, and lipreading is the skill that I have practiced most.

But this answer is too simple. The foundation for my success with communication was laid in my earliest years, at a deaf preschool. That was perhaps the only time in my life when I experienced full communication access each day. Everyone—students, teachers, speech therapists, parents, siblings—signed. From ages 2 to 5, I lived, breathed and conversed with people like me—at least, as alike as a young child understands. There was no reason for me to doubt myself or my abilities, so I grew fluent and confident with language. I learned its nuances, its facial and emotional expressions. I learned that it was not inaccessible, as it would sometimes later seem.

Self-confidence fuels the desire to practice and protects against the degradation of communication breakdown; but my ability to lipread is attributable not only to my own efforts, but also to the contributions of others. When I was less than a year old, my parents started me in speech therapy, which I continued for 18 years. There, I encountered the visual and physical fragments of the sound that was so absent from my world. This sound was mysterious to me. I could not grasp it—even with hearing aids—but I could see it. Under the tutelage of a succession of speech therapists, with support from my family, I became a student of its aftereffects.

In teaching me how to make sound's shapes with my own mouth, they taught me how to focus on their faces with the deepest intensity. Like a detective-in-training, I learned to recognize consonantal stops, the subtle visual differences between a "d" and a "g." (On the other hand, "p" and "b" are all but impossible to distinguish by lipreading alone, because their only difference is that one is voiced and one is not.) I learned how to zone in on the minutest changes in the muscles of the face. Over many years of drills and refinement, I learned how to construct the appearance of functioning like a hearing person. But I did not hear: I saw.

It is the first week of first grade, and the teacher has instructed us to line up by the door so we can follow her, duckling-like, to lunch. I do not know that she has asked us to line up in alphabetical order. My interpreter, who is usually around, seems to have disappeared. Satisfied to follow the other children, I take a spot in line and wait. Only then do I realize that my peers are talking, that they are rearranging themselves. I frown when the girl in front of me says something.

"Uh, what?" I say, not understanding her.

She says it again, to no avail.

"What?" I repeat, frustrated at the way the words brush off her lips and fly away.

She repeats herself. This time I understand that it is a question. Well, most questions are easily answerable with "yes" or "no."

I decide fast, "Yes." Surely a positive response will make the girl happy.

Instead, she frowns, and I realize I have said the wrong thing. Panicking, I tell her, "No," then, "Um, I don't know."

She giggles, as if I have said something funny, and whispers to a friend. Then she says it again—and everything clears in a rush. "What's your last name?"

As I answer, a cold surge rises in my chest. Without knowing it, I have made myself look too dumb to say my own name.

Sometimes I feel guilty that I lipread at all. I fear that I am betraying myself by accepting the conventions of the hearing world. I fear that I lack balance—that I am abandoning the communication tactics that work for me, in order to throw myself headlong at a system that does not care about my needs. When I attempt to function like a hearing person, am I not sacrificing my integrity to a game that I lack the tools to tackle, a game that in the end makes me look slow or stupid?

Deaf people—meaning Deaf people who live solely in the Deaf community, and hold on to an inherent pride in their Deafness—often speak of communicating as they please and letting the hearing world "deal with it." They believe in the beauty and, dare I say it, the superiority of sign language. Spoken language, compared with the visual nuances of signing, might as well be caveman guttural grunts.

When I lipread, I leave the clarity of sign language behind. I attempt to communicate with hearing people on their terms, with no expectation that they will return the favor. The standards I am striving for seem ridiculous: I am trying singlehandedly to cross the chasm of disability. Might not my stubbornness be of more harm than good?

I struggle with this. Some days I wonder what it would be like if I refused to speak. I could roll out of bed one morning, decide to take control of my communication on my terms, and make everyone write it down or sign, as other Deaf people do. Some days I resent myself. I wonder if I am weak, ashamed or overly anxious to please.

I am 12 and at a summer camp for the deaf. The entire group has just gone whitewater rafting and is stopping to get ice cream. My peers line up by the counter, signing to each other about the flavors they want. I smile and join, finding the conversation perfectly normal. But when the clerk speaks to us, the other kids freeze like mice after the shadow of a hawk has swooped over the grass.

With a jolt, I realize that they have no means with which to understand this hearing woman. Most do not speak, go to deaf schools, have never had reason to learn to lipread. Their barrier is the same as mine, but completely—instead of partially—insurmountable.

A younger Kolb is seen at summer camp. She is wearing a large hoodie and a baseball cap. She signs with her hands.HAPPY CAMPER: Summer camp was a place where everyone signed, but not everyone could lipread. (Photo: Courtesy Rachel Kolb)

"What did you say?" I ask the store attendant, looking her in the eye. My voice feels thick from disuse, but still I am aware of its clarity. The other kids stare at me, their hands slack.

"I said, would you like a free sample?" the attendant says. I understand her and sign the message to the others. They nod, and sign which flavors they want to taste. I repeat, speaking, to the attendant.

After the ordering, when I finally sit down, my own ice cream in hand, I feel strangely lightheaded. This—being able to endow spoken words with meaning, rather than having them translated by somebody else—is new for me. Because I have so often felt powerless, I have never realized the power that I possess.

What would I do, I wonder, if I could not lipread? How could I ever stand it?

Some people are all but impossible for me to lipread. People with thin lips; people who mumble; people who speak from the back of their throats; people with dead-fish, unexpressive faces; people who talk too fast; people who laugh a lot; tired people who slur their words; children with high, babyish voices; men with moustaches or beards; people with any sort of accent.

Accents are a visible tang on people's lips. Witnessing someone with an accent is like taking a sip of clear water only to find it tainted with something else. I startle and leap to attention. As I explore the strange taste, my brain puzzles itself trying to pinpoint exactly what it is and how I should respond. I dive into the unfamiliar contortions of the lips, trying to push my way to some intelligible meaning. Accented words pull against the gravity of my experience; like slime-glossed fish, they wriggle and leap out of my hands. Staring down at my fingers' muddy residue, my only choice is to shrug and cast out my line again.

Some people, though not inherently difficult to understand, make themselves that way. By viewing lipreading as a mysterious and complicated thing, they make the process harder. They over-enunciate, which distorts the lips like a funhouse mirror. Lips are naturally beautiful, especially when words float from them without thought; they ought never be contorted in this way. There are other signs, too: nervous gestures and exaggerated expressions, improvised sign language, a tic-like degree of smiling and nodding.

I sense that such people are terrified of not being understood. What they do not realize is that, when they are not at ease, I cannot be either. I am used to asking for repetition when I miss something, but if I do, such people will only freeze. In their minds, they have not tried hard enough. They turn this into a failing—instead of an unfortunate circumstance.

Encountering people who are nervous about lipreading gives me a strange complex. I wish only for them to be comfortable, not agitated or guilty. I want them to perceive me as more skilled, more normal, more approachable than they first thought. I do not want them to see me struggle. If I detect nervousness in a companion, I do my best to gloss it over—and present a semblance of normalcy, not the chaos I feel inside.

But despite its frustrations and misunderstandings, lipreading is sustenance for me. I once heard that prominent deaf educator Madan Vasishta said that he would rather have an incomplete conversation with a hearing person, one on one, than a conversation using a sign-language interpreter in which he understood everything. I take his point: The rawness of unfiltered contact surpasses even the reassurance provided by translation.

When the connection clicks, when I can read the curve and flow of a person's face, my ebullience soars. Our exchange is less like taking wild guesses at my own risk, and more like using the deftness of strategy and skill. I interact with hearing people as if I am one of their own. That they don't notice, don't remember that I am deaf! However unconscious, that is the greatest compliment of all.

Daniel is from Singapore. He speaks English, but his accent makes his syllables march in dizzying formations. To my eyes, his every utterance is bewildering.

Most people, once they figure out that I have such difficulty understanding them, stop trying. They feel the breakdown in the air, as I do, and they cannot tolerate its weight. But not Daniel. One day, he walks into my dorm room, says hi, and looks down to type on his cell phone. Thinking him sidetracked, I look out the window and wait. But soon he comes closer and shows me the screen.

How are you today? it says.

I grin. I want to leap up and hug him. "I'm fine," I announce. "How are you?"

He types: I'm pretty good. Sorry about my accent. I know it makes it hard.

"It's all right," I say. "I really wish I could understand you."

Daniel shrugs and smiles. How are your classes? Have you written anything new lately?

Anyone passing by in the hallway, hearing only my voice, would find this an odd, one-sided conversation. But, for me, it is perfect clarity.

Everyone has an Achilles heel, something that exposes her weaknesses. Mine is darkness. When it is dark, my appearance of communicative normalcy no longer stands. No speaker, no understanding can reach me. There is no way for me to penetrate any mind but my own, or to grasp whatever words other minds might exchange.

That sounds bleak, but it isn't really. With utter darkness comes resignation, a kind of peace. When it is completely dark, the responsibility for communication is no longer mine. Lipreading, writing, seeing: There is nothing more that I can do. I am free to retreat into the solace of my thoughts—which, in the end, is where I can feel most comfortable.

It's dim lighting, or bad visual aesthetics, that is a torment. When there is even the slightest sliver of light, there is still a chance. When lighting conditions are impractical or when I cannot squarely see the person who is talking, I still try. More often than not, I frustrate myself in the effort.

With lipreading, each day brings a moment in which I literally cannot do it anymore. I grow too tired of the guessing game that I can never quite win. The muscles behind my eyes ache from the strain. (Hearing is very different from sight, in that it does not involve muscular tension. I think of ears as very passive, whereas eyes are continuously moving to focus and see.) Often my corneas go dry; my vision gets blurry. The words on people's lips melt away, sliding down their faces like condensation on glass. I am back in the blind spot again.

The audiologist sits in the booth, where I see her face from my seat in a soundproof testing room. It is time for the tests I take every few years to monitor the ongoing status of my hearing, sometimes for official disability documentation. We have just finished a tone-recognition test, and now she will ask me to repeat back the sentences she reads. It is, of course, pointless to say that I will not be able to do it.

She places a piece of paper over her mouth, and I hear her voice as garbled noise, individual units barely distinguishable. I sit helpless, but once in a while take a guess. At most, I catch a word, or two. After nearly 40 sentences, I struggle to remain composed. This, such a simple exercise for anyone else, but for me —

I see her lower the paper from her face. My eyes latch onto her clear, articulate lips. "The bag of candy was on the shelf," she says.

"The bag of candy was on the shelf," I say, instantly smiling.

"The rabbit ran into the hole," she says.

"The rabbit ran into the hole."

We continue, then she wags her eyebrows and turns off her microphone. A new trick! "The mouse stole the cheese," she says, soundlessly. Any hearing person would spin into murkiness, but I can see, and that is enough.

"The mouse stole the cheese," I say, wanting to laugh.

Several more, almost perfectly, before she lays down her pencil. We gaze at each other. "You're amazing, you know that?" she says, and I glance down, letting my eyes take a rest. I smile and I smile.


Rachel Kolb, '12, is a graduate student in English from Albuquerque, N.M., and a Stanford intern.

Trending Stories

  1. 8 Tips for Forgiving Someone Who Hurt You

    Advice & Insights

  2. How to Find Your People

    Advice & Insights

  3. Should We Abolish the Electoral College?

    Law/Public Policy/Politics

  4. Bananas Are Berries?

    Culture

  5. The Huberman Effect

    Science

You May Also Like

© Stanford University. Stanford, California 94305.