The Depths of the Uncanny Valley
GameSpot investigates how video games really impact our lives--how they are changing our culture, what they're doing to our brains, and what this all means for people who play video games.
ou play video games, but do you have any idea how those video games are affecting you? They do, whether it's the subconscious way in which violence triggers chemicals in your brain, or your automatic repulsion at a game character that doesn't look quite human. With these ongoing reports, GameSpot will investigate how video games really impact our lives--how they are changing our culture, what they're doing to our brains, and what this all means for people who play video games.
Becoming uncanny
In 1970, roboticist Masahiro Mori published an article in the Japanese magazine Energy titled "Bukimi No Tani" (English translation: "The Uncanny Valley") detailing an observation that he had made from his experience working with humanlike robots. Mori noted that the more closely robots approximated human appearance and behavior, the more familiar they seemed to a human observer, until a point at which they resembled humans closely, but not perfectly. At this point, people would begin to react negatively to robots, citing feelings of eeriness or discomfort about their appearance. He called it "the uncanny valley," because of the way a graph depicting the correlation between familiarity and human likeness would dip suddenly and drastically, just before reaching perfect mimicry of the human appearance.
As technology improves and entertainment media such as movies and video games are able to more closely approximate realism, humanoid characters get dangerously close to what Mori described. For example, take a look at some of the most negative reviews that the motion picture The Polar Express received. The film attempted to create a highly realistic look through computer-generated imagery (CGI), but missed a few key points, such as the depiction of the characters' eyes and skin, which made them seem more doll-like than human, causing an involuntary repulsion among viewers.
So how does the uncanny valley work? Is it science? What are the factors that contribute to uncanniness--this unusual quality of realistic-looking characters that can seem so discomforting when we see them in action? Mori's observation may have opened up the discussion about this concept, but it left most of these questions unanswered. Only recently have scientists and roboticists begun to uncover the methodology and reasoning behind the uncanny valley, and in turn, how we can circumvent it.
Getting into the uncanny valley
"If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart." - Dr. Karl MacDorman, School of Informatics at Indiana University
Karl MacDorman is an associate professor in the human-computer interaction program at the School of Informatics at Indiana University and has been working with the uncanny valley hypothesis, using human participants, for the past year. In his own research on the subject, MacDorman has found scientific support for the hypothesis, which has revealed, among other things, that there are too many contributing factors for it to be narrowed down to a single theory. "If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart. This in turn is uncanny for a different reason than if part of its body is open, exposing wires and motors. I have identified about 10 possible causes for the uncanny valley, and I am sure there are many more."
The sheer number of factors and the precision of detail required to perfectly approximate humans makes the process seem extraordinarily daunting, though MacDorman believes that the most difficult of these problems is not in creating proportionately accurate humans, but in the timing and the interactivity of character movement: "I suspect the hard part will not be in finding the right physical proportions, but making the movements seem natural and well-timed, especially during interaction." This presents an extraordinarily poignant problem for video games, in which the player has control over the characters for the majority of the time. The characters must not only look realistic and animate accurately, but they must also react to control with perfect timing.
What makes this uncanny?
Of course, there are still plenty of problems within controlled situations. One such example of this can be seen in developer Quantic Dream's recent tech demo for its game Heavy Rain (working title), which has come under fire for uncanniness since it was first revealed at E3 2006. The trailer, which depicts an extremely realistic female character performing a monologue for the camera, is technologically impressive, yet many people have responded to it in a negative way, stating that there's something about it that is unsettling.
MacDorman lists several factors behind what makes the principal character of the video, Mary Smith, seem unnatural, although he recognizes that the video displays great technique nonetheless. "Presumably to enhance realism or reduce calculation, the game designers or animators exaggerate the depth of field, so that Mary's face is rendered in focus, but her hair and neck are out of focus and blurry. In addition, there is sometimes a lack of synchronization with her speech and lip movements, which is very disturbing to people. People 'hear' with their eyes as well as their ears. By this, I mean that if you play an identical sound while looking at a person's lips, the lip movements can cause you to hear the sound differently." He further cites her pale complexion and the fact that she behaves as a sociopath as reasons that most people will have trouble relating to her. "In general, sociopaths tend to seem odder than ordinary people--not only in their behavior, but in their facial asymmetries, which reflect developmental disorders--as anyone can tell by surfing Web sites that list photographs of known sexual offenders." This list of factors shows the breadth and depth of the uncanny valley problem, in that it goes far beyond what seems obvious (something like the lip synching) and into the more nuanced (the difference of the face structure of sociopaths).
"And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes." - Dr. Karl MacDorman, School of Informatics at Indiana University
Posing an even deeper problem for developers, MacDorman believes that animators working from inception to completion will have difficulty witnessing uncanniness in their own projects. "They build up characters that start off looking not particularly realistic and, therefore, not particularly uncanny. And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes. As artists, animators must always rely on their own sense of aesthetics. The problem is that they have lost what is 'common sense' to the rest of us." According to Elspeth Tory, the Animation Project Manager on Ubisoft's upcoming game Assassin's Creed, that's why it's absolutely critical that animators not work on their games in isolation. "You certainly start to get used to a character after a while and can occasionally lose a bit of the objectivity that you had at the beginning of the process. Getting feedback from others, especially the artistic director for animation, is an essential part of the pipeline in order to maintain a standard of believability."
Dealing with uncanniness
But it takes more than just feedback to get past the uncanny valley. Animators must deal with the phenomenon head-on, working to combat it from the beginning, not only by creating realistic-looking humans and animating them well, but also by making sure that the level of realism present is both believable and fun. Tory cites the uncanny valley as a hurdle in achieving realism with Assassin's Creed but distinguishes reality from believability, which is an equally if not more important goal. "Animating something that's believable gives you a bit of room for texturing a movement than simply making something that's realistic. A character that's realistic will seem to have ticked off a checklist of human characteristics, but a believable one will display nuances and subtleties that make them seem unique and alive." Tory notes that weight and timing are some of the most important aspects that contribute to a character's believability, and that hands are the most difficult part of the human body to animate.
Ubisoft has a good track record for believability, particularly with Prince of Persia: The Sands of Time, which was critically lauded for its excellence in animation. Alex Drouin, the Animation Artistic Director on both Sands of Time and Assassin's Creed, reveals some of the tricks they used to achieve such a full range of fluid animations. "Everything was hand animated in Prince of Persia: Sands of Time. We used a lot of 'blending' of animations to get specific effects and a wider range of possibilities, and different kinds of interpolations to fluidly link different moves together. We used the same tools for Assassin's Creed but we added full body IK (inverse kinematics) and a rag doll tool." These tools helped the Ubisoft animators create a game that looked both believable and realistic, without getting bogged down in the kind of realism that would make the game unenjoyable. Focusing on believability helped them to achieve their animation goals with Sands of Time, and the result was a very attractive and well-animated game.
Out of the valley...for good?
"We've put a lot more animation files in the game... More than anything I've seen before." - Alex Drouin, Animation Artistic Director on Assassin's Creed
So how does the future of research and technology help to resolve the problems of the uncanny valley once and for all? MacDorman hope his research will lay the groundwork for combating the uncanny valley, by narrowing down all of the factors that contribute to it and giving us a better understanding of all of the nuances that control our perception of a realistic human. "We know that the human brain doesn't even register many kinds of gaps in visual information. We don't notice that we have a blind spot. We don't notice when we blink. We don't notice floaters, particles of dust moving on the cornea. There would be no point in filling in details that we don't notice. Therefore, we need to identify what we do notice, and to fix that part, and this may not so much require new technologies as an improved application of existing technologies." But having new technology will help developers to work with our limited knowledge of these factors in the short term. Drouin says that the increased power of the PlayStation 3 has helped them to create more diversity in the Assassin's Creed character animations, adding to the believability of the game. "We've put a lot more animation files in the game... More than anything I've seen before. Therefore, variety is going to help us re-create a living and breathing world because our eyes are not used to seeing patterns in real-life movements." It is the combination of these two factors, research and technology, that, when applied correctly, will begin to strip away the problem of the uncanny valley for good.
So where does this all leave us? With a lot of work to do, it seems, but MacDorman's view is optimistic. "It is something that can be overcome through good design. By manipulating the many factors that influence whether a robot or game character is familiar or eerie, we can design around the uncanny valley for any degree of human likeness. As we come to better understand the norms of human interaction, I believe our androids will overcome the uncanny valley." Developers are showing that they are able to work around the problems that the uncanny valley presents, but as games get more and more complex, there is always more work to do. Getting more involved with robotics research, using advanced technology better, and continuing to animate the details that we notice over those that we don't will all contribute to a positive future for video game animation. In the end, though, it seems as though they've got a good grasp of what really matters. Says Drouin: "So is our game completely realistic? NO WAY... It's realistically fun!"
Previously: Empathy and Conditioning Violence
What do you think?
Share your thoughts on the article, discuss the topics with your fellow users, or sound off about the topic of the uncanny valley right here. Please keep your conversation on topic and your tone civil.
Got a news tip or want to contact us directly? Email news@gamespot.com
Join the conversation