The Depths of the Uncanny Valley

GameSpot investigates how video games really impact our lives--how they are changing our culture, what they're doing to our brains, and what this all means for people who play video games.

by

By Carrie Gouskos- posted July 8, 2006

ou play video games, but do you have any idea how those video games are affecting you? They do, whether it's the subconscious way in which violence triggers chemicals in your brain, or your automatic repulsion at a game character that doesn't look quite human. With these ongoing reports, GameSpot will investigate how video games really impact our lives--how they are changing our culture, what they're doing to our brains, and what this all means for people who play video games.

Becoming uncanny

Image of the Uncanny Valley, courtesy of Karl MacDorman

In 1970, roboticist Masahiro Mori published an article in the Japanese magazine Energy titled "Bukimi No Tani" (English translation: "The Uncanny Valley") detailing an observation that he had made from his experience working with humanlike robots. Mori noted that the more closely robots approximated human appearance and behavior, the more familiar they seemed to a human observer, until a point at which they resembled humans closely, but not perfectly. At this point, people would begin to react negatively to robots, citing feelings of eeriness or discomfort about their appearance. He called it "the uncanny valley," because of the way a graph depicting the correlation between familiarity and human likeness would dip suddenly and drastically, just before reaching perfect mimicry of the human appearance.

As technology improves and entertainment media such as movies and video games are able to more closely approximate realism, humanoid characters get dangerously close to what Mori described. For example, take a look at some of the most negative reviews that the motion picture The Polar Express received. The film attempted to create a highly realistic look through computer-generated imagery (CGI), but missed a few key points, such as the depiction of the characters' eyes and skin, which made them seem more doll-like than human, causing an involuntary repulsion among viewers.

So how does the uncanny valley work? Is it science? What are the factors that contribute to uncanniness--this unusual quality of realistic-looking characters that can seem so discomforting when we see them in action? Mori's observation may have opened up the discussion about this concept, but it left most of these questions unanswered. Only recently have scientists and roboticists begun to uncover the methodology and reasoning behind the uncanny valley, and in turn, how we can circumvent it.

Getting into the uncanny valley

"If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart." - Dr. Karl MacDorman, School of Informatics at Indiana University

Karl MacDorman is an associate professor in the human-computer interaction program at the School of Informatics at Indiana University and has been working with the uncanny valley hypothesis, using human participants, for the past year. In his own research on the subject, MacDorman has found scientific support for the hypothesis, which has revealed, among other things, that there are too many contributing factors for it to be narrowed down to a single theory. "If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart. This in turn is uncanny for a different reason than if part of its body is open, exposing wires and motors. I have identified about 10 possible causes for the uncanny valley, and I am sure there are many more."

The sheer number of factors and the precision of detail required to perfectly approximate humans makes the process seem extraordinarily daunting, though MacDorman believes that the most difficult of these problems is not in creating proportionately accurate humans, but in the timing and the interactivity of character movement: "I suspect the hard part will not be in finding the right physical proportions, but making the movements seem natural and well-timed, especially during interaction." This presents an extraordinarily poignant problem for video games, in which the player has control over the characters for the majority of the time. The characters must not only look realistic and animate accurately, but they must also react to control with perfect timing.

What makes this uncanny?

This character may look realistic, but there are many factors which make her also seem uncanny.

Of course, there are still plenty of problems within controlled situations. One such example of this can be seen in developer Quantic Dream's recent tech demo for its game Heavy Rain (working title), which has come under fire for uncanniness since it was first revealed at E3 2006. The trailer, which depicts an extremely realistic female character performing a monologue for the camera, is technologically impressive, yet many people have responded to it in a negative way, stating that there's something about it that is unsettling.

MacDorman lists several factors behind what makes the principal character of the video, Mary Smith, seem unnatural, although he recognizes that the video displays great technique nonetheless. "Presumably to enhance realism or reduce calculation, the game designers or animators exaggerate the depth of field, so that Mary's face is rendered in focus, but her hair and neck are out of focus and blurry. In addition, there is sometimes a lack of synchronization with her speech and lip movements, which is very disturbing to people. People 'hear' with their eyes as well as their ears. By this, I mean that if you play an identical sound while looking at a person's lips, the lip movements can cause you to hear the sound differently." He further cites her pale complexion and the fact that she behaves as a sociopath as reasons that most people will have trouble relating to her. "In general, sociopaths tend to seem odder than ordinary people--not only in their behavior, but in their facial asymmetries, which reflect developmental disorders--as anyone can tell by surfing Web sites that list photographs of known sexual offenders." This list of factors shows the breadth and depth of the uncanny valley problem, in that it goes far beyond what seems obvious (something like the lip synching) and into the more nuanced (the difference of the face structure of sociopaths).

"And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes." - Dr. Karl MacDorman, School of Informatics at Indiana University

Posing an even deeper problem for developers, MacDorman believes that animators working from inception to completion will have difficulty witnessing uncanniness in their own projects. "They build up characters that start off looking not particularly realistic and, therefore, not particularly uncanny. And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes. As artists, animators must always rely on their own sense of aesthetics. The problem is that they have lost what is 'common sense' to the rest of us." According to Elspeth Tory, the Animation Project Manager on Ubisoft's upcoming game Assassin's Creed, that's why it's absolutely critical that animators not work on their games in isolation. "You certainly start to get used to a character after a while and can occasionally lose a bit of the objectivity that you had at the beginning of the process. Getting feedback from others, especially the artistic director for animation, is an essential part of the pipeline in order to maintain a standard of believability."

Dealing with uncanniness

But it takes more than just feedback to get past the uncanny valley. Animators must deal with the phenomenon head-on, working to combat it from the beginning, not only by creating realistic-looking humans and animating them well, but also by making sure that the level of realism present is both believable and fun. Tory cites the uncanny valley as a hurdle in achieving realism with Assassin's Creed but distinguishes reality from believability, which is an equally if not more important goal. "Animating something that's believable gives you a bit of room for texturing a movement than simply making something that's realistic. A character that's realistic will seem to have ticked off a checklist of human characteristics, but a believable one will display nuances and subtleties that make them seem unique and alive." Tory notes that weight and timing are some of the most important aspects that contribute to a character's believability, and that hands are the most difficult part of the human body to animate.

Ubisoft animators aim for both believability and realism in games like Assassin's Creed.

Ubisoft has a good track record for believability, particularly with Prince of Persia: The Sands of Time, which was critically lauded for its excellence in animation. Alex Drouin, the Animation Artistic Director on both Sands of Time and Assassin's Creed, reveals some of the tricks they used to achieve such a full range of fluid animations. "Everything was hand animated in Prince of Persia: Sands of Time. We used a lot of 'blending' of animations to get specific effects and a wider range of possibilities, and different kinds of interpolations to fluidly link different moves together. We used the same tools for Assassin's Creed but we added full body IK (inverse kinematics) and a rag doll tool." These tools helped the Ubisoft animators create a game that looked both believable and realistic, without getting bogged down in the kind of realism that would make the game unenjoyable. Focusing on believability helped them to achieve their animation goals with Sands of Time, and the result was a very attractive and well-animated game.

Out of the valley...for good?

"We've put a lot more animation files in the game... More than anything I've seen before." - Alex Drouin, Animation Artistic Director on Assassin's Creed

So how does the future of research and technology help to resolve the problems of the uncanny valley once and for all? MacDorman hope his research will lay the groundwork for combating the uncanny valley, by narrowing down all of the factors that contribute to it and giving us a better understanding of all of the nuances that control our perception of a realistic human. "We know that the human brain doesn't even register many kinds of gaps in visual information. We don't notice that we have a blind spot. We don't notice when we blink. We don't notice floaters, particles of dust moving on the cornea. There would be no point in filling in details that we don't notice. Therefore, we need to identify what we do notice, and to fix that part, and this may not so much require new technologies as an improved application of existing technologies." But having new technology will help developers to work with our limited knowledge of these factors in the short term. Drouin says that the increased power of the PlayStation 3 has helped them to create more diversity in the Assassin's Creed character animations, adding to the believability of the game. "We've put a lot more animation files in the game... More than anything I've seen before. Therefore, variety is going to help us re-create a living and breathing world because our eyes are not used to seeing patterns in real-life movements." It is the combination of these two factors, research and technology, that, when applied correctly, will begin to strip away the problem of the uncanny valley for good.

So where does this all leave us? With a lot of work to do, it seems, but MacDorman's view is optimistic. "It is something that can be overcome through good design. By manipulating the many factors that influence whether a robot or game character is familiar or eerie, we can design around the uncanny valley for any degree of human likeness. As we come to better understand the norms of human interaction, I believe our androids will overcome the uncanny valley." Developers are showing that they are able to work around the problems that the uncanny valley presents, but as games get more and more complex, there is always more work to do. Getting more involved with robotics research, using advanced technology better, and continuing to animate the details that we notice over those that we don't will all contribute to a positive future for video game animation. In the end, though, it seems as though they've got a good grasp of what really matters. Says Drouin: "So is our game completely realistic? NO WAY... It's realistically fun!"

Previously: Empathy and Conditioning Violence

What do you think?

Share your thoughts on the article, discuss the topics with your fellow users, or sound off about the topic of the uncanny valley right here. Please keep your conversation on topic and your tone civil.


Discussion

350 comments
glitchgeeman
glitchgeeman

Rather creepy I have to admit, which is exactly why I love games like Wind Waker, No More Heroes, and Okami. The more we move towards realism, the harder it will be to create a unique artstyle. So yeah, bring on the realistic games, but at the same time, I'd like to see all that future hardware power be used to create some beautifully unique and artistic games.

xatman911
xatman911

Similar problem with anime style. It has to be realistic like FF to believable. lol.

zgreenwell
zgreenwell

Or you could avoid the uncanny valley by not moving towards more realistic characters. There is always that option too.

Turisman4
Turisman4

After watching a round of the new DEFCON, this actually starts to make sense.

Necrotide51
Necrotide51

It almost seems that most games, in terms of graphical and animated quality, are aimed at eventually becomming life simmulations, giving you so many options, they become more similar to each other. As games become more and more real looking, the ammount of ways an artist can give it a unique style also diminishes.

Newager
Newager

Realism cannot be made simply because we have technology. No matter how advance the technology in the future will be it still lacks one thing. We can see it from the games we play today. I think this lackness is heart. Characters in the games are made to be near perfect because people wanted to pass down to the next generations or those who play the game of what it means to be human. Definition of human is too wide if we add some of the aspects that humans have. Such as culture, race, skin color and everything we can think of to make human different from each other. Therefore, the animator tried their best to produce a motion of a digital character which are not human, to be like human. To create an imagery of what a human is. It is all just the means of passing down our culture from generation to generation and put the values in the heart. Humans are product of culture which means we learn one thing from another who pass the knowledge to us. It is all regarding how the culture is made. Every living being wanted to pass on their genes, culture, language... through poetry, music, joy, story... (Metal Gear Solid 2: Sons of Liberty) So do we, youth of today. We play video game because this is our culture, we live with it. This is how we interact with technology and what, in return, technology give us in our lives.

KGB002
KGB002

yeah I have noteced lately that vido game char. are way too... perfect. Humans arnt compleatly perfect... We dont all have smooth glowing skin, our nails are not that even,our hair isint shiny all the time, we all have some kind of scrape or scar and we dont move so smoothly all the time.

icey252
icey252

This amazing realisim is great and all, but I don't know about you peoples. I LIKE it when the game is obvious that it is not trying to be perfectly human. Unless itt is just plain scary.

Doomlike_Mitc
Doomlike_Mitc

The doctor dude is right, Heavy rain isnt realistic, I saw this at the beginning, her body her head, it just didnt moved right... the future will be awsom, can you imagine movies to be made by pc's only and no more actors are needet... sweet and imagine that in a game, the game realisme these days blows people away I think we are not long away from movie/video game games, when both are mixed and both have no difference in quality....

shady_825
shady_825

Zoo3, I felt the same way about that movie. What's interesting to me, however, is that I never felt that "uncanniness" watching Advent Children, which was also a Square-Enix CG. Does this mean that they've somehow reached a relatively perfect level of animation? As far as I know, this is the only piece of CG entertainment I've ever seen that doesn't give me the feeling I had watching things like the Heavy Rain trailer or the cutscenes of Hitman: Blood Money.

Sanguis_Malus
Sanguis_Malus

Good article, informative, intelligent and insightful. Top work Gamespot !

360guy
360guy

realism is a good thing, that allows people who play video games to indluence others to join the team thus allowing Gamers to have complete control over society, by fighting back with technology. All thats left after that are those like the U.S. Congress who hate videogames to be demolished and a new rule will be established hahaha

Hanz_Moleman3
Hanz_Moleman3

This "Uncanny Valley" is probably the reason why I hate final fantasy. The characters look human but a bit too human if you know what I mean.

twinheadrex1
twinheadrex1

nick1500 "I am waiting for the day tha games become so life like and our play station 56's and xbox 960 (in the future of course) take over our lives and dominate us for the end of time." Itl probly be around PS5 that that happens

vc152007
vc152007

hey if its a fun game i dont care about awesome detail

nick1500
nick1500

I am waiting for the day tha games become so life like and our play station 56's and xbox 960 (in the future of course) take over our lives and dominate us for the end of time.

zoo3
zoo3

final fantasy spirits within is a great example of this so called uncanniness. now there was a movie that had many photorealistic shots. but i was pulled out of the movie to reality whenever i watched a segment where an animator had given one of the characters a "thunderbirds" look. anyone remember that show? the way the FF characters would move and talk was as if they were puppets on strings. but when they were doing scenery shots or mocap, the movie looked great. this uncanniness is even more noticeable when you're an animator. ignorance is bliss in some areas of life.

JaMeS4418
JaMeS4418

There is a book about this called why video games are good for you.

haqerj
haqerj

I think this is the reason I hated the legend of zelda wind waker and just couldn't get in to it. well beside the whole sailing thing

harrisonm7
harrisonm7

The Heavy Rain demo is not a good example of the uncanny valley, the model is not detailed enough, the mouth area looks horrendous, the surroundings are not photorealistic, the lip synch is awful and there is just not enough facial movement in general. For some better examples, look at this site : http://www.softimage.com/products/face_robot/video_gallery/Default.aspx . Face Robot demos are really uncanny valley stuff.

Reap3r666
Reap3r666

Oh. My friggin' head! I'm so Wasted! What was the qustion? Intresting points GS. More to follow I hope.

mac_ronin
mac_ronin

great job carrie a real great job makes me wonder tho how big was the original version of it? haha in any case i luved it

neogeon
neogeon

oh come, are you guys serious? this is such a load, that video doesn't "freak me out" in the least, (hard rain that is) it annoys me because her jaw moves like she has down syndrome X10 and her lips arent synched for crap. The word all of you are looking for is not "uncomfortable" its "agitated" with the half-assed animation coupled to an otherwise well made polygon model. As for people saying stuff like "omg, think of grand theft auto if it looked 100% real, i couldn't play that," how is that any different from watching a cop movie with lots of violence? a cop movie actually has REAL people even though the violence is fake so it will always be more shocking than a 100% fake polygon simulation. To address another off the wall statement made about a "sims 9000" where technology is so advanced the sims could form a "consciousness" of sorts, artificial intelligence does exist and certainly can evolve, but a computer is nothing more than an advanced calculator. Emotions, etc. are beyond the grasp of a computer because it has no instinct or hormones like humans (and im not talking sex hormones, get your mind out of the gutter) and the bottom line is that it can only do what its programmed to do or adapt in a very systematic fashion based on its program. A real life Terminator would only be possible if some jackass programmed robots to kill people, same with the matrix. I appreciate the off color nature of this article but I think the results on the graph are more a result of the power of intellectual persuasion than a testament to biological and chemical reaction of human beings. Also, modern psychology has pretty much proven a correlation between ignorance and the belief in the supernatural (voodoo, magic, etc) and any uneasiness would come more from not understanding then technology rather than any real justified discomfort. For example, I have heard of primitive villages in africa confiscating cameras because they believed that the film/photograph trapped the individual's souls in the photograph (which anyone that hasn't been living in a hole for 200 years knows thats ridiculous). The same can be seen when a child sees a magic show (and hasn't been told its fake) and a reputable college professor sees the same show. The child is bewildered at the unexplainable mysteries while the college professor is anylyzing the slight of hand to discover the inner workings of the trick. The bottom line here is the only reason you should get freaked out by semi-real computer generated animation is if you have a paranoia of supernatural misfortunes (being eaten by a zombie, being eaten by a down syndrome housewife with a gun and horse teeth, etc).

ranger_waha
ranger_waha

Great article gs, way to pick an odd topic with obscure rootings, and make it relevant and cool.

ThundertmaerS
ThundertmaerS

so THAT's why I hated the final fantasy movie! great article Gamespot!

UzEE
UzEE

when i saw those shots of Heavy Rain, I know something was odd, something is missing. I cant point my finger on it but there is something. Thanx to this article, i got a lot better understanding of the issue

STEALTH5000
STEALTH5000

Very Interesting Article. Keep publishing this articles to keep us busy reading. It's really good.

grognard
grognard

It's articles like this one that set Gamespot apart from the crowd. Carrie's a pro and now I don't feel so bad paying a premium for this site. :)

anyulled
anyulled

wow, man, i certainly didn't know all that stuff, great article

T-bone123
T-bone123

I don't video games do anything. Its just like watching T.V....I hope

rwrden11
rwrden11

Video games, both the cause and solution to all of life's problems...

ffgothic
ffgothic

Interesting. very interesting

redbeef88
redbeef88

Great read, keep these coming! :D

Rayne-
Rayne-

Yea. Well, on reading the brief summary on the front page, I was kind of expecting something like the Economist article we saw around last year, dealing with CULTURE, and not mind synapses and reactions to lifelike CGIs. You didnt really deliver, and I dont find this all too interesting. I would really enjoy reading an article on Gamespot dealing with and discussing the escalating number of cases of people losing their jobs, dying in front of their computers, and throwing themselves out of windows because they think it's a portal into Azeroth. I love video games, and always have, but I still find it disturbing reading articles on 14 year old kids locking themselves in their rooms and playing Starcraft, WoW, what have you. There are sometimes MASSIVE ramifications with video games, including Addiction, and Anti-Social tendencies being fostered. I love what you guys do on the site, and, even though I havnt bought a new game for more than a year now, I still am an avid listener of the Hotspot and watcher of On The Spot. Thanks, and If you could respond, I'd love you for the rest of my life. Much Obliged. Osman.

ace1580
ace1580

It reminds me of how the Final Fantasy movie never took off. The female protagonist in that move was incredibly well animated, but everyone who watched it could just tell that something, that no one could quite point out, was not quite right.

GuttersnipeMav
GuttersnipeMav

Wow, I am impressed with the level of cerebrality that this article attained. I've always wondered why the woman in the Heavy Rain demo put me off, and I certainly expected the unsynchronized lip-movements and sounds coming from her mouth. I was surprised to see the character the woman is playing taken into account. Obviously, the scene is meant to be unsettling, even upsetting, but it's interesting to look at it in a way that's not purely emotional, but logical and statistical. Very well done, an enjoyable read, and very informative.

ethanraiden
ethanraiden

I see that people have pretty much finished with posting here, but I have to say that that was the greatest read ever to grace Gamespots pages.

rynmls
rynmls

video games created a world for gamers that they cannot achieve in real world, just like watching a movie, you can feel their happiness and pain, now that's a good quality game if the gamer are being attached to what they are playing...

Virtuson
Virtuson

WOOT WOOT!! PINK!! no but seriously...after thought i begin to remember moments like that, especially after watching the heavy rain thing.

xSpitFire3x
xSpitFire3x

All i know is video games have helped relax me and really bring out my creativity and narritive ablities. Games kick total ass and are WAY more helpful than people think!

lvlUNE3B
lvlUNE3B

hmm... they do effect if games become a major part in your life.

littlewiz77
littlewiz77

I thought, when watching ninja gaiden, during the train scene, ryu's eye looked very realistic...

Gleba2388
Gleba2388

I think that the woman's face in Heavy Rain is truly realistic, and Jean Reno's face in Onimusha 3 also was well done... so there can be good effort for some unbeliveably realistic graphics... just a matter of hard work...

i_love_my_ds
i_love_my_ds

It can never be perfect, so dont even try

MilesTormani
MilesTormani

Bleh. All I can say is that this is a big load of spoony. There's too many possible variables for it to be anywhere near scientific, so it just comes off as a 'straw hat' generalization. Then again, I never really got that weird feeling. I always just got the thought of 'Oh. That's not quite right in the graphics. It sticks out because everything else looks good.' Seen this in Oblivion more than enough to know I don't say 'omg teh errie lol'. Then again, I'm a single person, who tends to think. And, well, I don't really CARE that much, either.

yugimasa9999
yugimasa9999

This would inspire animators out there to think outside of the box. On the case of the theory...Its simply relevant to the gaming industry today as the next generation of gaming is starting already.