Nvidia unveils next-gen face-rendering technology

FaceWorks promises to create near photo-realistic facial expressions, lighting, and speech.

A new facial rendering technology that promises to push graphics past the uncanny valley and into the realms of photorealism has been unveiled by GPU-maker Nvidia. Dubbed FaceWorks, the technology renders subtle facial expressions, lighting, and details such as pores and wrinkles to create a more realistic-looking model.

FaceWorks is impressive, but don't expect to see it in games anytime soon.

The first results of the technology were demonstrated at the GPU Technology Conference in San Jose, California. A model of a male face called Ira was made to perform several different facial expressions against a range of different lighting setups. Ira was also made to speak, screaming "take my money!" when asked about Nvidia's portable Project Shield computer.

FaceWorks is based upon technology developed by the Institute for Creative Technology, which uses over 150 cameras arranged in a spherical space to capture 30 expressions of the human face. That data is then condensed from its original 30GB size to around 300MB and rendered in realtime.

While the technology is certainly impressive, it's not likely to be coming to video games any time soon. Despite shrinking the size of data, actually rendering Ira at 1080p resolution requires over 2 teraflops (2 trillion floating-point operations per second) of graphics processing power. That's around half the processing power of Nvidia's latest Titan GPU, which tops out at 4.5 teraflops, and costs around $1000 (£800).

By comparison, the PlayStation 3 generates a mere 230 gigaflops, while even the upcoming PlayStation 4 tops out at 1.84 teraflops. All is not lost for fans of high-res rendering, though. At the PS4 press conference earlier this month, Quantic Dream head David Cage demonstrated its own similarly impressive face-rendering, showing that such technology may yet make its way into next-gen games.

Written By

Discussion

297 comments
craigprime
craigprime

I must say, that face look's quite beautiful. Makes you wonder how close to the end we are in Graphical improvement. And when it reaches the end, the Xbox is doomed. The console after the 720 will be there last, assuming it would use this sort  of graphical capability.

zenogandia
zenogandia

Like I said before, it's not impossible to make these visuals for next generation systems, but the price of making a game like this would be insane. The technology to speed the development process is simply not there, hence it's way too costly to make. 

placksheep
placksheep

I made it halfway through :-) Both of these guys LOVE to talk. Good interview!

gatsbythepig
gatsbythepig

I learned about the uncanny valley over a decade ago.  I have yet to see a cgi model or mannequin that puts me in a position to experience the awful,  repulsive uncanny valley.  I'm beginning to think it doesn't exist for me(or at all).  Or, we haven't even come close to reaching the 'uncanny valley' yet.  I think the phrase is just really popular because it is easier to remember, since it rhymes.

Goemons
Goemons

They allways have trouble with the EYES and still it shows in this video no matter how detailed the skin looks the eyes dont look right Nvidia.

Hurvl
Hurvl

Small image makes me think of Kevin Spacey as Lex Luthor.

kurtven
kurtven

I have never seen anything from nvidias Demos in any game to yet

coolguyhero
coolguyhero

@Metronoid :you need good games high graphics in cheap rate how is this possible?.high graphics involves high cost but still ps4 did a good job.i will give you one example then you will get it .ps3 having 512 mb ram....256-256 share using that naughty dog created unchantered .now ps4 having 8 gb ram and capcom already created next gen game using  pentarey engine...imagine it just start when it goes further more beautiful game will come.here remember one thing games with good graphics can be achieved by various ways.just need to research..and ps3 already proved that.may be such graphics cant be achieved by ps4 but i am sure atleast it can touch below that...& when you need such graphics you need the price 1500$...& ps4 coming near 500 $ guess...and there is wrong feeling about ps4...sony made adeal with amd for coustom chips and memory which is not equal to individual technology & also purchassed in the mass ..obivously  amd must gave them a high quality product.just trust them & one day you will remeber my word.no offence just some points want to share and i shared it...thanks.

Metronoid
Metronoid

Maybe in PS6 you will be able to see this in games.

PipeBigboss
PipeBigboss

There is a nice improve in the lighting mapping but I don't know if it would be cheap for and end user to get

Mayleene
Mayleene

Who cares?  It's a game.  Just a video game.

erix43
erix43

Everything but the eyes are pretty nice. I don't like the eyes, unless he's been crying the whole time. 

warhawk-geeby
warhawk-geeby

It's the eyes.. alllllways the eyes.

Doesn't matter how realistic someone can make a face look the eyes always let it down.. They're like glass balls (not just in this presentation but many).

They're too clean.. and reflective.  You don't get the sense of 'character' that you do when you stare a real individual in the eyes, artificially rendered ones are always lacking true emotion.

princetom
princetom

HA HA HA HA HA!!! this is laughable, they call this next gen faces? I've seen what square enix has made for ps4 and this shit doesn't even come to their tech. I can honestly say that even the fox engine will surpass this once MGS ground zeroes comes out. we have L.A. Noire, that has one of the most realistic facial animations in history, the only problem with making it perfect is perfecting the skin of character, because they still look like animated people. plus look at this video, this is what i would call next gen animation: 

http://www.youtube.com/watch?v=fBR4cT-0sKY

Ultramarinus
Ultramarinus

The faces that we won't see before 2020 thanks to consoles and nearly every PC game being ported from them.

lhughey
lhughey

That looks great.  However, its in the best interest of hardware developers ot ensure that the technology is as power hungy as possible so you'll have to upgrade to their most powerful cards to get acceptable framerates.

hugoadan
hugoadan

wow, impressive graphics indeed!

kangsta1111
kangsta1111

Yes but do you poverty pc gamers have 8 gigs of ram like the ps4 will?











trolololol. 

IanNottinghamX
IanNottinghamX

@craigprime actually Nvidia is not working with Microsoft or Sony..they want nothing to do with them...

highlanderjimd
highlanderjimd

@craigprime Once this kind of detail is in open world games I think we are at the point where theres no need to go any further.  PCs are probably about 3 generations of GPU away. Consoles about 6 Generations away. Will be interesting to see who tries it first. I think a game like Heavy Rain where theres pretty much nothing else going on will incorporate such tech first.

Gle4se
Gle4se

@craigprime not. They have a lot of things to do. You can't do an entire world with only this face technology.

ashyblood
ashyblood

@gatsbythepig "awful" and "repulsive"? Are you sure you understand the definition of uncanny valley? It symbolizes CG animation that is so realistic (uncanny) that it is "creepy". How is that "awful" and "repulsive"? Your comment is rather confusing.

Virkhus
Virkhus

@kurtven Witcher 2 used the dangling tech (hair braids, cloth hanging from armour and such).

Metronoid
Metronoid

@coolguyhero The consoles hold the graphics power for PC games. The games in ps3 looks like all dx9 even the Pc ports using at max . Some games you have a improvement, but we know that this will hard be possible with the ps4 specs, to even use TressFx from the same AMD.

Ravenlore_basic
Ravenlore_basic

@Metronoid You will not be able to see this in ANY GAME for a while now. PS 5 is more like it as it will be 9 years from now 2022 when PS5 comes out. 

onixevil
onixevil

@Metronoid Did you see the shapes of the trees in GT Series ?.They've never changed since GT ONE   and you are talking about PS6  .

princetom
princetom

@Metronoid Are you kidding me?  this technology sucks

Now this is something we will probably see in PS6:

%s

Maybe even PS4 will run this with its Square Enix technology

andrewwittmaier
andrewwittmaier

@Mayleene Apparently lots of people care since there are 260 comments on this article and Nvidia has spent countless hours and copious amounts of money developing this technology.  I think there probably are people out there who don't care about this, and maybe they outnumber the people who do care.  Just because you don't care about it doesn't mean you should shoot your mouth off in a comments section where your opinion has nothing constructive to add.  

Double-Ego
Double-Ego

@warhawk-geeby Yeah eyes and hair are some of the hardest things to get right, hence why they made this dude bald.

They couldn't have left out the eyes though... especially since they wanted to show off those creepy animations.

placksheep
placksheep

@princetom 
This animated character doesn't look anywhere near as good as the NVIDIA demo. He's firmly in the uncanny valley. Though this is a lot more feasible from a logistical standpoint.

Metronoid
Metronoid

@kangsta1111 Liked the sarcasm in the end. Lol even the lousiest of the notebooks comes with 8gb ram this days.

Ladiesman17
Ladiesman17

@kangsta1111 

Brand new 8 Gb RAM cost around $30 these days, it's pretty affordable. also 16 Gb is already supported on the PC.

RAM mostly used for multi-tasking, 4 Gb actually is enough to play single games. more RAM basically is useless if CPU / GPU are weak.

I think PS4 need extra 8 Gb purposely for cloud streaming, and sharing / uploading videos on Youtube. both features are pretty demanding and intrusive to the gameplay. and Sony online infrastructure is the weakest compared to PC or Xbox, it's safe to assume that people will experience substantial lag while playing their games.

erix43
erix43

@kangsta1111 

Poverty PC gamers? 8 Gigs of ram? Are you serious? Sir, you're an idiot. Get a job and a college degree. Poverty PC Gamers, 8GB hehehehehe.. What a Knob Jock. 

Mamamf
Mamamf

@kangsta1111 XD ram is the cheapest thing on the market atm. but you won't need more than that for the upcoming years, you might have to update to faster memory not bigger. :)

henrikstorm
henrikstorm

@JustinKnight17 @zenogandia The thing is, there wont be enough money to be made, because of the pricing they'd need to profit from it. Its the same reason the next-gen games are costing 10-15% more. I dont even want to think about what a game like this would cost, if every model they had in the game would need this kind of attention.

Fanible
Fanible

@ashyblood @gatsbythepig 

Gatsbythepig's comment is really strange, but I don't think you have it either.  The experience that is the "uncanny valley" is simply that our familiarity with the human appearance makes artificial creations of it unsettling to take in.  That while it can be quite close to looking/acting like a human, there's still something off about it, and it makes you feel uneasy.  The uncanny valley is a close approximation of what we believe looks and acts like a human, but it's never perfect enough to fool us.  Instead, it just weirds some people out.

I'd say the least I can describe my own feelings towards it, is simply the recreation is never perfect and you can always easily tell the wrongs and faults put into a human recreation.  It never makes me feel off, but it's just easily noticeable that it's not real.  

As far as people in general go, I think most gamers feel the same way as well, since we're use to humans in games not looking or acting quite right, but we accept it because we expect it in games that it's not always going to be spot on.  If you get non-gamers to watch games they tend to react more negatively to depictions of human characters in games (especially cut-scenes) and will speak out more often than those actually playing the games will.  The same people probably tend to react more harshly to CGI in movies as well.

coolguyhero
coolguyhero

@Metronoid i am not a fan of console or pc just i am sharing my thoughts .i am having psvita ps3,pc and i play on all.just what i am saying that cost gives the quality and i believe at the end the only mateers is good exclusive games.consoles create more cexclusives games & thatswhy people buy them.infact now also ps4 is spc friendly but i am having doubt that all the developers will port those games to pc because then console will be in loss.thatswhy console are trying to give some competition to pc in terms of socail ,video and using other features like motion tracking etc.at the end games matter...:) cheers

princetom
princetom

@Ravenlore_basic  I would imagine it's not that hart to implement these visuals onto PS4, because we had L.A. Noire back in 2011. that game show some very great technology for realistic face expressions and if you were to combine that tech with either the fox engine or the new square enix engine, then most probably we would get the same result as this guy in my video. the only problem is that developers apparently can't handle new graphical engine quite well. nowadays we have to wait a long time for some of the games to come out. for example while ID software used their ID tech 4 engine, they were able to produce annual game from 2004 to 2009, but now they have to struggle with the technology and the have only made one game with that engine in 2011 and we are still waiting for news regarding the next game, but there is none

Metronoid
Metronoid

@princetom I hope too, the great of developers is the art they create in their engines !

princetom
princetom

@placksheep @princetom these NVIDIA demo characters look more cartoonish, then realistic. they look like something out of one of those animated movies for kids. this character is more realistic then anything I have ever scene in a tech demo. When I saw this demo for the first time, I was completely speechless. the most mind-blowing fact about this demo is that it's over 2 years old or even more. back then games didn't even look anything like this (but they still don't). This is what the next gen gaming should look like, because this animation capture the human features perfectly, including the skin, because gaming was problematic in the skin department and still is. the facials, the the haircut, the moving parts of the face are all perfectly executed. NVIDIA's demo showed more of a artistic style for games based on cartoonish art styles, but if this is what they are trying to throw as something toward realism gaming, then this demo ain't got shit on next gen. plus the ears on the bald guy look really fake

ashyblood
ashyblood

@gatsbythepig What sucks about that negativity non-gamers express about human characters in CG is that they seem to not even appreciate it, not the fact that it's realistic and certainly not the hard work that goes into creating such things. They watch The Adventures of Tintin and instead of being in awe, they focus on the imperfections and the little amount of artificiality that can be seen. 

This is something we all do though. For instance : If you see an ordinary girl, you look for the good, whereas when you see a super-hot girl, you might instantly notice the bad, if any. When we see things that come close to perfection we automatically star to look for imperfections - it's why we enjoy The Incredibles, Monsters vs. Aliens, and Ice Age but then when we see something like Tintin or something that's more realistic, we start to judge. It's because we start seeking the flaws. I don't know if it's our nature or if we're just spoiled but it is what it is. It definitely ain't going to stop creators from trying to climb out of that valley though. 

But honestly, I think the day we can recreate something that will fool us will be the day we also have really advanced artificial intelligence. Even with that though, the behavior and movement may be a bit inhuman and artificial.

Metronoid
Metronoid

@coolguyhero @Metronoid I have a lot of consoles, even the Wii U that I loved.  Im not a graphic freak, but I seeing the things the other way since I was most a console fanboy. I mounted my PC for the first time and I see the opnion from the PC gamers rigs. I think gameplay is more worth than graphics. The first console in this gen I bought was the Wii just for the new way to play, now the Kinect can go a little more deeper too in way of movement, not hardcore gaming of course. The next gen system I most wanted now is the next Xbox, they talk about VR coming to it and theres Kinect 2, hope they don´t ban used games or the sells will go down the day 1, ;) cheers mate

zenogandia
zenogandia

@princetom @Ravenlore_basic It's not hard, but imagine having to create those types of visuals for a 6+ hour game, not only the character models, but the whole game world... The amount of $$$ would be too much. It's not possible. 

ashyblood
ashyblood

@princetomFirstly - it's "better than", not "better then". As for your taste, I don't know how you can fix that. Apparently, nobody agrees with your claim that the Nvidia demo is "cartoonish". You seem to be impressed with the Janimation demo more only because he's making overly extreme facial expressions. You're entitled to your opinion but honestly, both demos are very realistic and if you cannot see that, I'm that sorry you're missing out.