Half-Life 2 hands-on benchmarks

Earlier this week, Valve Software released results from its internal benchmark testing of Half-Life 2, showing the latest DirectX 9 video cards from ATI outperforming the competing video cards from Nvidia. Yesterday, we finally got a chance to run some of our own benchmarks, using computers configured and provided for us at the Valve/ATI press event. Because Half-Life 2 has been designed, from the ground up, as a DirectX 9 game, taking full advantage of the latest programmable pixel shader technology, Valve claims that its benchmark utility should be an accurate predictor of performance for other future DirectX 9 games.

The benchmark consists of three segments, each depicting a scene from the lengthy movie of Half-Life 2 released by Valve at E3. The first unit is called "techdemo_5." It shows the first portions of the E3 movie, where the camera shows the source engine's various effects, including refractive water, translucent stained glass, and a character model made of water. The second unit, named "bugbait," shows the scene where Gordon Freeman uses special lure grenades to enlist the help of large alien bugs against combat troops. The final unit, entitled "city 17," tests performance in the outdoor level, where armed walkers wreak havoc in an urban setting. On every test setting, the Half-Life 2 benchmark outputs the numbers for each of the three units. For our purposes, the benchmark numbers you see here represent the simple average of the three scores.

The video cards included in our test were the Nvidia GeForce FX5600 Ultra (128MB), the ATI Radeon 9600 Pro (128MB), the Nvidia GeForce FX5900 Ultra (256MB), and the Radeon 9800 Pro (128MB). All cards were tested with a 2.8GHz Pentium 4 and 1GB of RAM. The only difference is that the FX5600 Ultra and the 9600 Pro were tested with P4s running on a 533MHz front-side bus, while the higher-end FX5900 Ultra and 9800 Pro cards were paired with a P4 running an 800MHz front-side bus. All cards were tested using the latest publicly available drivers (version 3.7 for the ATI cards, and 45.23 for the Nvidia cards).

Our numbers show that the ATI cards run much faster than their Nvidia counterparts under both DirectX 9 and DirectX 8.1. Interestingly, the Nvidia cards receive a large boost in performance when switching down to DX8.1 from DX9, whereas the ATI cards run at about the same speed regardless. Also telling is that Nvidia's flagship 5900 Ultra ran slower than ATI's more modestly priced 9600 Pro in DX9 full-precision mode.

One of the places in the benchmark demo where we were able to see a distinct difference between DX8 and DX9 graphics was in the water. The comparison picture you see was shot on the Radeon 9800 Pro under both DX8.1 and DX9, capturing the exact same frame within techdemo_5. Take a look at the comparison screenshots posted in the gallery above and notice the refractive properties of the water in each shot. You'll see that the DX9 version refractions and reflections are more obvious to the eye than under DX8.1.

More dramatic differences between DX9 and DX8 might be gleaned from the high-dynamic range lighting video released by Valve yesterday. The intensity of the lighting, the light bleeding around the edges of the windows, the soft shadows cast by the bug, and the combination of diffuse and reflective bump-mapping on the bug's carapace are all DX9-native effects that cannot be replicated as well under DX8.1. We were told at the event that Valve would create a DX8 version of the video soon, which, in conjunction with yesterday's video (which reflected Half-Life 2's DX9 full-precision mode), might provide a more definitive illustration of the differences between DX9 and DX8 effects.

ATI and Nvidia architectures differ in several ways, but one significant difference is in how each company's cards handle floating-point operations, a key component to creating advanced graphics with pixel shaders. ATI calculates both full- and partial-precision floating-point operations with 24-bit precision. Nvidia's cards calculate half-precision operations at 16-bit precision and full-precision operations at 32 bits.

To improve the performance of Nvidia cards under DX9, Valve has invested a considerable amount of time programming a special graphics mode for Half-Life 2 called "mixed mode." Running the game under mixed mode means that certain shaders will have been reprogrammed using partial-precision calculations, which could potentially compromise the quality of the graphics but would offer a boost in speed. Under mixed mode, the game engine will also rely more upon traditional texture fetches to render graphics and less upon programmable pixel shaders. This should conceivably help the Nvidia cards in speed--as one of the strengths of the FX series architecture is memory bandwidth--whereas the ATI architecture allows for more floating-point operations per cycle.

From the graph above, we see that performance improves dramatically on the 5900 Ultra under mixed mode, but the card still lags behind the 9800 Pro when running at full precision. The 5600 Ultra also saw a modest boost in speed under mixed mode, but the averaged score is still under 15 frames per second.

Yesterday, Nvidia responded to the uproar over the latest Half-Life 2 benchmarks by saying it will soon be releasing new software drivers for its cards that should markedly improve performance in the game, as it includes optimizations specific to Half-Life 2. The company claims that any results obtained from testing Half-Life 2 with the publicly available 45.23 drivers are "invalid" because development on that series of drivers stopped long ago. As a result, they wanted to provide more attention to developing the new version 50 drivers.

When asked about the new version 50 beta drivers, Valve representatives responded by saying they'd never seen the very latest version that Nvidia had been distributing to members of the press this week. Valve also stressed that they are firmly against game-specific and benchmark-specific optimizations included in drivers, as such tweaks make it more difficult to predict performance in other games programmed under the same API. They also noted that game-specific optimizations can also introduce stability issues when developers patch games, when the engine is licensed out to other developers, or when new mods and mod updates are released.

Valve remained tight lipped about Half-Life 2's release date at this week's press event. However, Gabe Newell, Valve's founder and managing director, did note that the Half-Life 2 benchmark program will be included in the box when the software does ship and that it will be distributed to the press on September 30. Taken in combination, these two bits of information make it doubtful that the game will ship on September 30, Valve's announced release date.