FarCry v1.2 SM3.0 Path Analysis

The Benchmark

Since we are focused on one game, our testing methodologies will be a little different this time around, and we'll take the opportunity to go in-depth on our testing method for this game as well. We will look at 6 different levels of the game, and for each Level Analysis, we will look at image quality as well as demo benchmark performance.

One of the largest caveats about benchmarking in FarCry is that demos don't work like one would expect. For example, in Unreal Tournament 2004, we can start a game, play our hearts out, start and stop recording somewhere in the middle, and we have a very cool little benchmark of the action. This repeatable benchmark is a fair representative of gameplay as far as benchmarks go. This seems reasonable for a game with a built-in demo mode.

FarCry, on the other hand, will record the movement of the player through a level without recording any of the players actions (like firing a weapon or pressing a button to open a door), and none of the other characters in the level are recorded either. When a demo recorded in single player mode is viewed, all the AI controlled characters man the same posts that they would in the game; only they ignore the player moving through the level when the demo is running.

This means that some demos have instances of passing through locked doors, and AI bots making their normal rounds regardless of the player (and can be in different locations for different runs of the benchmark). And as if this wasn't enough, the worst part of the whole experience (have you figured it out yet?) is that demos are entirely absent of any fighting, conflict, or gunfire.

We tried many times to benchmark this game using FRAPS, but our ability to be repeatable was worse than what demo playback gives us.

So, why are we at all OK with using FarCry's built-in demo mode? Because much of FarCry game play has to do with sneaking around, walking through the levels, and taking in the scenery. No, it's not the all-encompassing perfect benchmark, but it isn't the worst thing that we've seen either (*cough* - 3dmark - *cough*). We've compared the demo mode to our very non-repeatable FRAPS benchmarks of walking around levels and we are comfortable with the reliability of the scores that we get from the demo for that purpose.

Also, when we were informed that this patch was coming down the pipe, NVIDIA sent along a couple demos to test the performance difference with the new SM3.0 path enabled. Ubisoft is going to include these 4 demos with their patch, but we were obviously a little wary of just throwing these numbers up. We took a close look at the demos, and we are including them alongside our original custom demo and a new custom demo that we recorded for this article. The reason why we are including the NVIDIA provided demos is that they are definitely sections of the game that are really parts of the gameplay. Whether these are representative of overall gameplay or not, there are definitely experiences in the single player mode of the game that are represented by the demos.

It is important to keep the numbers in this test in perspective. We are specifically investigating the benefit and impact of CryTek's new SM3.0 path. To explore the technology fully, it is necessary to look both at parts of the game that benefit most from the update as well as those that don't see that much impact. We trust that the demos provided by NVIDIA will highlight the best results that we can expect to see, and it just so happened that our original demo and the new one, which we recorded, see less change (as we will soon discover).