Sometimes setbacks happen. I had been planning an article on top to bottom graphics hardware performance in Far Cry 2 for this week. This was already pushing it with a platform change, and it just got to be too much to get it all finished in time. In time for what? Well, in this case it was the fact that AMD just released a new hotfix driver for Far Cry 2 that fixes a couple rendering issues the original hotfix had. It wouldn't do to publish an article with numbers from an old driver, so here we are retesting things.

But that's no reason to disappoint our readers. Far Cry 2 is a decent looking game that many have been interested in for a while. I haven't really played the game much (just doing testing) so I can't comment on the goodness of the game. But I can comment on the complete awesomeness of the benchmark tool it comes with.

The Far Cry 2 benchmark tool owns all other built in benchmark tools around. So far there are only two issues we have with it. We want it to be able to take screenshots at a specific frame or at specific intervals, and we want it to be just a little more stable. The former issue is of less importance than the latter, as there are other ways to take screenshots. But stability is the only thing that really stands in the way of some incredible analysis. If running more than 5 to 10 different test cases, we've found the tool to be likely not to complete. If individual runs fail, we'd still like to see the benchmark continue, but unfortunately it just stops.

This is a big issue when performing 60 runs. That's right, 2 DX versions, 2 AA settings (off and 4x), 3 quality settings, and 5 resolutions. It's a data collectors dream come true. Not only do they keep CSV files with frame data for all frames from each of the 3 loops we are running per test, but they build a nice html file with all the data for easy access and even display a graph of instantaneous framerate per frame. Even though I haven't had the chance to re-run our AMD numbers, here's a sample of what we're looking at:

Comparing this to the DX9 and to the Radeon HD 4870 in both DX9 and DX10 will be quite informative. As we can see, even at the maximum possible settings, we still get playable framerates. This is really a testament to the engine, as Ubisoft were able to do some great things with the visuals while still providing excellent playability across the board.

Under DX10, High and Very High quality settings don't differ in performance nearly as much as Ultra. In fact, you can get Very High quality with 4xAA for about the price of Ultra quality in most cases with this test under the GTX 280. Performance with 4xAA tends to drop off faster as resolution increases than without regardless of quality mode. This makes sense because as resolution increases the added memory needs for 4xAA increase as well. I'll get in to more analysis of the data when we get the numbers for the article done.

The benchmark tool offers lots of other cool features beyond just running billions of tests in an automated way. For instance, we can run fixed time length demos in which the camera travels a path and frames are rendered as fast as possible -- like the game is played. These are similar to FRAPS tests, but the data that's collected is much more repeatable because the camera path is fixed. We can also run a fixed number of frames timedemo. A timedemo is useful because it renders the exact same set of frames each time as fast as possible. This way, we have a more consistent work load and know what's going on better. Of course, this stresses different things. Each type of test has it's use, and buidling in this kind of flexibility is incredible.

We commend Ubisoft for creating such an AMAZING benchmarking tool. I hate SecureROM because dealing with activations is tough when we build and/or reinstall new systems all the time and our hardware is always changing. But the benchmark tool almost makes up for it. Almost. Every other game developer needs to copy this tool immediately and put it in every game.

Anyway, I'm off to bed for now. I'll be testing this more tomorrow with the latest drivers and will get this wrapped up some time early next week most likely. I'm hoping to have some neat surprises in here that are made possible by the flexibility of the benchmark tool. But no promises ;-)

POST A COMMENT

14 Comments

View All Comments

  • aguilpa1 - Saturday, November 01, 2008 - link

    A couple of very old 8800GTX's Q6600 @ 3.2Ghz are faster than a single 280GTX on this engine. It is a well optimized graphics engine. I did the last run at 1280 x 1024 for apples to apples since my native resolution is 1920x1080 as opposed to the comparison resolution of 1920x1200.

    Resolution 1920x1080:
    High Quality NoAA 75.01
    Very High Quality NoAA 65.26
    Ultra High Quality NoAA 58.17
    High Quality 4AA 65.78
    Very High Quality 4AA 52.33
    Ultra High Quality 4AA 43.97

    1280x1024:
    Ultra High Quality 4AA 54.64



    Reply
  • Rock Hydra - Thursday, November 06, 2008 - link

    I don't know why exactly that is, but I remember the thread in the forums showing 2 old ass video cards running Doom 3 in SLI. 3DFx maybe, well anyway it ran at playable frame rates, but quality was bad. So I mean, if your card can't render everything, it will perform faster. Not necessarily the case, but also a thought about why performance might be better. Reply
  • aguilpa1 - Saturday, November 22, 2008 - link

    WTF are you talking about? The quality is the same, and it renders everything same as the 280GTX and the 4870's. Reply
  • marlinman - Friday, October 31, 2008 - link

    The tool IS cool. Not convinced that 3 loops is sufficient though given that during each loop a slightly different scene is displayed. I've noticed (with a little suspicion) that the first loop invariably boasts a higher fps than subsequent loops (in earlier games the reverse invariably holds!). Stability of tool is fine here - with fixed settings, I ran 30 loops (10 for each demo map size) without incident. The first ten results (which are quite representative)

    43.47
    36.67
    35.33
    37.56
    38.67
    36.25
    37.51
    36.59
    38.97
    37.12

    suggest I think that averaging the first 3 runs could be misleading(?)
    Reply
  • Atechie - Friday, October 31, 2008 - link

    I still don't get the love for the canned benchmark tool...how does the result compare to actually playing the game...or wasn't that a factor? Reply
  • BLHealthy4life - Friday, October 31, 2008 - link

    I cannot get DX10 and AA to work with this game at all! I've tried 178.24, 180.42 and 180.43, NONE of those drivers will give me DX10 and AA at 2560x1600!

    Am I the only one having this f'ing problem?!
    Reply
  • doomedtofrag - Friday, October 31, 2008 - link

    Hello Derek,

    I understand performance analysis is the primary concern, but it would be nice if you could include in-game screenshots showing quality differences under various settings. Your site does similar thing when it comes to digital cameras (of course that is their primary function). As a gamer, I would love some in-game screenshots, as my faith in the plethora of gaming sites is dwindling day-by-day...
    Reply
  • ThomasS31 - Friday, October 31, 2008 - link

    Derek,

    Do you plan to investigate the new Stalker game AMD DX10.1 vs nVidia DX10 quality performance situation as well?
    Might worth a short, but good quality article on that. :)

    Tamas
    Reply
  • ThomasS31 - Friday, October 31, 2008 - link

    Hi Derek,

    Please note (also in the article) that the GeForce DX10 and Radeon DX10.1 card are not doing the same amount and type of calculation with AA enabled!
    So might worth noting that one-to-one compare is not possible or "fair", both on performance and quality.
    (Similar, but not he same like the Assassin Creed case...)

    Regards,
    Tamas
    Reply
  • thebeastie - Friday, October 31, 2008 - link

    I been looking at getting a new PC for the last 6months at least but just aren't happy with whats out there.
    The main idea behind the new PC is to play games.

    I been waiting for the new Nehalem CPU but I have been thinking I am probably better off making sure I get the latest GPU compared to the new Nehalem.

    Would be good to get a comparison of whats more worth while throwing money at. I think a lot of people suffer from this problem.

    Reply

Log in

Don't have an account? Sign up now