New Testing Methodology

Every twelve to eighteen months it makes sense to upgrade our test beds in order to best represent what is available on the market. How the upgrade occurs depends on what is being tested, and in the case of our APU reviews it is clear that due to the wide range of graphics options available, as well as at different price points, that we have to adjust our gaming testing.

For 2015 our CPU performance testing regime remains untouched aside from the late 2014 addition of Linux-Bench for a glimpse into Linux based performance. On the gaming side, our games have been updated to the following:

  • Alien Isolation (First Person Survival-Horror)
  • Total War: Attila (Strategy)
  • Grand Theft Auto V (Open World Sandbox)
  • GRID: Autosport (Driving)
  • Middle-Earth: Shadows of Mordor (Action-Adventure)

Because budgets for gaming graphics cards can vary, or users decide to keep the same card for several generations, we will be testing each of these titles in both low, medium and high end graphics setups. This means we can see where the bottlenecks are for CPU performance at each stage. We have also been able to source both AMD and NVIDIA cards for most of these areas, should one side of the equation scale more than the other.

The GPU sections are split into three based on where they fit in their independent stacks rather than for direct competition:

Low-end:
 - Integrated Graphics
 - ASUS R7 240 2GB DDR3 ($70)
 - Dual Graphics (where applicable)

Mid-range:
 - MSI GTX 770 Lightning 2GB ($245-$255 on eBay/Amazon, $330 new)
 - MSI R9 285 Gaming 2GB ($240)

High-end:
 - ASUS GTX 980 Strix 4GB ($560)
 - MSI R9 290X Gaming LE 4GB ($380)

On the low end, we have selected settings in order to make the current best integrated graphics solutions score between 45 and 60 frames per second. On the mid-range and high-end, we typically pull out 1080p maximum settings or almost-maximum.

The Shadows of Mordor (SoM) benchmark throws up a little interesting teaser as well due to the use of its Dynamic Super Resolution technique. This allows us to render at 3840x2160 (Ultra-HD, or ‘4K’) with our settings despite using a 1080p monitor. As a result, we also test SoM at 4K ultra with our mid-range and high-end graphics setups.

For the high-end setups, as we have managed to source 2 cards of each, means that where applicable we can test both SLI and Crossfire setups. We apply this to Shadows of Mordor at 4K as an extra data point.

For clarity, this means:

  Integrated
R7 240 2GB
Dual Graphics
GTX 770 2GB
R9 285 2GB
GTX 980 4GB
R9 290X 4GB
Alien Isolation 720p Ultra 1080p Ultra 1080p Ultra
Average Frame Rate Average Frame Rate Average Frame Rate
Total War: Attila 720p Performance 1080p Quality 1080p Quality
Average Frame Rate Average Frame Rate Average Frame Rate
Grand Theft Auto V 720p Low 1080p Very High 1080p Very High
Average Frame Rate
%FPS <60 FPS
Average Frame Rate
%FPS <60 FPS
Average Frame Rate
%FPS <60 FPS
GRID: Autosport 1080p Medium 1080p Ultra 1080p Ultra
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Middle-Earth:
Shadows of Mordor
720p Low
 
1080p Ultra
4K Ultra
1080p Ultra
4K Ultra
4K SLI/CFX
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate

For drivers, we locked down the 350.12 WHQL versions from NVIDIA soon after the launch of GTA V. Similarly, the 15.4 Beta drivers from AMD are also being used. These will remain consistent over the next 12-18 months until the next update.

All of our old (and new) benchmark data, both for CPU and graphics performance, can be found in our benchmark database, Bench.

We have a variety of benchmarks here, including legacy benchmarks such as CineBench 11.5 and TrueCrypt, which are not published in the main review. All CPUs/APUs that have been tested in our new 2015 style will be labeled in the dropdown menus by having its launch price listed, e.g. ’AMD A10-7850K (95W, $173)’. With any luck over the course of the next six months we will be adding new data and re-testing older processors for the database in order for our readers to compare old with new.

AMD A8-7650K Review AMD A8-7650K Test Setup, Overclocking
Comments Locked

177 Comments

View All Comments

  • akamateau - Tuesday, May 12, 2015 - link

    Your comments assume that ANAND provided benchmarks using DX12 and they did not. ALL of the GRAPHIC benchmarks were with either synthetic benchmarks or game benchmarks using DX11.

    DX11 cripples the performance of ALL APU's, IGP and dGPU. Draw calls ARE a measure of CPU-to-GPU "bottleneck" or elimination thereof. You can not render a polygon until you draw it.

    DX12 enables CPU core scaling; basically increased draw calls are a function of the amount of multithreaded cpu cores. DX11 does not allow multithreeaded gaming.

    DX11 may be current but why should I base hardware purchases from testing based on obsolete software AND benchmarks?

    DX12 will be in widespread use by game developers by Christmas.

    Anand has psent quite a bit of time and money testing hardware on obsolete benchmarks TO WHAT END?

    Starswarm and 3dMArk API Overhead Test are available but ANAND ignore them.

    Why?

    AMD's APU was designed to FLY using Mantle and DX12. It is not AMD's fault that Intle IGP is so poorly designed. That is Intel's problem.

    Test Intel IGP using the latest API and you will see. Comaparatively test AMD and Intel using obsolete benchmarks with DX11 and ANAND is lying to the consumer and can not be trusted.

    AN unbiased and well balanced piece should use legacy benchmarks, they should also use the very latest available. ANAND di not do this.
  • rp1367 - Tuesday, May 12, 2015 - link

    "Starswarm and 3dMArk API Overhead Test are available but ANAND ignore them.

    Why?"

    Because they want to hide the truth. "It is hard for a person to wake if he is asleep because he pretends to be asleep but infact he is not. He just want to fool you because of his stupidity"

    The refusal to support the upcoming DX12 give as hint that the review is biased and something fishy going at the backdoor. I am not an IT guy and new on this site but i could easily detect what is the difference between biased and unbiased review.

    The reviewer and Anadtech guys for sure are all intellegent guys but they allowed themselves to be succumed by their own personal interest.
  • Gigaplex - Tuesday, May 12, 2015 - link

    Or its just something as simple as DX12 not being released yet, the performance is likely to change, so is an invalid test for comparing hardware at this time. The benchmarks you refer to are only valid as a preview for potential gains.
  • akamateau - Tuesday, May 12, 2015 - link

    Windows 10 with DX12 will be released in less than 2 months. Mantle is final and DX12 is final. Anand has it but ignores it.

    By Christmas ALL new games released will be DX12.
  • Gigaplex - Tuesday, May 12, 2015 - link

    DX12 may not be final. API probably is, runtime is likely close, drivers likely won't be.

    And you're delusional if you think all new games released at the end of this year will be DX12. It takes years to develop a AAA game, so they would need to have started before DX12 was available. The market for DX12 will be tiny by Christmas as DX12 will be Windows 10 only. Not everyone will be willing or able to upgrade the OS. Not all hardware even supports DX12. You're completely ignoring the history of previous DirectX roll outs.
  • akamateau - Tuesday, May 12, 2015 - link

    Windows 10 with DX12 will be released in less than 2 months. Mantle is final and DX12 is final. Anand has it but ignores it.

    By Christmas ALL new games released will be DX12.
  • V900 - Tuesday, May 12, 2015 - link

    Ah, so basically the choice is buy an AMD APU and get shoddy performance now, and great performance in a year, or buy an Intel/Intel-Nvidia solution and get great performance now and great performance in a year!

    So theres really no reason to get the AMD is what you're saying?
  • JumpingJack - Tuesday, May 12, 2015 - link

    I will bet you what ever you make in a year that not all games released between DX12 release and christmas of this year will be DX12 native.
  • akamateau - Tuesday, May 12, 2015 - link

    @Ian Cutress

    "as well as having parallelizable code"

    ARE YOU NUTS?

    You really need to cut the crap.

    Mutlthreaded gaming will come as a result of DX12 and Asynchronous Shader Pipelines and Asynchronous Compute Engines.
  • galta - Tuesday, May 12, 2015 - link

    I don't think he is nuts, but you seem a bit angry.
    From a CPU perspective, multithreaded games need not wait for DX12. They could have been written before.
    Anyway, we have a clear statement of you: DX12 will make AMD shine. We should talk again on Christmas.
    Just keep in mind that the same was said when DX11 was about to be released, with known results...

Log in

Don't have an account? Sign up now