At a satellite event to Intel's Developer Conference last week, AMD held an evening soiree with invited analysts and press to talk about their new upcoming Zen microarchitecture. This was intended to be a preview of tomorrow's Hot Chips presentation, and we've already covered the juicier parts of the presentation in terms of microarchitecture announcements as well as diving deep into the Server-based Naples implementation and what the motherboards told us from memory and IO support. 

You can read both here:

AMD Zen Microarchitecture: Dual Schedulers, Micro-op Cache and Memory Hierarchy Revealed
AMD Server CPUs and Motherboard Analysis

There was one other element to the presentation that requires further discussion and analysis, if only to clean up some of the misinformation already present online and to label what was shown with a long list of potential caveats which most people seem to have passed by almost entirely. As part of the show, AMD compared the performance of their Zen engineering sample to an Intel Broadwell-E processor. 

In this test, they told the audience that each system was running eight cores, sixteen threads, and will all cores set to 3 GHz (implying no turbo). Memory arrangements were not disclosed, nor storage: we were told to assume comparable setups. 

We were too busy trying to decipher what was on screen (and take note of the results) to actually photograph the benchmark as it happened (there are videos online), but the benchmark they showed was Blender, an open source rendering engine, with a custom multithreaded workload. The test was to render a mockup of a Zen based desktop CPU, with an effective workload of 50 seconds for these chips. I've seen numerous reports about this result saying the difference was 1 or 2 seconds, but with rarely a mention of the benchmark length, which is as important. The overall results were

  Blender Time to Render / sec
Intel Broadwell-E
Core i7-6900K
8C / 16T
3 GHz all-core
49.05
AMD ZEN
Engineering Sample
8C / 16T
3 GHz all-core
48.07
(-0.98 sec, 1.998%)

All things being equal (we'll get to that in a second), this would suggest that an 8-core AMD has a ~2% advantage over Broadwell-E at the same clock speeds. Despite this result, there are a lot of unverifiable parts to the claim which makes analysis of such a result difficult. I want to go through each of them one by one to ensure everyone understands what was presented. 

I'll preface this analysis with two things though: one is that AMD was very careful in what they said at the event, and only said as much as they needed to. Thus is why the string of caveats for this benchmark test is fairly long. But also, AMD has to set expectations here: if they choose an environment and test that represents the peak, or relies on something special, users will feel burned again after Bulldozer. AMD has to temper those expectations but still represent a methodology that is effective to them. By leaving so many cards on the table, this can both be a help or a hindrance.

But given the measured and calm, professional nature of the event, as opposed to the wild and wacky AMD events of the past, it was clear (either by design or chance) that the words used said only as much as they needed to. Along with the microarchitecture discussions, it was designed to provide a good stepping stone on to the Hot Chips presentation a few days later.

So, caveats. Or disclaimers not readily provided. Let's start at the top. 

1) The Results Are Not Externally Verifiable At This Time, As Expected

We were told the setups of the systems being used, but were unable to confirm the results manually. This is typically the case with a high level, early look at performance and other companies do this all the time.

This being said, it would look bad on reports if it to turns out or someone finds a chasm between pre-launch and launch data, so the aspect of reporting this data without understanding this caveat is fundamental. The basis of publishing scientific papers is repeatability and verification - while this wasn't a scientific presentation, it is important to keep it in the back of your mind when you hear any benchmark numbers (AnandTech included - our numbers are designed to be verifiable and we want to have a history of honesty with our readers, especially when it comes to custom software/workloads we cannot disclose). 

2) No Memory or TDP Numbers Were Provided

We were able to determine that the AMD-based systems were running 2x8 GB of DDR4-2400, although we did not get a look at Intel's memory arrangement. Similarly, due to the ES nature of the CPU, TDP numbers were also not shared however we did see all the AMD systems use either the AMD Wraith cooler (which is rated at 125W) or the new near silent cooler (95W). That tends to peg the system at a peak power consumption and some of AMD's current competitive parts actually use a cooler designed for the bracket above in TDP (e.g. A10-7860K at 65W uses the 95W cooler, A10-7890K at 95W uses the 125W cooler). 

3) Blender Is an Open Source Platform

One of the issues of using open source is that the code is editable by everyone and anyone. Any user can adjust the code to optimize for their system and run a test to get a result. That being the case, it can be difficult to accurately determine the code base for this test, and is relatively impossible to determine the code base of Blender that was compiled for this test.

Even in the base code, there could be CPU vendor specific optimizations in either the code or compiler that influences how the code manipulates the cache hierarchy with the workload and adjusts appropriately. It also doesn't help that Blender has elements in the code called 'AMD', which relates to a series of internal rendering features not related to the company. Going down the optimization for specific CPU microarchitectures leads on to another more philosophical issue...

4) Did It Actually Measure IPC? (The Philosophical Debate) 

In the purest sense, measuring the number of instructions per clock that a set of instructions can perform can determine the efficiency of a design. However, the majority of highly optimized code bases do not have general-purpose code - if it detects a particular microarchitecture it can manipulate threads and loops to take advantage of the code design. How should IPC be measured is the main question: using identical code bases makes it easier to understand but are often non-real-world compiler targets, or highly optimized code to show the best of what the processor can do (which means that IPC performance is limited to that benchmark)? With the results we saw, if the difference of about a second in just under fifty seconds translates into a 2% difference, is it accurate to say that this is a 2% IPC increase, or does it rely on optimized/non-optimized code? Optimizing code, or profiling compilers for specific code targets, is nothing new. In the holistic view, most analysts use SPEC benchmarks for this, as they are well-known code structures, even though most benchmarks are compiler targets - while SPEC is not particularly relevant for the real world workloads, it does give an indication about performance for unknown architectures/microarchitectures.

5) The Workload Is Custom 

One of the benefits of software like SPEC, or canned benchmarks like Cinebench, is that anyone (with a license) can pick up the workload and run with it. Those workloads are typically well known, and we can get performance numbers out that have known qualities in their interpretation. With a custom workload, that is not always the case. It comes down to experience - an unknown workload can have a lop-sided implementation of certain branches of code which is unknown when it comes to running the results. This is why rendering one scene in a film can take a vastly different time to another, yet the results for the 'benchmark' are significantly different depending on the architecture (one prefers lighting, another prefers tessellation etc.) Using known or standard workloads over long periods of time can offer insights into the results, whereas new workloads cannot, especially with so few results on offer.

6) It Is Only One Benchmark

There is a reason for AMD only showing this benchmark - it's either a best case scenario, or they are pitching their expectations exactly where they want people to think. By using a custom workload on open source software, the result is very specific and cannot be extrapolated in any meaningful way. This is why a typical benchmark suite offers 10-20 tests with different workloads, and even enterprise standard workloads like SPEC come with over a dozen tests in play, to cater for single thread or multi-thread or large cache or memory or pixel pushing bottleneck that may occur. Single benchmarks on their own are very limited in scope as a result.

7) There's Plenty about the Microarchitecture and Chip We Don't Know Yet, e.g. Uncore

One of the more difficult elements on a processor is managing cross-core communication, as well as cross-core-cache snooping. This problem is overtly exponential, with the plausibility of more direct connections per core as the numbers go up. Intel has historically used a torus (ring) interconnect between cores to do this, with their large multi-core chips using a dual ring bus with communication points between the two. We suspect AMD is also using a ring bus in a similar methodology, but this has not been discussed at this time. There's also the interconnect fabric between the cores and other parts of the chip, such as the Northbridge/Southbridge or the memory controllers. Depending on the test, the core-to-core communication and the interconnect can have different performance effects on the CPU. 

8) Clock Speeds Are Not Final, Efficiency Not Known

Performance of a CPU is typically limited by the power draw - there is no point getting a small amount of performance for a large increase in power such that efficiency has decreased. AMD has stated that power consumption and efficiency was a premier goal as this microarchitecture was developed.

At the demonstration, we were told that the frequency of the engineering samples was set at 3 GHz for all-core operation. We were told explicitly that these are not the final clock speeds, but it at the very least it puts the lower bound on the highest end processor. In reality, 3 GHz could be a walk in the park for the final retail units, depending on how much difference there is between the chips on display and what people will be able to buy. We are under the impression that the CPUs will have turbo modes involved, and those could be staggered based on the cores being used.

But this is why I said that 3 GHz is the lower bound of the high-end processor. We know from these results (assuming point 1 in this list) that the best processor from AMD will do at least 3 GHz. There's no indication of power, and thus there's no indication of efficiency either, which is also another important metric left in the ether.

9) We Will Have to Wait to Test

Everyone wants the next technology yesterday, so the 'gimme gimme gimme' feeling of new stuff is always there. AMD has already stated that general availability for Zen and Summit Ridge will be Q1, which puts the launch at four months away at a minimum. At this stage of the game, while AMD is trying to be competitive with Intel, they don't want to generate too much hype and give the game away in case it goes incredibly pear-shaped. There's the added element of the hardware and software being finalized or updated.

Since I've been reviewing, no CPU manufacturer has handed out review units four months before launch (in all honesty, we're lucky to get a launch date sample a week in advance these days). In fact we'd have to go back to Nehalem and Conroe to find something that was sampled early; however Conroe just passed its 10th birthday and in that case, Intel knew they were on to a clear winner ahead rather than just 'meeting expectations'. Also, early samples of a great product will mean users will wait for it to come out, which results in revenue loss (the Osborne effect) unless you have zero stock and/or an uncompetitive product that no-one is buying. In this decade, no x86 CPU manufacturer has offered samples this far out. I'd be more than happy for that to change and I would encourage companies to do so, but I understand the reasons why. 

Some Final Words

Much in the same way that taking an IQ test tells you how good you are at an IQ test, it is typically an indication that you are good/bad at other things as well (most well-engineered IQ tests go through a lot of spatial reasoning, for example). In this circumstance, a CPU performing a Blender test is only as good as a Blender test, but given what we know about the Zen microarchitecture, it is probably also good at other things. Just how good, in what metric and to what extent, is almost impossible to say.

AMD has given a glimpse of performance, and they’ve only said as much as they needed to in order to get the message across. However it has been up to the media to understand the reasons why and explain what those caveats are.

POST A COMMENT

177 Comments

View All Comments

  • azazel1024 - Wednesday, August 24, 2016 - link

    To play devils advocate, I disagree with the mention that the lower bound of the high end part is 3GHz for Zen. There is nothing to suggest that. To milk the benchmark AMD could have run it with a stouter cooler and upped the voltage to juice it to 3GHz. Or 3GHz could be the turbo speed, it was a pretty short benchmark. I'd assume that the chip IS capable of 3GHz, for awhile at least, but that might also be with OC'ing. I think that is the most that it suggests.

    I'd think AMD wouldn't want to milk things TOO much, so I'd expect the 3GHz is probably at most a turbo speed which it can handle with a stock cooler for the duration of the test without excess thermals. It also could be within the base clock performance. So that means that the base clock is probably somewhere between 2.6-3.1GHz for final chips and turbo performance is probably 3-3.4GHz as a random guess.

    As for actually IPC, I don't think it says a thing other than custom benchmarks can be made to make Zen look good. That said, I have some hope here. I don't need it to be equal to or better than Broadwell-E. If it can achieve 80+% of the single thread performance of Broadwell-E and at the same time can beat the performance or at least equal the performance of a HEXAcore Broadwell-E for a Zen 8 core, 16 thread part at somewhat less platform price, AMD has a sale out of me.
    Reply
  • Nagorak - Wednesday, August 24, 2016 - link

    I have to agree that I don't know what the point of this article was. It was always clear it was an isolated quick test and didn't necessarily mean anything. That great warranted maybe a paragraph, the rest is just wasted words. Reply
  • diglo1 - Wednesday, August 24, 2016 - link

    People here should realize that AOTS cant take advantage of More then 8 threads and Zen has double of that. Plus engineering sample Zen has lower clockspeed per Core then full fedget 4790 so no Wonder it loses in single Core performance and thats still what games want more. Now something like 5960x or 6900k would have shown similar performance like Zen engineering example at same clockspeeds. You must realize that currently 4c 8t i7s perform better in gaming vs any 8c 16t simply because having More cores means lower clockspeeds and clockspeed is the käy to single Core performance if IPC is the same. I5 Lost simply because it didnt have enough threads but decent clockspeed. Also Zen had 3,2ghz turbo and I7 had 4ghz or was it even More. So no surprises here Zen is delivers close to skylake depending on the spesific tasks.
    1. Zen is not at it's final clockspeeds
    2. Zen is More comparable to 5960x or 6900k both in terms of multi threaded tasks and gaming
    3. Reqular i7s beats higher end i7s in gaming simply because higher turbo clockspeeds
    4. Zens ipc is not 100% confirmed.
    5. Zen is going To make a difference By offering similar performance compared to Intel By lower prices.
    6. No matters the similarities both architectures have pros and cons.
    Reply
  • darkfalz - Wednesday, August 24, 2016 - link

    I would like AMD to do well again, but I can't really trust their marketing. Two things make me suspicious:

    1. AMD has been in the red for many quarters in the last few years. I can't really seeing them having the R&D time and money to create a competitive product other than Intel getting lazy. If they did it, hat's off, but I'm sceptical.

    2. AMD's benchmarks and marketing have always been selective and dodgy. I remember when the FX series launched and AMD released a bunch of ridiculous gaming benchmarks at high resolution/high AA claiming similar performance to Intel CPUs (because they benchmarks were highly GPU bound, obviously, taking the GPU out of the equation).
    Reply
  • ruthan - Wednesday, August 24, 2016 - link

    I no more believe AMDs numbers at all, they promised than Polaris would much more power efficient than Maxwell, it was 100% lie, sadly all so-called analysts dont care about promises vs. reality checks.

    Here are these old lies slides, youtube videos are gone:
    http://www.pcper.com/reviews/Graphics-Cards/AMD-Ra...
    Reply
  • Morg72 - Saturday, August 27, 2016 - link

    You might want to read that article you linked again. The only time maxwell is mentioned is when they compared what must have been the 460 to the gtx 950. Other than that, it only discusses that Polaris is a big leap in performance per watt over previous gen Radeon GPUs. Reply
  • Alouette Radeon - Thursday, September 15, 2016 - link

    Hey it could be worse, it's not like they used wood screws on a fake card with the ceo waving it at investors and claiming it to be real. Wood was fitting however because when Fermi was finally released it ran so hot that you could Bar-B-Q a T-Bone on it. How come you trust nVidia after that fiasco? Or what about that gong show where nVidia renamed the GeForce 9800GTX+ as the GTS 250 and increased the price by 30%? Or wait, remember when they were sodomising their own board partners to the point that BFG died? Is it any wonder that XFX and PowerColor flipped "dear leader" the bird and started making Radeon boards? That was unprecedented as no board partner had ever left ATi. Or maybe the debacle that ensued when Charlie Demerjian exposed the way that nVidia was threatening to deny samples to review sites that didn't use only "nVidia-approved" games in their reviews (read: games that performed better on nVidia hardware) because they tried to do the same thing to theinquirer.net and he wouldn't have it. Either you're too young to have seen all that (around 2009) or you have the memory of a fruit fly. My point is, sure, AMD has done some stupid things but when it comes to all-out dick moves, nVidia has ATi outscored by a score of ten to one at least. Hell, they redid their driver package to disable PhysX and CUDA if ANY ATi graphics products were detected in the system. Ask yourself now why you're so quick to condemn AMDbut not anyone else? "LET HE WHO IS WITHOUT SIN CAST THE FIRST STONE!" (I read that somewhere... Heheheh) Reply
  • Haawser - Thursday, August 25, 2016 - link

    Can't wait to buy a 4C/8T Zen to go with my RX 480. Should be plenty fast enough to drive a single card in DX12/Vulkan/VR. Which is my primary reason for still having a PC.

    As much as I like the sound of having 8C/16T, I couldn't really justify it for what I want to do. So as long as the 4C/8T is cheaper than a 4C/8T i7 (and ~as fast) I'll go with the 'mini-me' Zen, and save some money. If I change my mind, or find some use for it, I can always sell the 4C/8T and buy the 8C/16T anyway.

    All I really want from *any* CPU is that it doesn't bottleneck my graphics card. So I think a 4C/8T Zen should be fine, as long as they don't want silly money for it. Like $300...
    Reply
  • Gastec - Thursday, September 8, 2016 - link

    Could you be more of a troll? "primary reason for still having a PC" Like having a PC is such a frikin big deal. It's hilarious really. What else are you STILL clinging to for no real reason? Life? Reply
  • lolipopman - Sunday, September 25, 2016 - link

    Why are your feelings hurt?

    Not everyone has the time or money to shell out 1000s of dollars for a rehashed CPU just to gaze at a few renders.
    Reply

Log in

Don't have an account? Sign up now