Testbed Setup and Testing Methodology

Thunderbolt's adoption, till now, has mainly been driven by the Mac ecosystem. Solutions for DIY PC enthusiasts are not widespread. GIGABYTE has a few motherboard models with Thunderbolt integrated, while Asus's Thunderbolt-capable motherboards need an add-on PCIe card. Our Windows-based testbed for direct-attached storage (DAS) systems is based on the latest Z97 chipset from Intel. The build originally started out as a multimedia editing machine. The requirements of a DAS testbed fit in nicely with the original purpose of the build. The following table details the configuration of our testbed.

AnandTech DAS Testbed Configuration
Motherboard Asus Z97-PRO Wi-Fi ac ATX
CPU Intel Core i7-4790
Memory Corsair Vengeance Pro CMY32GX3M4A2133C11
32 GB (4x 8GB)
DDR3-2133 @ 11-11-11-27
OS Drive Seagate 600 Pro 400 GB
Optical Drive Asus BW-16D1HT 16x Blu-ray Write (w/ M-Disc Support)
Add-on Card Asus Thunderbolt EX II
Chassis Corsair Air 540
PSU Corsair AX760i 760 W
OS Windows 8.1 Pro
Thanks to Asus and Corsair for the build components

We decided to prepare the testbed with 32 GB of memory in order to make a large enough RAM drive available for our file copy tests. Similar to our NAS testbed, we used OSFMount to allocate 20 GB of the DRAM for a RAM disk partition.

The internal layout of the Z97-Pro (Wi-Fi ac) board is presented below.

It should be noted that the Thunderbolt add-on card can only be connected to PCIe lanes hanging off the PCH (and can't be connected directly to the CPU). Apparently, this is a requirement for vendors to obtain Thunderbolt certification from Intel. In the above layout, we see that the add-on card can only go to the bottom PCIe slot (PCIEX16_3). In addition to the hardware restriction, the BIOS actually operates this slot in x1 mode, with the other three lanes being shared with the ASMedia USB 3.0 ports in the rear panel and the x1 PCIe slot (PCIE X1 SLOT4). The BIOS setting was altered to operate the x16 slot in x4 mode (maximum possible bandwidth for the card). The BIOS entry also has a PCIe cache line setting for Thunderbolt. It was left at 32 bytes (the default) for our evaluation.

Our testing methodology for DAS units takes into consideration the usual use-case for such devices. The most common usage scenario is transfer of large amounts of photos and videos to and from the unit. The minor usage scenario is importing files directly off the DAS into a multimedia editing program such as Adobe Photoshop.

In order to tackle the first use-case, we created three test folders with the following characteristics:

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray (the same that we use in our robocopy tests for NAS systems)

For the second use-case, we take advantage of PC Mark 8's storage bench. The storage workload involves games as well as multimedia editing applications. The command line version allows us to cherry-pick storage traces to run on a target drive. We chose the following traces.

  • Adobe Photoshop (Light)
  • Adobe Photoshop (Heavy)
  • Adobe After Effects
  • Adobe Illustrator

Usually, PC Mark 8 reports time to complete the trace, but the detailed log report has the read and write bandwidth figures which we present in our performance tables. Note that the bandwidth number reported in the results don't involve idle time compression. Results might appear low, but that is part of the workload characteristic. Note that the same CPU is being used for all configurations / will be used for future DAS reviews. Therefore, comparing the numbers for each trace should be possible across different DAS units.

Introduction Performance Evaluation - 2big Thunderbolt 2
Comments Locked

42 Comments

View All Comments

  • ssj3gohan - Wednesday, July 9, 2014 - link

    So... thunderbolt is slower, more power hungry and more expensive than USB 3.0. And it's effectively unavailable. I can see why Intel integrates USB 3.0 into their chipsets and not Thunderbolt.

    Of course, I understand that this is a function of volume; Thunderbolt doesn't really sell and is an Intel exclusive whereas USB 3.0 controllers are made by every man and his dog, so it's much more optimized both in cost and data stream efficiency, not to mention driver compatibility. But this still begs the question: why even include Thunderbolt? It's objectively worse than USB 3.0 and just needlessly increases the price of both Apple and peripheral products.
  • mmrezaie - Wednesday, July 9, 2014 - link

    I would like to see the benchmarks under Mac OS too. They don't rely on bios drivers for thunderbolt and use their own. I think Linux will have the same driver later this year but I think on Mac thunderbolt driver is more optimised.
  • Essence_of_War - Wednesday, July 9, 2014 - link

    Agreed. I'd really like to see a USB 3.0-Thunderbolt shoot-out under OS X and Linux as well!

    I don't know if this is possible Ganesh, but if you could get access to a mac with thunderbolt, it would be pretty neat to see! :)
  • techwiz2100 - Wednesday, July 9, 2014 - link

    Yea I'm a little concerned by these numbers, I mean... Isn't the Mac Pro supposed to be able to drive a 4K display over TB2? I would imagine that's a much more bandwidth heavy application than file transfer. Also maybe the devices themselves or the add-on card are the limitations?
  • Essence_of_War - Wednesday, July 9, 2014 - link

    Yeah, I think it's pretty clear that in no case is the theoretical limit of TB being saturated.

    It's possible that the speed differences are within the margin of error, but my first guess would be some sort of driver/controller issue for TB on Windows.
  • M/2 - Wednesday, July 9, 2014 - link

    I'm a Big Mac Mini fan... I've been trying to rationalize TB, and just can't quite get there. I use external drive over the network for media and backups. I wound up buying a 4-bay USB 3.0 box for $250; with 4-4Tb drives, I have 12Tb of RAID 5 storage for under $1000. So I really don't get the price tag on a 2-bay TB drive.

    I get 230 MB/s on Blackmagic. That's not that much slower than my buddy's Promise 2 RAID, the reason being the spinning drive are the main limiting factor. And, 30+ % cheaper. And 230 is fast enough for anything I need . You really need 6 bays or SSDs to take advantage of TB.

    If I had to do it over, I'd consider the OWC thunderbay box. $450 vs. $250 for the Dyconn Quartz4 box I bought. Just depends what you're using if for. I'm almost there!
  • M/2 - Wednesday, July 9, 2014 - link

    I may be off on the TB speeds. ..just looked at a review on owc , it's twice the speed of USB 3 on RAID 5 ... Like I said, I might decide different today
  • ganeshts - Wednesday, July 9, 2014 - link

    As I mentioned in the concluding remarks, for these particular devices, TB is useful only in the daisy chaining scenario. Otherwise, USB 3.0 is going to deliver better performance.

    Apologies for not testing with Macs. I should have probably noted in the very beginning that we are going to start monitoring Thunderbolt performance on Windows from my side. For Mac-based coverage, Anand is the best bet (for example, the LaCie Little Big Disk review that he put out earlier this year -- which I have also linked in the article).
  • M/2 - Thursday, July 10, 2014 - link

    Agreed! USB 3.0 beats any 1 or 2 drive configuration (IMO), simply because because the drives can 't fill the TB bandwidth (unless you're using SSDs). Sure, TB is cool, but adding USB3 would make a more versatile package.

    FYI, I see the OWC thunderbay price is now $500, up $50, people must be buying them. Still too much of a premium IMO (unless you're using SSDs and/or really need 500+ Mb/s)
  • ciparis - Friday, July 11, 2014 - link

    You really need to include the Mac numbers; otherwise, nobody will be holding PC manufacturers to task in making these devices perform remotely near their potential. The current situation is abysmal, and with zero reason other than near total incompetence.

Log in

Don't have an account? Sign up now