Preliminary Battery Life

Of course, the display doesn’t make a phone, so in the interest of trying to present more information to someone that might be pre-ordering the phone today, I also went ahead and ran some initial battery life tests. Battery life is arguably the most important part of any phone, so we’ve made some serious efforts to improve our testing here even more than before. As discussed in previous articles, our new web browsing test for 2016 brings a new suite of web pages that more realistically represent what people are viewing today, and adds a scrolling component that adds a steady state load to test the display pipeline in addition to CPU governors/schedulers and their ability to react appropriately to these loads. In order to properly test devices in comparable workloads, we attempt to disable all background tasks and target 200 nit display brightness in order to give a good idea for what relative battery life will be for an average user.

Web Browsing Battery Battery Life 2016 (WiFi)

Given the 28HPm SoC and 10.09 WHr battery, you might think that the Xperia X would perform horribly on our tests, but in practice the Xperia X is actually an excellent performer in our web browsing tests. Of course, we still need to collect more data to see exactly how it performs, but this test tracks fairly well with experiential battery life as it provides a fairly significant CPU and network load rather than a purely display-bound one.

Storage Performance

One of the other major areas of interest for testing here is storage performance. As we’ve seen in phones like the iPhone 6s, the combination of SoC and storage performance improvements can result in significant, perceivable improvements in user experience. Incidentally, due to a general lack of marketing here it’s fairly easy for most OEMs to cut costs here because most reviewers don’t really remark on it and by the time storage performance actually becomes a concern a few months down the road no one is paying attention other than the users of the device. In order to at least try and test storage performance, we use AndroBench 4 to get a better idea for overall performance, but this is really just a first order approximation and hides an immense amount of information that you would want to properly evaluate storage performance.

However, before I get into results it’s worth covering exactly what settings we’ve chosen for this test, and why as I’ve increasingly seen quite a bit of confusion over how we get the results that we do. By default, Androbench 4 is using relatively ridiculous settings. For example, for sequential reads and writes it uses a 32MB block size when the average sequential read/write is more like 256KB or 512KB. It also sets 8 IO threads by default, which to some extent is unreasonable because such a test setting is really mostly designed to favor storage with a UFS controller and using such aggressive settings can actually have a detrimental effect on eMMC performance due to resource contention issues. In general, it also may not make a lot of sense to implement multithreaded IO in programs as single-threaded IO is going to be simpler to implement and in general more likely to be used in real programs.

Internal NAND - Sequential Read

Internal NAND - Sequential Write

Now that we’ve gone over the test conditions, we can look at the actual storage performance, and storage performance is honestly at least slightly disappointing when Xiaomi and other OEMs are shipping significantly faster storage in phones that are hundreds of dollars cheaper. Sony is using Toshiba eMMC of some sort, but I wasn’t able to figure out exactly what it was as the model name isn’t anything that shows up on Google. Of course, it’s possible for a phone to end up being faster than expected despite differences in storage performance, but this sort of cost cutting is concerning when pretty much all of the high-end phones in the market are shipping solutions that are clearly faster regardless of protocol and storage standard.

Initial Thoughts

Overall, the results I’ve seen so far from the Xperia X are concerning. The asking price is well within range of a flagship smartphone, but I’m just not seeing where the phone can justify its asking price so far. I don't really have anything against Sony here, but I'm definitely feeling let down given the praise that I've often seen around Sony phones. The phone is good in some ways, but it's almost unacceptable in others.

To try and explain what I mean, we can start with the display. It's clearly high quality because the display has a fairly impressive gamut for an LCD, and both peak brightness and contrast are quite respectable. Viewing angles are also excellent and the display is laminated quite closely to the cover lens to give the impression that the display is almost painted on the glass. However, the poor calibration really sours the whole experience because just about everything looks wrong on the phone, whether it's logos and UI elements or photos and videos. I'm not sure whether other reviewers just didn't notice or overlooked these issues, but it's a significant issue for me and it's evident to me that this problem has been ongoing for multiple generations now.

Another major concern for me is storage performance, as it appears to be below what we’d expect for a phone of this time and price as well. To some extent I'm willing to give a pass to the HTC 10 for using an eMMC solution because it isn't a clear-cut case of cost-cutting in the hopes that no one would notice, but in the case of the Xperia X it's really quite obvious that performance lags behind the competition. Of course, it'll take some Discomark testing to see just how much of an impact there really is, but this kind of component selection is concerning to some extent.

However, there are some silver linings to be found. In the case of battery life, the Xperia X appears to be noticeably above par, on the order of 10-15%, which just goes to show that battery size cannot be used to approximate battery life when comparing between devices. It's actually quite impressive to see how Sony is getting so much battery life from a relatively small battery, and I suspect that a combination of the Snapdragon 650 and efficient display goes a long way here. Sony definitely deserves credit here for shipping a phone with such high battery efficiency.

Despite such silver linings, I still think it's too early to deliver a definitive verdict on the Xperia X. I like parts of it quite a bit, but other parts are concerning to say the least. To really weigh all of these things and see the balance will require a full review, which should be forthcoming in the near future.

Display
POST A COMMENT

45 Comments

View All Comments

  • RichieHH - Tuesday, May 31, 2016 - link

    You're joking, right? The Z3 was top of the heap in many ways. I still use mine. The Z5 is also a superb phone. Reply
  • Eden-K121D - Tuesday, May 31, 2016 - link

    Most OEM's except for Apple or Samsung cut corners on storage , build Quality or industrial design or all of those Reply
  • arnoudw - Tuesday, May 31, 2016 - link

    "Of course, for those that aren’t familiar with display testing, this may seem a bit out of expectation as in general most reviews will generally state that the Xperia phones have had generally acceptable or excellent displays, but our testing here attempts to avoid relying upon subjective color preferences and rather holds all mobile displays to the same industry-wide standard for content creation on the web."

    I second this. I've done reviews of Sony phones on the largest Dutch website about technology, Tweakers, and howerver I've liked the battery life and feature set of Xperia smartphones of the last years a lot, the display has been a let down.
    Up until the Xperia Z3 it looked like Sony didn't use ips panels at all, resulting in horrible contrast when viewed under even a small angle. And the color calibration has been consistently horrible (for example, http://tweakers.net/reviews/4228/5/sony-xperia-z5-...

    This was the case on all high end Xperia smartphones without exception. All. Of. Them. The camera is the same thing. Sony is making kick ass camera sensors which power all high end phones, but camera samples from Xperia's are always noisy. It's improving, but it's not nowhere near the competition even now.

    It's a shame, because we all know Sony could do so much better, but for some reason have been failing for years.
    Reply
  • BedfordTim - Tuesday, May 31, 2016 - link

    If it is does well a plastic body removes the need for a case and significantly reduces the final size of a phone so I would consider it a positive rather than negative. Reply
  • BedfordTim - Tuesday, May 31, 2016 - link

    Sorry no edit function. I meant "If it is done well" Reply
  • Spectrophobic - Tuesday, May 31, 2016 - link

    I was excited when they announced the X Performance, being SD820 and 1080p. But Sony still can't get screens right. Oh well...

    Looking forward to the OnePlus 3. Or perhaps a Nexus 5X refresh with a 820?
    Reply
  • beginner99 - Tuesday, May 31, 2016 - link

    I like the screen size but it's larger and heavier than phones with similar screens like Galaxy S7 while also having a smaller battery. And I hate on-screen buttons. Look at that last pic. bottom 20% of phone is wasted space with on-screen buttons and bezel. With the galaxy S7 (or S6 or Mi5) you get more screen real estate at lower weight and with a larger battery. Reply
  • techcrazy - Tuesday, May 31, 2016 - link

    While benchmarking is nice way to measure a phone's performance. I would like to learn how it performs in the real world scenario. I saw couple of videos where I saw Xperia X outperform HTC 10 and LG G5 in opening apps. Joshua please focus on the real world performs as well. Thanks in advance. Reply
  • ikjadoon - Tuesday, May 31, 2016 - link

    Yes. I understand how it is "preliminary", but I think it's quite weird to see a focus on sequential performance. When is that a bottleneck in an Android phone? I honestly don't know. Is random 4 KB access still important in day-to-day use? Is that Windows only? What queue depths are important?

    In a Windows SSD environment, I understand: in real-world use, every SSD under consumer applications will perform nearly identically. The only noticeable differentiator is with large file transfers: TLC vs TLC w/ caching vs MLC vs SLC (and then PCIe vs SATA protocol limitations on top of all that).

    But, at QD1-4, pretty much every SSD gives similar enough numbers where normal users won't notice a difference.

    ---

    But, with Android? I've absolutely no idea. Is random performance more or less important than sequential?
    Reply
  • JoshHo - Tuesday, May 31, 2016 - link

    The issue here is mostly that AndroBench 3.6 and 4.0 random read/write results aren't comparable so we have to retest these things. Reply

Log in

Don't have an account? Sign up now