The 2020 Browser Battle: Surfing With Speed
by Brett Howse on September 10, 2020 8:00 AM EST- Posted in
- Software
- Browsers
- Mozilla
- Google Chrome
- Microsoft Edge
Scripting Performance
As websites have gotten more and more complex, the browser’s scripting performance has continued to be extremely important. To quantitatively analyze this performance, we turn to several web-based scripting tests. The specific tests we normally leverage are being expanded to include a wider variety of workloads to get a better handle on how each browser can handle them. And, just for fun, we are including some incredibly old scripting tests just to show why they are not used any longer.
As far as hardware we are turning to the Microsoft Surface Laptop 3, featuring the Intel Core i7-1065G7 which is based on Intel’s 10 nm Ice Lake platform. It is set to run in its maximum performance mode during these tests to give the CPU its maximum boost.
WebXPRT 3
Rather than focus on a specific function, Principled Tehcnologies developed WebXPRT to simulate several real-world tasks with HTML5 and JavaScript, including photo enhancement, OCR scanning, AI, and more. If you would like to delve into how this benchmark works, and how the score is calculated, check out the report on WebXPRT as well as scoring.
In this first test, Firefox is the clear winner, easily outclassing all of the Chromium browsers which are all within a few points of each other. Interestingly, classic Edge and new Edge are almost tied. Internet Explorer does not fare as well.
Speedometer 2.0
Created by Apple’s WebKit team, Speedometer 2.0 is a relatively simple test which involves adding, completing, and removing items in a to-do list. Speedometer 2.0 was created to measure the browser responsiveness. The test performs the to-do tasks with some of the most popular JavaScript frameworks available, and the score is a geometric mean of the different implementations of the to-do app, and then an arithmetic mean of the geometric means for each iteration. Apple warns against comparing the scores of different browser engines as they will do different work, which is why we use the same browser in hardware reviews.
For this test with different browsers, the scores are actually fairly similar for most browsers, although classic Edge and Internet Explorer both do poorly, with the latter not even loading the page. Interestingly, Chromium Edge does outperform the other Chromium browsers by a few points here.
JetStream 2
Combining JavaScript and Web Assembly benchmarks, and featuring 64 subtests, JetStream 2 includes part of some older popular scripting tests such as SunSpider and Octane 2, and each benchmark is weighed equally in the results with a geometric mean over each individual benchmark score to provide the final JetStream 2 score.
Again, we see Microsoft’s Chromium Edge slightly edge (pun intended) the other Chromium browsers. Mozilla Firefox is not quite as fast here as the Chromium browsers, and classic Edge and Internet Explorer both failed to load the test at all.
Octane 2.0
The original Octane was released in 2012, and the later Octane 2.0 was one of the most popular scripting performance benchmarks around. It focused on peak scripting performance, and as is the case, over time browser scripting engines had optimized their code to solve many of the bottlenecks that Octane was originally designed to put a spotlight on, and as such, the benchmark was retired by Google in 2017. New bottlenecks that were not being represented in Octane were common in popular sites, and this is a typical scenario with all benchmarks, but the web moves at a rapid pace.
Here we see some wide variation in results, with Chromium Edge again at the top of the stack, followed closely by Opera, and of course both are based on Chromium, but Google Chrome was not as performant on this test. All of the Chromium browsers did outperform all other browser though, with Firefox slightly behind Chrome here. Classic Edge was a bit further back, and of course Internet Explorer with its very outdated scripting engine was far behind the rest, although at least this test did run in IE 11.
Kraken 1.1
Created by Mozilla, Kraken runs several test cases from real-world applications and libraries, and includes 14 subtests. Results are reported in milliseconds to complete the tests, and as it is time-based, lower results are better.
Mozilla Firefox is our fastest browser in this test, although only by a hair. Again we see Microsoft’s implementation of Chromium Edge outperform the other Chromium browsers, although all of them are in the same range of results. Classic Edge takes a definite step back in performance here, and Internet Explorer trails the pack by a wide margin.
SunSpider 1.0.2
The last version of SunSpider was 1.0.2. Once this was likely the most popular scripting test around, and likely due to that, developers optimized their code to quickly overcome this test, and then found ways to work around it. SunSpider was retired and is no longer a relevant test, but since it is not too difficult to run, it was added just to see how modern browsers perform on it.
Interestingly Internet Explorer, and the newer browser based off of its code, classic Edge using EdgeHTML, both perform suspiciously high on this now retired benchmark. Whether that is due to Microsoft’s Chakra scripting engine still containing “optimizations” for SunSpider is somewhat irrelevant, as not only has the benchmark been retired, but both Internet Explorer and classic Edge are both also defunct as well.
Summary
With so many browser makers now using Chromium as their base, and contributing to its code, it should not be a huge surprise to see the Chromium browsers offering the highest scores here. Firefox does very well though, only a couple of points behind. Classic Edge, which has not been developed for over a year, is still quite compatible, but poor Internet Explorer is way behind the rest.
121 Comments
View All Comments
biofrog - Thursday, September 10, 2020 - link
Thanks, very interesting.I thought I'd try my real-life test opening a 49MB HTML log file. Time to finish loading was:
Chrome: 14 minutes 46 seconds
Firefox: 7 seconds
Edge: cant even open local files
IE: 12 seconds
Nope, still not going to use Chrome until they get that fixed. It used to work around v50-ish. At roughly 12,657 times slower than Firefox, it seems Chrome isn't feasible for my work.
Khenglish - Friday, September 11, 2020 - link
Can we get a RAM usage comparison for similar tasks? Historically I used to see Chrome use the most memory, and edge classic use the least. Lately Firefox seems to be using ridiculous amounts of memory. It'd be good for a more in depth comparison.RickyBaby - Friday, September 11, 2020 - link
I'll echo the sentiments of several others. My #1 concern is the ability of a browser to protect my privacy and block ads. NOTHING ELSE COMES CLOSE. Why doesn't AT or any other site respect our wishes and provide the details we actually want to know ? And, yes, you already know the answer. GOOGLE IS EVIL. Apple, MS are no different. Is it true that FF is owned by the Chinese and our data is mined and sent there ? That's what we really want to know ... along with which plug-ins work best is stopping the spying ? How about a review of TOR browser ? Yeah. Right. We get it. AT and all the other tech sites are owned by the same EVIL we are trying to avoid. Pretty much figure that I just got banned. And since I'm going down ... FB is EVIL too. Twitter is EVIL. Both ban anyone who isn't complicit to their EVIL. What the hell happened ? Speaking for the average geek ... we want our internet back and yes it was stolen from us. Thanks.The Garden Variety - Friday, September 11, 2020 - link
So, other than that, Ms. Lincoln, how was the play?Tewt - Thursday, September 24, 2020 - link
I agree, RickyBaby. I think there is too much curation on the internet. The fun of discovery of new sites and ideas appears to have vanished. Now, most of my results always bring back just the top news(political, tech, life, etc) sites or retailers. It feels like there are no new discoveries. I used to be able to go back 10, 13, 20 pages and there would be a ton of different websites. Now, each page is mostly the same. That is boring and stifling.lmcd - Friday, September 11, 2020 - link
None of these benchmarks would even be relevant if Mozilla had the muscle that the Chrome team does. Instead, all of our JS libraries would interop with asm.js utility libraries, and Chrome's weak optimizations there would lead to massive performance wins for Firefox.It's disgusting to me how the Chrome team gets to pick and choose the standards that win. Their selections are regularly wrong, sometimes outright harmful.
Sivar - Friday, September 11, 2020 - link
I like Edge overall, but it occasionally freezes when loading a website. The browser is still responsive, the "Loading" notification appears, but that tab is permanently defunct. It will never, ever load the page. It won't respond to a refresh request. I have to close it and open the same page in another tab, which has so far always immediately loaded.This happens on both Windows 10 and iPhone, with or without extensions.
six_tymes - Friday, September 11, 2020 - link
the new edge has been the fastest browser since December of 2019.OreoCookie - Sunday, September 13, 2020 - link
Edge has replaced Chrome on my Mac. Their UI isn't as good as Safari, but it is a capable and fast browser. The only criticism — which isn't really Microsoft's fault — is that the number of ads in Youtube has exploded. That shows me *how much* Google syphoned off my personal data in return. Not cool and I am glad to suffer a tad to have a shred more privacy.GeoffreyA - Saturday, September 12, 2020 - link
Have been using Firefox for a few years and am quite happy with it. Used to use Chrome on the desktop, though admittedly, still on my phone (along with Opera). As for IE, with all its faults, I think many, like myself, feel a bit nostalgic towards it.