Browser speed comparison - N86, N97, iPhone 3G, T-Mobile G1 and more

Published by Rafe Blandford at 15:14 UTC, July 20th 2009

Rafe compares the relative performance of browsers on various mobile devices including the Nokia N95 8GB, Nokia E71, Nokia E75, Nokia N86 8MP, Nokia N97, Nokia 5800, T-Mobile G1, and Apple iPhone 3G. Which mobile device has the fastest browser? The results may suprise you...

Introduction

Accessing the Internet from a mobile device has been an important trend in mobile for the last 15 years. In the last few years this trend has accelerated such that acessing to the Internet from a mobile device is now one of its key functions. For some users it is even more important than voice calls, although for others it remains a secondary function.

In this feature I look at the key metric of page load time in a number of Nokia's 'S60 on Symbian' devices. To give some context with the rest of the market, I also compare the Apple iPhone 3G and the T-Mobile (Android) G1.

Despite being seen as cutting edge in its early days, the S60 browser is now, in many reviews and commentaries, seen as slow and out of date, but this is not really true. For those that see S60 as something of a backwater, the results below may come as something of a suprise.

How I tested

All the devices were tested over the same WiFi connection (8 Mbps ADSL) from the same location (router about 1 metre away). I tested using WiFi as it was the easiest common data bearer and is likely the best way to test pure browser performance (i.e. the connection speed is not a bottleneck in these tests).

For each device, every site's load time was measured five times. The highest and lowest times were discarded and the remaining times averaged to give the results below.

You should bear in mind that these results should been taken as indicative only. It's not really possible to carry out a fully valid comparison on the live web. There will be variations in dynamic page elements (e.g. advertising), and connectivity (load on host server, routing etc.) that will effect the results. These variations certainly showed up in the raw results; dropping the outliers and averaging over multiple loads helps mitigate against this, but it's not perfect.

It's also worth noting that different sites have different redirect policies for mobile devices. Different versions of the site are served up for different devices (variations in image size are the most common variation), and some sites redirect some devices, but not others. As a result of these variations I opted to use the desktop version of the sites in all cases (except bbc.mobi). In the case of nytimes.com, it was not possible to use the front page as there was no 'view desktop version' option (for Nokia phones), instead nytimes.com/pages/world/ was used.

Futhermore in all the devices tested below it was possible to start reading and navigating around the page before the page finished loading. This is something you would do in real world usage, especially on pages that take a long time to load.

Nokia browser comparison

In the first sets, I compared a number of Nokia devices from the last few years: the Nokia N95 8GB (one of Nokia's iconic smartphones, which runs S60 3rd Edition Feature Pack 1), the Nokia E71 (Nokia's most popular enterprise-focussed device, which also runs S60 3rd Edition Feature Pack 1), the Nokia E75 (an S60 3rd Edition Feature Pack 2 device), the N97 (Nokia's first touch Nseries, which runs S60 5th Edition) and the N86 (Nokia's cutting edge non-touch smartphone, which runs S60 3rd Edition Feature Pack 2).

From these results it is possibel to draw out a number of conclusions:

• The Nokia N86 was the fastest device, followed closely by the N97, with the E75 and E71 further back and the N95 8GB is the slowest.
   
  
• On average, the N86 is roughly twice as fast as the E75/E71 and three times as fast as the N95 8GB.
   
• The most noticeable differences are on larger sized web pages. Mobile websites, such as bbc.mobi, have similar load times across all devices.
   
  
• The degree of difference between the tested devices varies greatly from site to site. The N86 loads the timesonline.com 3.5 times faster than the N95 8GB, but for digg.com is only about 2/3 as fast. Similarly the E71, E75 and N95 8GB have little to choose between them on youtube.com, but the N95 8GB is considerably slower on nytimes.com.
   
  

Here are the tabulated results (averaged times in seconds):

So what's the difference? Hardware may play some role; the N86 (and N97) does have a faster CPU than the E75/E71 (434MHz vs 369MHz). However, the Nokia 5800 (results below), which has the same 369MHz processor as the E75, has very similar in performance to the N86. The big difference is in the software; the N86, and N97 have a newer version, 7.1, of the S60 Web browser (the 5800 has 7.0).

The current S60 browser product was first introduced in a number of 'Series 60' 2nd Edition devices. However it was not until S60 3rd Edition that it became the default browser (when Web and WAP browsing were combined into a single application). At the time it was heralded as giving a 'best-in-class' mobile browsing experience, both because of UI features like 'mini-map' and 'visual history', but also because of its ability to handle the 'full web'. In subsequent versions of S60 3rd Edition there were a few minor updates (as can be seen in the performance of the E75/E71 against the N95). At the same time competing browsers (Safari on the iPhone, Web on Android [both based on the same Webkit engine as S60 Web], and Opera 9) were released such that the S60 browser was no longer cutting edge.

When S60 5th Edition was first introduced, a year and 8 months ago, Nokia noted that it would ship with an updated version of the S60 browser, which would include an updated version of the Webkit rendering engine and performance improvements. The new browser, version 7.0, was first seen in the Nokia 5800 XpressMusic, version 7.1 is used in the N86 and N87 (and includes the WebKit update) and, as the results show, it does offer a significant performance improvement.

But what about S60 3rd Edition devices? How is it that the N86 has the new version 7.1 of the S60 browser? Around the same time that S60 5th Edition was released, Nokia said that a certain parts of the platform would be de-coupled from the S60 platform release cycle. This means that some application updates are not dependent on a new version of S60 being released. The N86's browser is a good example of this.

See the bottom of this page for a video example of these tests.

Comparison with other platforms

So how does the new 7.1 version of the S60 browser measure up against the browsers of competing platforms? This second set of results provides some context as it measure the results against the Apple iPhone and T-Mobile G1 (Android).

From these results, it is possible to draw out a number of conclusions:

• In general the Nokia devices with version 7.1 of the S60 browser out-performed the iPhone 3G and T-Mobile G1. The iPhone and G1 had similar results (with the exception of guardian.co.uk), although the iPhone 3G just edges it.
     
  
• There was significant variation between different websites. All the devices delivered a similar result on theregister.co.uk and digg.com, but the N86 was twice as fast on nytimes.com and timesonline.com and three times as fast as the iPhone 3G and G1 on youtube.com. 
   
• The Nokia browser does better on smaller web pages (google.co.uk). This was also apparent when accessing the mobile version of some of the sites tested (e.g. m.youtube.com in 2.4 seconds versus 3.2 seconds). This seemed to be because the page started loading (a connection was made) quicker on the Nokia devices.
   
  
• The Nokia 5800 was generally a little quicker than the N97, despite running on inferior hardware. This is likely the result of the 5800 having more mature firmware that is better optimised to its hardware than the more recently released N97. The result underlines the importance of software over hardware.
  

Here are the tabulated results:

These results show that version 7.1 of the S60 browser is able to compete very effectively with two of its main rivals.

Incidentally, I suspect that the G1 and iPhone have better Javascript performance than the Nokia browser (based on the results for some of the Javascript-heavy websites). Unfortunately it not possible to judge this easily in a quantitative manner (the S60 browser is unable to complete the Sun Spider JavaScript bechmark test). Perhaps this is where Nokia's browser development team should concentrate their efforts in the next development cycle?

Because I did not have one to hand, I was unable to include Apple's recently released iPhone 3GS in these tests (nor the Palm Pre for that matter). Both the iPhone 3GS and Palm Pre have significantly improved browser performance over the iPhone 3G. Indeed, the speed improvements, which, in the case of the iPhone 3GS, are advertised as being twice as fast, are one of the main selling points of the device. This would suggest that the iPhone 3GS and Palm Pre would have a slight edge over the Nokia N86, especially on Javascript-heavy sites. However the devices would all be in the same league, with the overall results likely dependent on which websites were used for testing. It is also worth noting that the iPhone 3GS (Cortex A-8, running at 600 MHz) and Palm Pre (OMAP 3, Cortex A-8, running at 600 MHz) have next generation chipsets, compared to the N86. All in all, that's an impressive result for the N86 and S60's 7.1 browser.

Of course, there's a lot more to a browser that page load performance. The UI and other functionality of the browser is also important; for example, the zooming controls of S60 5th Edition are inferior to those of the iPhone. On the other hand the S60 browser has a basic RSS client and supports Flash content (including video).

The need for speed? Opera Mini and Skyfire

If page load time is your most important priority then the native browsers of each platform may not be the best solution. Proxy-based browsers should, because of their architecture, give significantly better results. Here is a set of results comparing the native browser with Opera Mini and Skyfire (all tested on the N86).

The message here is clear. If you're purely interested in speed then Opera Mini is the best choice. Opera Mini does have its limitations though, especially when dealing with dynamic sites and video. Skyfire may not be quite as fast as Opera Mini, but it should deliver the 'full' web experience. Skyfire also has its limitations and has a non-standard interface.

In both cases, the speed advantage is gained thanks to two factors. Firstly, offloading some of the rendering work to the server and secondly, less data transfer (a combination of proprietary methods and compression). Hence Opera Mini and SkyFire are particularly good solutions to use if you're on a slow data data connection such as GPRS or EDGE.

In terms of platform, S60 on Symbian and Android have an advantage over the iPhone here in that they allow the installation of third party browsers. For example, Opera Mini and Skyfire are available via the Ovi Store, whereas Apple blocks third party browsers from appearing in the iPhone App Store.

Here are the tabulated results:

Conclusion

It is important to realise all these tests offer the best case scenario. To get similar results, you'll need to be running over a high quality WiFi or HSDPA connection. The real world reality is, of course, often very different. Nonetheless, these do offer a useful insight into browser performance.

They key take away from these article is that version 7.1 of the S60 browser is a major step forward in performance and rendering terms. It is faster than the iPhone 3G and T-Mobile G1. It is likely in roughly the same league as the iPhone 3GS and Palm Pre (despite the Nokias running on lower-resourced, previous generation, hardware).

The improvement is most apparent on non-touch devices (after all, the touch devices never had the 'old' browser). Despite being released just a few months apart, the E75 and N86 have very different browser performance because they have different versions of the browser. Given the size of the improvement over the previous versions of the S60 browser (comparable to, or even greater than, iPhone 3G to 3GS) it is surprising it has receieved relatively little comment or publicity.

S60 5th Edition has version 7.1 of the S60 browser as a standard component, hence it ships with the Nokia 5800, Nokia N97, Nokia 5530, and Samsung i8910. For S60 3rd Edition, version 7.1 of the browser currently ships with the the Nokia N86. The new browser will likely ship in forthcoming S60 3.2 devices, and earlier devices may receieve the browser update via a firmware update.

Video - Nokia N95 8GB vs Nokia E75 vs N0kia N86 - Browser Speed test

For the benefit of the video, I recorded the devices loading the same page side by side. Since there are several devices using the same connection (and starting at slightly different times) the exact results are less reliable than those above, although the same general pattern can be seen.

This video is also available in HD resolution.

Categories: Comment, Software
Platforms: General, S60 3rd Edition, S60 5th Edition

Quote:

Originally Posted by Unregistered It was an Iphone 3G,right? No 3Gs?! Why? You are comparing new Nokia devices with an "old" Iphone 3G. Do the test again with the new Iphone 3Gs. Be fair-Play true. (I'm not an Iphone Fan!)

We all know the new iphone is much better. The point here is that only a few weeks ago we only had the previous iphone to compare to. Yet even then so many people wrote the s60 browser off as slow. But this test clearly shows that the browser is pretty good and fast. This test could have been done only 1 month ago when there was no iphone 3gs yet people back then were still claiming s60 browser sucks. When it clearly doesn't.

My 5800 is actually slightly faster than even the n97. Very impressive! :-)

Quote:

Originally Posted by Unregistered It was an Iphone 3G,right? No 3Gs?! Why? You are comparing new Nokia devices with an "old" Iphone 3G. Do the test again with the new Iphone 3Gs. Be fair-Play true. (I'm not an Iphone Fan!)

Did you actually read the article? The author talks about the 3GS and notes its performance would be probably be similar or better than the N86, which I think is being generous given some of the results.

Quote:

Originally Posted by Unregistered You actually showed also one of the main problems of Nokia's S60: Once the phones are sold the companies barely care about the phones or just keep bugfixing. The main advantage of the iPhone is that even the first 2G iPhones got the new firmware. .

I've had the mid-range 5800 since December, and had V11, V20 and 21 firmware

So what are you suggesting?

Quote:

Originally Posted by bheetebrij I second a question asked by someone else: Does this version 7.1 ship on future models of Nokia E and N series phones? Specifically, I am eyeing up the Nokia E72... which has a 600Mhz processer I just read on Forum Nokia. That + version 7.1 of the browser + optical navikey should make for a very decent browser experience... What do you guys think? Rafe... fantastic article!

yes the new browser ships on all the newest phones. and thank god it does. the latest browser performance has been incredibly improved compared to the n95 days. i just got the update to the 5800 firmware and the browser was suddenly incredibly fast over wifi. they definitely must have done some rendering fixes because it's just as fast as in Rafe's video in this article. and when i compare it next to the android G1 and iphone 3G it completes the page loading in the same amount of time. well done nokia. and considering this performace is happening on a lower spec CPU, just imagine how good it could be on a 600mhz CPU.

92 Comments / Post New Comment