One of the ideas behind "Right First Time", as you might guess, is that you ultimately save money and time by making sure that the product you release is as perfect as possible before letting the customer have it. Looking across the phone world, we've seen the original Nokia N95, the new N96, the T-Mobile Android-powered G1 and and a rather large number of Windows Mobile-powered devices, all made available on the High Street before they'd actually been finished.
The attitude of the manufacturers seems to be "Oh well, we must get to market first, it won't matter if it's buggy, we can always fix things later and update customer's firmware". But what about the huge damage to reputation caused by those first 1% of customers being repeatedly let down by flaky software and spreading the word: "Avoid"?
Internally, most of these phones - I'll use the N96 as my example here - must undergo some testing. Nokia probably has test suites that have to be run on each device that gets shipped. And no doubt a handful of real people who play around at weekends and try to spot the bugs they can. But nothing, repeat nothing beats the stress testing that a batch of typical early adopters will put a product through. In the case of the N96, you've only got to load up 20 apps, 1000 songs and a few movies and then start to use the device as a phone all day, every day, to discover instabilities and problems at the rate of about one an hour. Would that be so hard for Nokia to pick up back at base?
In addition to the automated test suites, Nokia would do well to formally go through what they did informally for the release of the N95 8GB - i.e. recruit a number of real world power users in a 'closed beta' (ok, ok, so I was one of them) and tell us to 'break it'. The automated test suites will catch some bugs, but certainly not all. It takes the randomness and unpredictability of a real user wanting to do real things with a phone, if you want to start catching the bugs that will also catch out Joe Public when he buys in the High Street.
Whichever way manufacturers choose to test, it simply isn't acceptable to have them relying on the aforementioned (clueless) Joe to be the guinea pig for a device that's known to be unstable. I'm a geek, you're a geek (probably), and we can handle various errors and crashes on early firmware. We know how to work round problems, we know how to back up, how to reset, how to check for and then update firmware. Joe doesn't.
Ideally, companies like Nokia should do more real user tests in-house. Failing that, go down the closed beta route. But please don't actually make a phone available to the mass market until it's stable and reliable. The N96 is, unfortunately, a classic example of how not to do it. Even the latest v11 firmware is horribly flaky and yet most N96 users in the UK acquired it as a 'finished product' from their network or High Street store and are simply left with the impression that the N96 is 'disappointing'.
Whereas the average power user on AAS would probably call it 'disappointing, but with an awful lot of potential once Nokia have taken the extra 3 months to actually finish its software'.
In the meantime, if Nokia, Sony Ericsson, Samsung, etc. would like to recruit a number of real world phone stress testers, they know where to look.....
Steve Litchfield, 7 Nov 2008
