The short answer is yes.
There are just too many lines of code in the smartphone operating systems for weird errors to pop up. And they’re rarely obvious (except with hindsight) and are likely to be as a result of lots of smaller factors suddenly lining up to trigger an... interesting… software configuration; like the alarm bug on the iPhone, like the Nexus 1 forgetting its screen calibration, like the deferred message bug in Symbian^3, look around and you’ll be able to populate quite a large list of small quirky issues.
Being fair, this is not a “Symbian is better than iOS/Android/WinPhone7/etc” article. It’s here to add some realism to the coverage of the iOS alarm bug and to think about how it happened. People are likely to ask “didn’t Apple think to test the alarm clock with a 2011 date?” and that’s unfair. The functional testing regime probably tested the alarm clock (and it worked) and tested the date handling code (and it worked) so the logical thought process in the minds of the tests was work + work = work.
The modern smartphone OS is incredibly modular – you can see that in the number of core “applications” that are present. Each of these are tested in their own environment, but they do have to trust that the other elements used are also tested. Now take all the apps you have, and all the bits of base OS code they need to talk to (data channels, alarm servers, user input, listening for the “pause there’s a call coming in” message, etc) and you’ll begin to realise that the combinations of checking every piece works with every other piece results in a ridiculous number of checks to be undertaken that would take longer than the lifespan of the universe.
Oh and then you’d need to test that all these combinations work when you install Angry Birds. And when Angry Birds is running in the background. And when it’s installed on the memory card instead of the internal memory. So that’ll be four more universes needed.
And this is why testing has switched to the modular approach. And unfortunately that is going to lead, in rare circumstances, to the issues we’ve seen in Android, iOS and, yes, Symbian. Considering the complexity, I’m surprised that we don’t see more of these issues.
The answer is not a raw increase in testing, because even with a larger budget you still couldn’t test everything. What’s needed is confidence in the modular design of the operating system, and a fast way to patch smartphones when issues are spotted in the wild.
We’ve already seen Nokia deliver “software improvements” outside of a full firmware update over the air, and the new Symbian^3 calls home automatically to check for these updates, as do Android handsets and (on the desktop) the iTunes software (in the case of iPhones). That’s part of the equation, but the ability to spot these flaws as soon as possible is just as important.
If you have a flaw in your smartphone, what do you do? Most people are going to be calling the support line of the network first, and then possibly the retailer who sold it to them. But that information has to flow back to the manufacturers smoothly and quickly, and then analysed to spot any pattern.
Throw in the fact that any fix then needs some functional testing to make sure that it’s not introducing further problems into the already complex system and a “quick” fix might take a some time, especially if it’s over a popular holiday.
The world has come to rely on its technology without fully understanding how it actually works – and that’s led to the current furore around the alarm bug. I’ve no doubt we’ll see more issues like this pop up occasionally, because that’s just how things work in the modern smartphone.
-- Ewan Spence, Jan 2011.