At the Smartphone Show last month in London S60 gave developers a preview of three very important technologies that are to be integrated in to S60: Touch UI, Sensor Framework and UI Accelerator Toolkit. All three technologies have a key role to play in the advance of the user experience with S60 phones. In this article I will review what we know about the technologies and consider their implications.

The announcement at the Symbian Smartphone Show was not a formal unveiling of the new version of S60 or about a specific product; Nokia provided few technical or implementation details. Instead the announcement was a platform message to make developers aware of what is coming next. The new technologies are part of the next version of the S60 platform (version 5, presumably S60 5th Edition) and this means we will see any number of devices and applications taking advantage of the new technology. The message from Nokia was that this is about the evolving the leading mobile platform. Remember there are more than 125 million S60 devices in the market and another 70-85 million will ship next year. These new technologies will be in millions of devices in relatively short order and those that can take advantage of them first will benefit most.

Touch UI

Ever since the Nokia 7650 S60 (then known as Series 60) has been a soft key driven user interface. Amongst smartphones as a whole this has always been the dominant device style. This has partly been driven by Nokia's belief that such devices should be usable one handed. However there has always been a consistent demand for touch screen based devices as demonstrated by Sony Ericsson UIQ phones and Windows Mobile Professional based devices. Such devices were initially stylus based and required two hands to operate anything more than the basic functions. However devices such as the Treo range from Palm and more recent Sony Ericsson and Windows Mobile Professional phones have demonstrated it is possible to create touch based devices that can also be use one handed. Touch as an input mechanism has also received a lot of attention thanks to the recent launch of phones such as the Apple iPhone and LG's Prada handset. In adding touch support to S60 Nokia is responding to a trend that has been developing for the last five years.

For S60 the driving force behind adding touch screen support to the platform is to allow its licensees to develop a wider range of S60 powered devices. Clearly though this is major step forward for S60 as it allows licensees to create an array of devices that are clearly differentiated from the traditional S60 phone design. Nor is it a simple matter of adding support for touch screen hardware; more significant is the optimisation of the S60 UI for use with either stylus of finger touch input. As a taster of things to come, at the Symbian Smartphone Show, Nokia showed a video demonstrating a concept touchscreen device running the S60 Touch UI.

Nokia have only offered a few glimpses of the UI itself, we will have to wait for a formal unveiling before any initial assessment can be made. However a certain amount can be ascertained from the screenshots on the S60.com website. As Nokia indicated the UI has a familiar S60 look and feel with the familiar soft key commands at the bottom or side of the screen. Some phones will provide the traditional hardware keys for these as shown on the concept device in the demo video; others will be purely touched based. To allow for finger touch it appears the amount of screen space that these occupy have been increased. Equally familiar are the pop-up menu which gives access to further commands. A notable addition is the presence of a toolbar to access the most common functions. For example in Messaging read mode these are set to reply, forward and delete; in Gallery's slideshow mode the toolbar provides play, forward and back controls.

In terms of input both an on-screen 'hunt and peck' keyboard for use with a stylus and handwriting recognition input have been shown in the demos. It seems likely there will also be a finger touch keyboard too (as seen on the N800) and a finger based dialler. With existing support for traditional keypads and full QWERTY keyboards each device is likely to mix and match different input methods according to the targeted niche or market. There is great regional variation in what might be considered the best solution: in the US QWERTY keyboards are popular, while Asian markets may prefer stylus based hand writing recognition because of their logographic or syllabic character.

One very important detail is that the new version of S60, including the new Touch UI, will be fully backwardly compatible with S60 3rd Edition. This means that applications written, using standard UI components, for current S60 devices should also run on the new touchscreen (and non-touchscreen) devices powered by S60 5th Edition. Nokia appears to have a similar approach to UIQ where there is a single code based for applications regardless of the input mechanism or form factor (an agnostic UI). Developers are able to optimise their applications for particular combinations of inputs and form factors, but this is not a requirement. This is similar to the way that S60 applications can be optimised for the wider screen of the E90 using an SDK plug-in. For UIQ the transition to an agnostic UI occurred by moving from touch based device to non-touch devices the opposite is true for Nokia. I suspect Nokia will have an easier time of making this transition because it is adding something rather than taking it away. However I would expect that, initially, there will be fewer applications optimised for touch screens which will have consequences for the overall user experience.

Support for tactile feedback is built into the S60 touch user interface. Tactile feedback gives a physical pulse when tapping or clicking on the screen. Current implementations of tactile feedback tend to be quite crude (e.g. phones simply activating their vibrate function, or game console controllers with a rumble pack). Nokia has been working to improve the capabilities of tactile feedback technology in a project code named Haptikos (Finnish for touch). Rather than a generalised feedback it seeks to localise the feedback to the key being pressed and mimic the feeling of pressing a physical key. Instead of a general vibrate the user will feel the tactile feedback (with a distinct up and down response) under the key that they have pressed. Nokia believe tactile feedback is important because it more closely replicates the feeling of pressing a physical key and therefore makes touch screens easier and more pleasurable to use. At the recent Way We Live Next Event (http://www.nokia.com/A4632105) Nokia were demonstrating this technology and a blog post on the Red Ferret journal explains that it is planned to be included in Nokia's forthcoming touch based devices and has some additional details (http://www.redferret.net/?p=9533).

Sensor Framework

The Sensor Framework brings standardised access to sensors (e.g. motion and orientation, proximity and light) to the S60 platform. This gives you an additional way to interact with the phone. Initially the most common usage will revolve around accelerometer and position sensors with motion triggering certain events on the phone. For example turning a phone over might silence a ringing phone or turn off an alarm. Tapping the side of the phone while in the music player could skip forward to the next track, while tapping the front of the phone would play or pause music. There are some S60 phones with sensors built in already available and these also demonstrate what is possible. The Nokia 5500 used an accelerometer to create a pedometer application and to detect a tap on the side of the phone (with user configurable consequences). Recently it was made clear that the N95 had an accelerometer and a variety of basic concept applications have been developed as a result including automatic screen rotation, a tilt meter for cars, and a light sabre program. 

Other examples can be found from phones already on the market, the iPhone touch screen deactivates when you hold it up to your face, and some Sony Ericsson phones can be shaken to stop or play music. On the S60 platform phones such actions will be user configurable, although the extent of this is largely dependent on what the phone manufacturer decides to allow, furthermore the options available in any given phone will vary depending on the type and sensitivity of any sensor used. However because the Sensor Platform offers an easy way to implement such offerings it is should become common place on S60 phones, to the extent that this represents the next major step forward in the way we will interact with our phones. In many ways it offers a wider range of and more compelling use cases than touch interaction and may have a greater impact because of its broader applicability.

The framework is designed so that manufacturers can add additional sensors which might allows different use cases. For example many phones already include a light sensor to adjust the brightness of the screen. If this was made available through the Sensor Framework it would be possible for applications (e.g. navigation) to automatically switch to a night mode (dark background) to reduce glare.

The Sensor Framework is also a key addition for third party developers. In the same way that the Mapping Framework enables access to location and mapping information and functions the Sensor Framework will allow developers to build their applications to take advantage of the sensors built into the phone. The benefit of the Sensor Framework is that there will be a standardised way for developers to access and take advantage of them. As with most 'new' technology the most innovative use cases have yet to be even thought of. 

UI Accelerator Toolkit

The new UI Accelerator Framework which is built on top of Symbian's FreePlay technology is designed to allow manufacturers to rapidly develop graphical effects, scalable layouts and new types of interactions. This includes things like transitions and animation within the UI. At such there is no discrete functionality, but it could be used to highlight or help explain certain functionality on screen in a pleasing away. Eye candy or the 'wow' factor is an important part of the consumer user experience. It will also for greater personalisation of the device, something that has been high up the list of user demands in feedback Nokia has received.

http://www.allaboutsymbian.com/images/features/N95/screenshots/Screenshot0033.jpg

The multimedia menu in the Nokia N95 classic is an example of things to come. Here there is lots of movement with an ever changing background and floating animated icons all of which can be customised by the user. Building the UI Accelerator into the framework will improve the performance of such graphical fripperies, but will also make them quicker and easier to implement.

A note on S60 Design

The design of S60 is built around a user experience framework. The framework is based around several key areas which can be thought of as fundamental building blocks in a mobile platform, they include: input, output, functionality, performance and design/usability. Each of these areas may has various components and enablers and together constitute the platform. This gives the platform flexibility and the ability to continually evolve as new versions are released. 

The introduction of multiple screen resolutions and orientation (output) and the addition of QWERTY keyboard support (input) in S60 3rd Edition are good examples of how this works. Many of the technologies supported by S60 are optional, for example screen rotation and QWERTY keyboards are supported, but they are not used in all products.

The same applies to the new Touch UI; at is simplest it is just another input mechanism. Clearly Touch UI is more than that since there design changes to the UI of S60 to optimise for touch usage and there is the tactile feedback (output) component too. Similarly the Sensor Framework can be considered another input mechanism while the UI Accelerator Toolkit looks to improve performance and enables new graphical effects and functionality. Nokia have underlined that the Touch UI is optional. It is, therefore, best to see it as enabler of a new family of touch based S60 devices. In the future we will continue to see traditional keypad / soft key based S60 devices, but we will also see touch and keyboard based S60 devices and touch only S60 devices too.

A note on Nokia's Touch History

It is worth realising that Nokia has something of a history with touch based devices. The Nokia 7700 and 7710, which ran Series 90 on Symbian, are the best examples. Series 90 was a fully touch based UI and had grown out of a joint Nokia / Psion project (though in one sense its roots can be traced all the way back to the original Psion organisers). As a UI it was ahead of its time and the Nokia 7110 showed much promise, but the hardware was under powered and the software was sluggish and never quite felt finished. In relatively short order Nokia killed off Series 90, partly because of their belief that phones should be one-handed devices and perhaps realising the impracticality of maintaining multiple platforms. 

In 2004/5 Nokia announced that the functionality of Series 90 would be rolled in to S60 and it was expected that this would be completed in 2006. Clearly  it has taken longer than originally envisioned, although whether that was because the scale of the task grew to include other objectives or was under-estimated or because there was a lack of urgency is open to debate. While the focus is on functionality it is likely that some of the UI concepts of Hildon may also filter down in S60 (as an aside Hildon also lives on elsewhere - as the UI for Nokia's Maemo based Internet tablets). 

The upshot of this is that touch is not new to Nokia. The new S60 Touch UI will draw on a pool of prior knowledge and experience. While it is probably fair to say that the enthusiasm generated by the iPhone has given Nokia pause for thought and the S60 Touch UI extra impetus it should not be seen as a direct response to Apple's entry into the mobile market. Nokia will be keen to bring its own offerings to the market, but they also will bear the hallmarks of the world's most successful mobile manufacturer: the lessons learned in 20 years in the mobile business, the philosophy of the open platform, and the paramount aim of connecting people.

Frequently asked questions

The new technologies, in particular S60 Touch UI, have been a hot topic. Here are some of the questions that have come through the AAS mailbox in the last few weeks.

Q: When will we see S60 Touch devices?
A: Nokia says the second half of 2008. It is possible the announcements will come before then.

Q: What sort of devices will there be?
A: The whole point is that S60 enables manufacturers to create all kinds of devices, and there are ever increasing number of possible permutations and thus this question is completely open to speculation. It seems likely that initial devices may include both keyboard based devices (like the Sony Ericsson W960 or Treo 700) and slate types devices (like the iPhone). Bear in mind that both finger touch and stylus touch is supported so that allows for further permutations. Even though its not required it seems likely that the majority touch devices will have at least directional controls (which can be used to navigate the UI). Remember there will still be a large number of variety of non touch devices; the Sensor Platform and UI Accelerator Toolkit technology will be more widely implemented that Touch.

Q: Will other S60 licensees produce S60 Touch devices?
A: There is no certainty in anything. All S60 licensees (Nokia, Samsung, LG, Lenovo) will have access to the new version of the platform. However, reading between the lines, the most likely candidate, aside from Nokia, is Samsung.

Q: How will UI element y work?
A: The simple answer is that we don't know yet. Nokia have deliberately been fairly evasive about the exact nature and features of the S60 Touch UI. You can have a look at the screenshots on the S60.com website which gives some clues. One of the aims is to create a UI familiar to existing S60 users, therefore there are not going to be wholesale visible changes, rather it will be about enabling touch interaction (e.g. 'thicker' softkey buttons).

Q: What about capacitive touch screen and multi-touch?
A: We don't know. Nokia say they are aware of the various options available and are looking to create the optimum user experience (that is marketing speak for 'we're not telling you yet').

Q: What happened to S60 4th Edition?
A: 4 is an unlucky number in some Asian countries and thus is rarely used in global products.

If you have other questions, send them in, and we will do our best to answer them here.