At the Smartphone Show last month in London, S60 gave developers a preview of three very important technologies that are to be integrated into S60: Touch UI, Sensor Framework and UI Accelerator Toolkit. All three technologies have a key role to play in the advance of the user experience in S60 phones. In this article, I will review what we know about the technologies and consider their implications.
The announcement at the Symbian Smartphone Show was not a formal unveiling of the new version of S60 or any specific product plans: Nokia provided few technical or implementation details. Instead the announcement was a platform message to make developers aware of what is coming next. The new technologies are part of the next version of the S60 platform (version 5, presumably branded as S60 5th Edition) and this means we will see any number of devices and applications taking advantage of the new technology. The message from Nokia was that this is about 'evolving' the leading mobile platform. Remember there are more than 125 million S60 devices in the market and another 70-85 million will ship next year. These new technologies will be in millions of devices in relatively short order and those that can take advantage of them first will benefit most.
S60 Touch UI
Ever since the first device, the Nokia 7650, S60 (then known as Series 60) has been a 'soft key' driven user interface. Amongst smartphones as a whole, this has always been the dominant device style. This has partly been driven by Nokia's belief that such devices should be usable one handed. However there has always been a consistent demand for touch screen-based devices, as demonstrated by Sony Ericsson UIQ phones and Windows Mobile Professional based devices. Such devices were initially stylus-based and required two hands to operate anything more than the basic functions. However, devices such as the Treo range from Palm and more recent Sony Ericsson and Windows Mobile Professional phones have demonstrated that it is possible to create touch based devices that can also be used one handed. Touch as an input mechanism has also received a lot of attention recently thanks to the recent launch of phones such as the Apple iPhone and LG's Prada handset. In adding touch support to S60, Nokia is responding to a trend that has been around for the last five years.
For S60, the driving force behind adding touch screen support to the platform is to allow its licensees to develop a wider range of S60-powered devices. Clearly, this is a major step forward for S60 as it allows licensees to create an array of devices that are clearly differentiated from the traditional S60 phone design. Nor is it a simple matter of adding support for touch screen hardware; more significant is the optimisation of the S60 UI for use with either stylus or finger input. As a taster of things to come, at the Symbian Smartphone Show, Nokia showed a video demonstrating a concept touchscreen device running the S60 Touch UI.
Nokia have only offered a few glimpses of the UI itself, we will have to wait for a formal unveiling before any initial assessment can be made. However, a certain amount can be ascertained from the screenshot animations on the S60.com web site. As Nokia indicated, the UI has a familiar S60 look and feel with the familiar soft key commands at the bottom or side of the screen. Some phones will provide the traditional hardware keys for these, as shown on the concept device in the demo video; others will be purely touch-based. To allow for finger touch, it appears the amount of screen space that these occupy has been increased. Equally familiar is the pop-up menu which gives access to further commands. A notable addition is the presence of a toolbar to access the most common functions. For example, in Messaging read mode these are set to Reply, Forward and Delete; in Gallery's slideshow mode the toolbar provides Play, Forward and Back controls.
The screenshots show the UI in landscape mode. The familiar application grid launcher is present
as are the usual soft key command areas. The second screenshot shows an open Options menu.
Note the Active Applications entry at the top of the menu - this was introduced in S60 3rd Edition Feature
Pack 2 to promote the visibility of the multi-tasking capabilities of the phone.
Here we see the media overview before transition into a full screen view via the built in slide show functionality.
In terms of input, both an on-screen 'hunt and peck' keyboard for use with a stylus and handwriting recognition input have been shown in the demos. It seems likely there will also be a finger touch keyboard too (as seen on the N800 and, famously, the iPhone) and a finger based dialler. With existing support for traditional keypads and full QWERTY keyboards each device, is likely to mix and match different input methods according to the targeted niche or market. There is great regional variation in what might be considered the best solution: in the US, QWERTY keyboards are popular, while Asian markets prefer stylus based hand writing recognition because of their logographic or syllabic character sets.
Shown here is an example of the UI in portrait mode. Not the toolbars above the soft key
areas and the hand writing recognition input area.
One very important detail is that the new version of S60, including the new Touch UI, will be fully backwardly compatible with S60 3rd Edition. This means that applications written using standard UI components for current S60 devices should also run on the new touchscreen (and non-touchscreen) devices powered by S60 5th Edition. Nokia appears to have a similar approach to UIQ where there is a single code base for applications regardless of the input mechanism or form factor (i.e. it's an agnostic UI). Developers are able to optimise their applications for particular combinations of inputs and form factors, but this is not a requirement. This is similar to the way that S60 applications can be optimised for the wider screen of the E90 using an SDK plug-in. For UIQ, the transition to an agnostic UI occurred by moving from touch based devices to non-touch devices and the opposite is true for Nokia.
Support for tactile feedback is built into the S60 touch user interface. Tactile feedback means giving a physical confirmation when tapping or clicking on the screen. Current implementations of tactile feedback tend to be quite crude (e.g. phones simply activating their vibrate function, or game console controllers using their rumble pack). Nokia has been working to improve the capabilities of tactile feedback technology in a project code-named Haptikos (Finnish for touch). Rather than giving generalised feedback it seeks to localise the feedback to the key being pressed and to mimic the feeling of pressing a physical key. Instead of a general vibration, the user will feel the tactile feedback (with a distinct up and down response) under the key that they have pressed. Nokia believe tactile feedback is important because it more closely replicates the feeling of pressing a physical key and therefore makes touch screens easier and more pleasurable to use. At the recent Way We Live Next Event Nokia were demonstrating this technology and a blog post on the Red Ferret journal explains that it is planned to be included in Nokia's forthcoming touch based devices and has some additional details.
Sensor Framework
The Sensor Framework brings standardised access to sensors (e.g. motion and orientation, proximity and light) to the S60 platform. This gives you an additional way to interact with the phone. Initially the most common usage will revolve around accelerometer and position sensors, with motion triggering certain events on the phone. For example, turning a phone over might silence a ringing phone or turn off an alarm. Tapping the side of the phone while in the music player could skip forward to the next track, while tapping the front of the phone would play or pause music.
There are a number of S60 phones with sensors built in already available and these also demonstrate what is possible. The Nokia 5500 uses an accelerometer to create data in its pedometer application and to detect a tap on the side of the phone (with user configurable consequences). Recently it was made clear that the N95 had an accelerometer and a variety of basic concept applications have been developed as a result, including automatic screen rotation, a tilt meter for cars and a light sabre program. Motion sensors have particularly strong use cases in gaming and I fully expect to see this used in the N-Gage gaming platform.
All three technologies were introduced in a keynote presentation at the Symbian Smartphone Show.
Other examples can be found in phones already on the market, e.g. the iPhone touch screen deactivates when you hold it up to your face, and some Sony Ericsson phones can be shaken to stop or play music. On the S60 platform phones such actions will be user configurable, although the extent of this is largely dependent on what the phone manufacturer decides to allow, furthermore the options available in any given phone will vary depending on the type and sensitivity of any sensor used. However, because the Sensor Platform offers an easy way to implement such offerings it should become common place on S60 phones, to the extent that this represents the next major step forward in the way we will interact with our phones. In many ways it offers a wider range of (and more compelling) use cases than touch interaction and may have a greater impact because of its broader applicability.
The framework is designed so that manufacturers can add additional sensors which allows for additional use cases to be added. For example many phones already include a light sensor to adjust the brightness of the screen. If this was made available through the Sensor Framework it would be possible for applications (e.g. navigation) to automatically switch to a night mode (dark background) to reduce glare.
The Sensor Framework is also a key addition for third party developers. In the same way that the Mapping Framework enables access to location and mapping information and functions, the Sensor Framework will allow developers to build their applications to take advantage of the sensors built into the phone. As with most 'new' technology, the most innovative use cases have yet to be even thought of.
UI Accelerator Toolkit
The new UI Accelerator Framework, which is built on top of Symbian's ScreenPlay technology, is designed to allow manufacturers to rapidly develop graphical effects, scalable layouts and new types of interactions. This includes things like transitions and animation within the UI. At such there is no discrete functionality, but it could be used to highlight or help explain certain functionality on-screen in a pleasing away. Plus eye candy or the 'wow' factor is an important part of the consumer user 'experience'. It will also make for greater personalisation of the device, something that has been high up the list of user demands in feedback Nokia has received.
The multimedia menu in the Nokia N95 classic is an example of things to come. Here there is lots of movement, with an ever changing background and floating animated icons, all of which can be customised by the user. Building the UI Accelerator into the framework will improve the performance of such graphical fripperies, but will also make them quicker and easier to implement.
A note on S60 Design
The design of S60 is built around a user experience framework, built around several key areas which can be thought of as fundamental building blocks in a mobile platform. They include: input, output, functionality, performance and design/usability. Each of these areas may have various components and enablers and together constitute the platform. This gives the platform flexibility and the ability to continually evolve as new versions are released.
The introduction of multiple screen resolutions and orientations (output) and the addition of QWERTY keyboard support in S60 3rd Edition are good examples of how this works. Many of the technologies supported by S60 are optional, for example screen rotation and QWERTY keyboards are supported, but they are not used in all products.
The same applies to the new Touch UI; at is simplest, it is just another input mechanism. Clearly Touch UI is more than that since there are design changes to the UI of S60 to optimise it for touch usage and there is the tactile feedback (output) component too. Similarly, the Sensor Framework can be considered another input mechanism while the UI Accelerator Toolkit looks to improve performance and enables new graphical effects and functionality. Nokia have underlined that the Touch UI is optional. It is, therefore, best to see it as an enabler of a new family of touch based S60 devices. In the future we will continue to see traditional keypad/soft key-based S60 devices, but we will also see touch and keyboard-based S60 devices and touch-only S60 devices too.
A note on Nokia's Touch History
It is worth realising that Nokia has something of a history with touch based devices. The Nokia 7700 and 7710, which ran Series 90 on Symbian OS, are the best examples. Series 90 was a fully touch-based UI and had grown out of a joint Nokia/Psion project ('Hildon', though in one sense its roots can be traced all the way back to the original Psion organisers). As a UI it was ahead of its time and the Nokia 7110 showed much promise, but the hardware was underpowered and the software was sluggish and never quite felt finished. In relatively short order, Nokia killed off Series 90, partly because of their belief that phones should be one-handed devices and perhaps realising the impracticality of maintaining multiple platforms.
Example screenshot from the Nokia 7710
In 2004/5 Nokia announced that the functionality of Series 90 would be rolled in to S60 and it was expected that this would be completed in 2006. Clearly it has taken longer than originally envisioned, although whether that was because the scale of the task grew to include other objectives or was under-estimated or because there was a lack of urgency is open to debate. While the focus is on functionality it is likely that some of the UI concepts of Hildon may also filter down into S60 (as an aside, Hildon also lives on elsewhere - as the UI for Nokia's Maemo based Internet tablets).
Finger keyboard as seen on the Nokia N800.
Frequently asked questions
The new technologies, in particular S60 Touch UI, have been a hot topic. Here are some of the questions that have come through the AAS mailbox in the last few weeks.
Q: When will we see S60 Touch devices?
A: Nokia says the second half of 2008. It is possible the announcements will come before then.
Q: What sort of devices will there be?
A: The whole point is that S60 enables manufacturers to create all kinds of devices, and there are ever increasing number of possible permutations and thus this question is completely open to speculation. It seems likely that initial devices may include both keyboard/keypad-based devices (like the Sony Ericsson W960 or Treo 700) and slate types devices (like the iPhone or HTC Touch). Bear in mind that both finger touch and stylus touch is supported, so that allows for further permutations. Even though it's not required, it seems likely that the majority of touch devices will have at least directional controls (which can be used to navigate the UI). Remember there will still be a wide variety of non-touch devices; the Sensor Platform and UI Accelerator Toolkit technology will be more widely implemented than Touch.
Q: Will other S60 licensees produce S60 Touch devices?
A: There is no certainty in anything. All S60 licensees (Nokia, Samsung, LG, Lenovo) will have access to the new version of the platform. However, reading between the lines, the most likely candidate, aside from Nokia, is Samsung.
Q: How will UI elements work?
A: The simple answer is that we don't know yet. Nokia have deliberately been fairly evasive about the exact nature and features of the S60 Touch UI. You can have a look at the screenshots above and on the S60.com web site which give some clues. One of the aims is to create a UI familiar to existing S60 users, therefore there are not going to be wholesale visible changes, rather it will be about enabling touch interaction (e.g. 'thicker' softkey buttons).
Q: What about capacitive touchscreens and multi-touch technology?
A: We don't know. Nokia say they are aware of the various options available and are looking to create the optimum user experience (that is marketing speak for 'we're not telling you yet').
Q: What happened to S60 4th Edition?
A: 4 is an unlucky number in some Asian countries and thus is rarely used in global products.
If you have other questions, send them in, and we will do our best to answer them here.
Rafe Blandford, All About Symbian, 13 Nov 2007
