The Nokia 5800 already supports multitouch, sort of...
Published by Tzer2 at 10:21 UTC, April 29th 2009
The Nokia 5800 is criticised by some for being a single-touch device that can only register one finger press at a time. But is this actually true? Not quite. With a bit of clever interface design the existing 5800 hardware is already capable of registering two simultaneous screenpresses at once, and you can see how this works in the full story below (you can even try it for yourself, and there's a video of a 5800 game using it). The reason this is worth mentioning is that Nokia is already using a more limited version of this method in their just-released Maemo 5 SDK, which is the latest version of the platform they use on their internet tablets. If Nokia's already using it there, it seems very likely they could use it on their S60 devices too.
How multitouch on the 5800 can work
This example might seem pointless because it doesn't do anything useful, but that's because the QWERTY keyboard hasn't been designed with multitouch in mind. Apps designed for multitouch would use it properly and do stuff that is useful.
The purpose of this example is just to de monstrate how the 5800 detects keypresses, so you can see how multitouch could work.
The beauty of this example is that you can try it out yourself on your own 5800. Just start an application that uses text entry and select the full screen QWERTY keyboard. Now, do the following...
Hold down Q, and the Q button lights up:
Hold down P, and the P button lights up:
Now, hold down P and Q simultaneously, and the button directly between your touches lights up (either R or T, depending on exactly how you're pressing Q and P):
All you need to do to have a simple multitouch interface on existing 5800 hardware is interpret touches exactly between two controls as simultaneous presses of both controls.
You might be thinking that this is silly because people could activate multitouch by accident if they do a single touch between two controls. However, there are ways to avoid this problem.
You could then tell the device to only record multitouch when another button is already active. This means that if someone touches the screen between two controls it won't activate multitouch by accident, because no other button was active at the time the touch happened. In theory this would prevent multitouch working with two truly simultaneous touches, but you would very very rarely put both fingers onto two parts of a screen at exactly the same time, it's far more likely that you would touch one part first and then another part. Also, application interfaces could be designed to discourage truly simultaneous touches.
You could also design apps in such a way that there's no point in touching the area between them, for example a racing game could have all the controls at the sides of the screen with the middle used for displaying the race itself.
The benefit of this technique is that it requires no extra hardware, and it could work with even the cheapest touchscreen devices, though it may require more careful planning of where on-screen controls go. This is old news really, many developers are already using this method, but most people don't seem to realise that this is possible.
This isn't as flexible as "true" multitouch, as it wouldn't be directly recording the true position of the two touches, but that probably doesn't matter much because the user wouldn't have to ever know how it works. Clever and careful design of app interfaces can make this method function in a way that is virtually indistinguishable from traditional multitouch.
The extra care needed for interface design may make life more difficult for app developers, but the much wider range of devices they could reach would provide them with far greater sales potential. Very few devices do support true multitouch, so any technique which allows multitouch on all touchscreen devices is potentially very valuable indeed. It would also make simultaneous development for multiple touch-based platforms much easier.
Nokia are already using this method in the recently-released Maemo 5 SDK using the name "two-touch", where you hold down your finger on one part of the screen while touching elsewhere in order to activate a status menu: the interface would be registering two simultaneous touches, which is multitouch (of a kind, at least). Nokia could just as easily implement this technique on future S60 touch interfaces, and app developers can already use this technique for 5800 games and applications right now if they want to.
UPDATE: AAS reader Micky! has alerted us to a video of a game that uses this technique on the 5800:
Steve's take/thoughts:
There's more though. As I indicated a few months ago, the timing of the keypresses and their 'release' could also be used. Extending the thoughts above, I can see how the user might:
- press and hold the screen at 'q'
- 0.2 secs later, press and hold the screen at 'w'
- 0.2 secs later again, release the screen at 'q'
- finally, release the screen at 'w'
There's more than enough information here for the touchscreen driver to pass on coordinates as appropriate, effectively giving multi-touch (more like 'dual touch', since handling three spots at once might get too hairy!) on a non-multi-touch, resistive virtual keyboard. In this case processing the text input "qw", even though both letters were held down at the same time for a while.
Is there a technical reason why this technique wouldn't work?
Categories: Hardware, Editorial Thoughts
Platforms: Series 60, N-Gage, S60 5th Edition
News Discussion
EDIT. Here you go..
http://www.youtube.com/watch?v=XndW7YlvYjU
| The operating system will only see it as one touch though, it isnt seeing two touches and merging them. |
If I hold down the Q and P buttons on the 5800's keyboard, the OS registers it as R or T, even though I haven't touched R or T. The two touches are being merged into one which is exactly between them. I could alter the keyboard app to interpret R/T as Q and P being pressed simultaneously, and from a user point of view that would effectively be multitouch. Obviously that would be useless in the context of a keyboard because you wouldn't want to enter two letters simultaneously anyway, but in other applications this could be quite useful.
If I let go of Q without letting go of P, the phone goes back to registering P because that's still being held down. And if I press Q again while holding down P, it goes back to registering R or T. It all happens instantly, there's no delay or uncertainty.
Yes this is a property of resistive screens, and that's exactly why this has such great potential because it would potentially allow simultaneous keypresses on cheaper touchscreen devices.
The problem with this way of doing things is that pressing top left and bottom right will effectively touch the middle of the screen, and pressing top right and bottom left also effectively touches the middle of the screen. So I guess its fine for having a button layout like in that youtube video, but its quite limited (good start though :D)
| I mean it isnt the OS that is merging them, it is the touch screen. The OS has no idea if you pressed R/T or the Q and P simultaneously - it just sees a touch midway between them. I totally understand you can make multi touch like games in this way, but you could have done this years ago on earlier touch screen phones too |
The reason I'm posting this as a news item is that Nokia just released the beta SDK for Maemo 5, and it uses the above method to detect multiple touches in its own interface. If that's happening on Maemo, there's a good chance Nokia will use it on S60 too.
Users don't care if multiple keypresses are detected through the OS or the hardware, all they care about is whether they can press two buttons at once, which they can on the 5800 (and all other touchscreen devices) if the app is designed carefully.
| Oh, yeah. This sounds like Jack putting on bras and saying, look I'm Jill, sure I am.. Lame. |
Watch the video, it works. The game responds to multiple simultaneous keypresses. That's multitouch.
It's not exactly the same kind of multitouch as the iPhone's, and it might not be as easy to program or as flexible in design terms, but it works.
I wonder what Apple's multitouch patent actually covers? If it was purely the concept of responding to multiple presses then the patent is worthless, because it seems programmers have been doing this on resistive screens for years.
| No doubt you will get phone manufactures saying their device supports "multi touch" when it truly doesn't. |
It might not be as good as the iPhone's, but it's a heck of a lot cheaper to manufacture.
Saying only the iPhone counts as a multitouch device is like saying only plasma screens count as television sets.
I'm hoping that some of those gestures were already used on the big screen in Minority Report, but I guess that most of those gestures were 2 handed, similar to the demos of Microsoft Surface.
The phone can only register the single touch location. So if you touch both p and a and get g, the phone only knows g. The phone can't disciminate between a single touch of g and two touches on p and a. Therefore the phone and any software on the phone cannot do anything useful with this. Same goes for individual points, the phone only sees a single touch regardless of how it is achieved.
| This is no use. The phone can't disciminate between a single touch of g and two touches on p and a. |
If you have a genuine single touch, no other part of the screen would have been selected when the single touch happens, and the phone could assume it was a single touch.
If you have a genuine double touch, then some other part of the screen would already have been selected before the second touch, and the phone could assume it was a double touch, and then it could work out which on-screen controls it could have come from.
And even if that didn't work, you could still use this method for multitouch because you could design an application where there is no reason to touch the screen between two buttons (which, again, is something I talked about in the article). If you don't have to touch the screen between the controls in an app, then it doesn't matter if the app can distinguish multitouches or not. For example, if you put the controls of a driving game at the sides, and there were no controls at all in the middle, then the phone could correctly assume that any touches in the middle must be double-touches of controls at the sides.
| Therefore the phone and any software on the phone cannot do anything useful with this. |
| Same goes for individual points, the phone only sees a single touch regardless of how it is achieved. |
I know the above isn't quite what Tzer2 meant, but hey, it's what *I* mean!
Yes, the user presses Q and P at 0.2s delay.
OS receives:
* User pressed screen at Q coordinates
* after 0.2 s, user pressed screen at T coordinates (where in reality user pressed at P coordinates)
From this, the application is unsure if it was a second touch on T or if it was a multitouch on Q and P. How can the application distinguish this? Not enough information.
I guess someone needs to play with it and see how it feels. Multitouch would be nice on qwerty though, one of the things that makes it feel a bit weird is having to type discreet keypresses all the time.
|
Yes, the user presses Q and P at 0.2s delay. OS receives: * User pressed screen at Q coordinates * after 0.2 s, user pressed screen at T coordinates (where in reality user pressed at P coordinates) From this, the application is unsure if it was a second touch on T or if it was a multitouch on Q and P. How can the application distinguish this? |
Going by what is in the main article (which I wrote) it is possible to distinguish between multitouch at sides and single touch in middle:
1. User presses Q and keeps it pressed down.
2. User presses P while Q is still pressed down (which phone interprets as T).
3. Phone sees that it has a T pressed down, and checks to see if any other key was being pressed just before the T.
3. There was zero delay between another key and the T being pressed, so the phone now knows that it isn't really single T but actually a multitouch involving two different keys either side of T.
4. Because the last button before T pressed was Q, the phone can assume that one of the pairs in the multitouch is a Q
5. For a multitouch with Q to produce T, the other multitouch key would have to be P, because T is halfway between them.
6. The phone now knows that it is receiving a multitouch with Q and P being pressed simultaneously.
Whether this would work reliably on the QWERTY keyboard is unclear, because QWERTY has lots of controls which are all very small and very close together, so there might be frequent mistakes with neighbouring letters mistaken for each other. For multitouch by this method to be reliable, the controls would have to be bigger and spaced apart more so that the phone can be surer about which pair of controls produce which central touch.
But that is all irrelevant because this article isn't about doing multitouch on QWERTY. I mentioned the QWERTY example purely because it allows the user to see how the 5800 interprets multiple touches, people can try it out for themselves using their own 5800. (To be honest I'm not particularly bothered whether QWERTY gets multitouch, I think it's more important to see predictive text added to the 5800's QWERTY.)
To get back on topic, the article is about how multitouch could be done in apps that are designed to use it.
As I said in the article, even if distinction between single and multitouch was impossible, it doesn't matter because if an application has an interface designed for this kind of multitouch, the application doesn't have to distinguish anything.
For example, if an app interface has all its controls at the sides of the screen, the user will never touch the middle of the screen, so the application will never have to make any distinctions. All touches detected in the middle will be multitouch of the controls at the edges, because there are no controls in the middle.
And even if the user does touch the middle of the screen, the method detailed at the beginning of this reply can be used to distinguish between genuine single touches and genuine double touches.
The programmer of the game seems to have done pretty much what this article was talking about, by placing the touch controls at the edges of the screen.
You might say side screen controls are useless, but they're not, in fact side screen controls are extremely common, and they're very suitable for a device like the 5800 where the horizontal mode is designed to be used with two thumbs, while the vertical mode is only designed for use with one thumb (so no multitouch is needed).
Portable games consoles for example usually have controls on both sides of the screen, so doing virtual versions of these controls would work well with games. All kinds of other apps could use side controls too, such as media players etc.
Because the 5800 is designed to be used with two thumbs in horizontal mode, side controls would suit its ergonomics too. (One of the best media players on Nokia's internet tablets is called Canola, and it uses controls along the two edges not for multitouch purposes but purely to make it easier to use with thumbs.)
In other words, it could get the initial letter from the initial contact and the second letter from the second release. Other info in between is just irrelevant.
Steve
PS. Maybe I should just patent the thing and then make a million pounds!
www.check2cash.co.uk
right???