Paper Reading #24: Gesture avatar: a technique for operating mobile user interfaces using gestures

Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures


Authors - Hao Lü and Yang Li 


Authors Bios - Hao Lü is a graduate student at the University of Washington and focuses on topics in human computer human interaction.
Yang Li is a researcher at Google and earned his PhD from the Chinese Academy of Sciences.


Venue - This paper was presented at the CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems.


Summary


Hypothesis - In this paper, researchers state that mobile devices must compensate for the inaccuracies in touch-based interaction by wasting space on bigger than required buttons and widgets. The researchers propose a system, Gesture Avatar, that recognizes gestures associated with specified GUI widgets and performs the desired function. The hypothesis is that this kind of a system will reduce errors and be preferred by GUI designers that won't have to wast real estate on over sized buttons anymore.


Content - Gesture Avatar operates by cycling through 4 state:

• Initial State - Begins when user presses finger to surface.
• Gesturing State - The touch trace is underlayed onto the underlying interface.
• Avatar State - If the bounding box of the gesture indicates that it is a gesture and it is recognized as a gesture, then it becomes a translucent avatar that can be manipulated.
• Avatar Adjusting State - Allows user to change association of avatar, move avatar, or delete avatar.

A gesture can either be a character or a shape. If it's a character, then Gesture Avatar will recognize it and use it to search the content on the page. If it's a shape then a shape recognizer attempts to match the gesture to an interface widget as close as possible. Distance from the gesture is also a factor when deciding what object is to be associated with the gesture. To correct for mismatched objects, users can draw a next gesture that will find the next closest similar object and associate the gesture with that.

Methods - Some predictions as to how Gesture Avatar will help users is:

• Gesture Avatar will be slower than Shift, a touch accuracy enhancer, for bigger targets and faster for smaller ones.
• The error rates for Gesture Avatar will always be lower than Shift especially when a user is walking or moving.

The researchers implemented Gesture Avatar in a Java based Android 2.2 app. 12 smartphone users were selected to participate in an evaluation of Gesture Avatar. Half of the participants did the Gesture Avatar evaluation first and half did the Shift first but both groups did both. The test consisted of 24 small objects being displayed in the top half of the screen with a target object in red so as to be easily identified. The target object is also blown up to 300 pixels below the objects so the user can easily see the contents of the target object. The test begins when the user taps the large target. The objects have both ambiguous and non-ambiguous cases and are distributed evenly on the screen. Participants practiced with the app they were evaluating, tested the app for the first 12 sessions while sitting on a stool, and tested the app for the last 12 sessions while walking on a treadmill. Each session consisted of 10 individual tasks.


Results - The time results showed what was predicted in that Gesture Avatar was slower than Shift when the targets were large but was significantly faster when target size decreased. Also as expected, Gesture Avatar showed no change in error rates in all of the test cases whereas Shift was affected by all changes and higher in all tests than Gesture Avatar. All of the researchers' hypothesis were supported by the study.


Conclusion - The researchers conclude by stating that Gesture Avatar has proven itself better than many similar products on the market and just needs to begin integration with existing software in order to emerge as a preferred method of input for users.


Discussion


I think the researchers proved their hypothesis by creating a system that results in lower success rates when attempting to select small objects on a device. I think this application is particularly valid because if wide support emerged for this product and a standard was created, UI designers could stop wasting space on overly large interactive objects.