Paper Reading #10: Sensing foot gestures from the pocket

Sensing foot gestures from the pocket


Authors - Jeremy Scott, David Dearman, Koji Yatani, Khai N. Truong


Authors Bios - Jeremy Scott is a graduate student at MIT and received his Bachelor's from the University of Toronto.
David Dearman is a PhD student at the University of Toronto.
Koji Yatani is a PhD student at the University of Toronto interested in interactive systems.
Khai Truong is an associate professor at the University of Toronto specifically interested in human-computer interaction.


Venue - This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.


Summary


Hypothesis - In this paper, researchers discuss a way to use foot gestures to perform tasks in a mobile environment and develop a system that supports this. In doing so, they hope to prove their hypothesis that such a system can learn over time from users to recognize more accurately in addition to the primary goal of allowing users to perform tasks without having to focus on visual input and visual feedback.



Methods - The researchers decided the first thing to do was study possible foot gestures that could be used in the product. The four gestures explored were dorsiflexion, Plantar flexion, heel rotation, and toe rotation. Participants were asked to hold down a button on a mouse and perform one of these gestures rotating to a specified angle from the start position and releasing to indicate completion. The setup consisted of 6 Motion Capture cameras and a laptop, informing the participant of what task was to be performed and recording information received from the cameras. Participants began the study with the training phase that consisted of 156 gestures and visual feedback informing the user of their progress. Next, the testing phase began with no visual feedback and 468 gestures. Lastly, participants were interviewed and asked to rank the gestures in order of preference. A second study was conducted using the same procedure and equipment later in testing the researchers' machine learning algorithms by using different number of users' data as training data and the rest as test data. Also heel rotation and Plantar flexion were the only 2 gestures tested due to the results of the first study (see below). 2 different classification procedures were used for the machine learning portion of this study. Leave-one-participant-out (LOPO) consisted of using all but one of the particpants' data as training data and then testing on the remaining participant. Within-participant (WP) consisted of a single user performing a gesture many times with all but one of those trials used as training data and the remaining trial used as test data.


Results - The initial study found raising the heel, or Plantar flexion, to be the most accurate and preferred gesture for vertical angles. Plantar flexion also showed a consistent error rate across all angles whereas the other gestures increased in error as the angle increased. Among the rotation gestures, both heel and toe rotation were comparable in regards to error and range but heel rotation was greatly preferred by the participants. The second study tested gesture recognition using a phone located in a front pocket, back pocket, and hip mount which resulted in successful recognition 50.8%, 34.9%, and 86.4% of the time respectively.  Higher percentages resulted when the algorithm only had to determine which gesture type was being performed (heel rotation or Plantar flexion).

Contents - The researchers developed a program to recognize foot gestures using data collected with a phone's accelerometer. The workflow for the program consisted of a user wearing a phone in a pocket, initiating the system by placing a foot at the origin and performing a double tab, and performing the desired gesture which would then be recognized by the system and executes the desired command. To optimize this method, the recognition algorithm would have to be very robust and able to adapt to an individual so the researchers integrated machine learning into the workflow and did several quick tests to help develop the machine learning algorithm. They found 34 features that could be used in gesture recognition and implemented them into an initial application that was used in the second study.


Conclusion - The researchers conclude that foot gestures are a viable option for user input. Because the WP procedure worked significantly better than LOPO, the researchers also conclude that this type of program would be best implemented by learning from an individual user before use by that person.


Discussion


I think the researchers achieved their goal of proving the viability of using foot gestures as input and they mostly convinced me of this in their testing. I would have also liked to have seen a framework built that other developers could use to begin using this technology in real-world applications because I think the usefulness of this ability is still questionable to most people. Nevertheless this technology could prove crucial in the future of device interfacing and help make device interfacing a very transparent and non-feedback dependent task.

Deliverable #2: Proposal and Week 1 Results

Project Proposal

Andy Hampton, Neal Roessler, Zach Casull, Chandler Cell

            We chose professional gamers as our target group for our ethnography project. We chose this group because of all the stereotypes and assumptions that are usually applied to this group without a second thought. Although, some of us have experience with gaming as a pastime, we decided to proceed because none of us can even imagine the work that we know is necessary to compete on a serious level.

            The expectation from this group is that it is male dominated with varying personalities from the quiet loners to the obnoxious attention cravers. Tendencies towards violence and verbal abuse will be interesting subjects to observe while studying this group. The scope of this study will range from individuals to local group competitions and whatever else we can find that can provide us some insight into the minds of professional gamers. From this study we hope to learn enough about how professional gamers tick so as to be able to suggest a product that could be useful to them.

            This study is appropriate because of the many personalities and behaviors we anticipate to observe while conducting the study. Demographic observations will show trends in the gender and age of participants which can further be broken down and compared to our own experiences with what would be considered “normal” people in these ranges. The violence and verbal observations mentioned above will also be valuable given the recent Supreme Court case regarding violence in video games and its effect on people who play games which fit squarely in our target group. This study is not inappropriate as we have no reason to gather personal information beyond age, gender, and possibly appearance. Conducting interviews and attending public gaming events also prove appropriate and do not in any way break laws. This study will be exciting because of the breadth of people we anticipate on studying in thier very animated, high-powered environment that keeps them interested in this pasttime and motivated to practice in the massive volumes that they do.

This study is feasible because we will cover many different aspects of professional gaming conducting several interviews with acquaintances that play games professionally as well as observing competitions and tournaments. We also hope to eventually participate in the activity if the opportunity presents itself to one of us or one of our acquaintances whom we can follow.

Week 1 Results

Andy Hampton

            We attended a Gears of War 2 tournament at Clockwork Games and Events in College Station, Texas which we found out about through a search engine on the internet. This experience proved to support some of my hypotheses and disprove others.

            Upon arrival in the outlet area where Clockwork was located, I found it on the side of one of the buildings with poor signage out front. This led me to believe that people who go here are a fairly tight knit bunch because advertising was nowhere to be found and the hours of the venue were odd, seemingly catering to an after school crowd. Once inside, I noticed a pretty sophisticated setup consisting of a row of 8 XBOX 360 consoles mounted about 1 foot off the ground on the wall with 8, approximately 20 inch, screens and 2 larger screens above them. The lighting was poor, as to be expected, and the interior was actually quite nice with about 6 tables, for card games I assume, and 2 couches in front of the console laden wall along with other various chairs. There was also a counter as I assume this place also sold games, cards, and snacks.

There were about 15 people there representing ages 9 to 40. There was talk that one of the participants there was a former Major League Gamer. We learned that people that usually come to Clockwork play card games and video games. Everyone there was relaxed and enjoyed the company of fellow gamers and it became apparent that many of them were familiar with each other suggesting a regular crowd usually played at Clockwork. They were happy to converse with us and answered a lot of our questions. It was heavily dominated by guys but there were two girls that attended but did not play.

It was explained to us that the Gears of War 2 tournament would consist of 10 participants in a 1v1 double elimination tournament bracket following Major League Gaming rules. Julio is the tournament manager and assigns the rules of the game and acts as referee by preventing cheating that might take place such as screen peaking, the act of looking at the opponents screen to gauge where their character was located on the game map. The winner received a new copy of Gears of Wars 3 sponsored by FX Games, represented by Chris who has known Julio for many years, and some prize money. We entered in 2 of our own members, Neal and Zach, to support the event and immerse ourselves a little.

In the first game there was a 9 year old kid that beat a 18 year old and he was calling him out by saying “Now Suck This!” Everyone was animated and very into the tournament, yelling at the TV’s and having a good time. There was a lot of smack talk, sarcastic hostility, and jokes among everyone there but as a bit of a surprise everyone seemed pretty outgoing and swearing was kept under control, if it even occurred.

We met Jacob who is in high school and aspires to be a professional. He plays at least 3 hours a day and recently played a $1000 tournament but got beat by some “hardcore kids”. He said it is a lot about getting in some people’s heads especially if they are younger than you, you can get in their heads with some smack talk he says, and “put the kids in their place”.

Overall, participants of the competition exhibited unforeseen outgoingness which may have been a result of being in their comfort zone as was apparent given the pre-existing relationships between the gamers. As was predicted most of the gamers had spent large amounts of time playing the contest game and were therefore very good at them relative to us. Some interesting observations were the age ranges which varied wildly and the highly competitive environment that developed after the tournament started which differed with the very laid back atmosphere that existed beforehand.

Paper Reading #9: Jogging over a distance between Europe and Austrailia

Jogging over a distance between Europe and Austrailia


Authors - Florian Mueller, Frank Vetere, Marting R. Gibbs, Darren Edge, Stefan Agamanolis, Jennifer G. Sheridan


Authors Bio - Florian Mueller was a PhD student at the University of Melbourne at the time of this paper's publication and he is now working as a Fulbright Visiting Scholar at Stanford University.
Frank Vetere is a senior lecturer at the University of Melbourne researching Human-Computer Interaction and Interaction Design.
Martin Gibbs is a lecturere at the University of Melbourne and is investigating interactive technology as it relates to society.
Darren Edge is a researcher at Microsoft Research and has a PhD from the University of Cambridge.
Stefan Agamanolis was a researcher at the Distance Lab in Scotland at the time of publication and has a PhD from MIT.
Jennifer Sheridan was a researcher at the London Knowledge Lab at the time of publication of this paper and has a PhD from Lancaster University in Lancaster, UK.


Venue - This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.


Summary


Hypothesis - In this paper, researchers explain a framework they have built called Jogging over a Distance that allows jogging to be more social. In doing so, the researchers hope to support their hypothesis that this framework can be used to collect data on how people perform social exertion activities and use said data to build more socially conscious exercising software that promotes long term user participation.


Contents - Jogging over a Distance requires 2 users to agree on a time to begin jogging and input a target heart rate into the system. The users then put on a headset, heart rate monitor, and computer/phone. As the users jog, their heart rate data is sent to a server where the information is compared to the target rates of the users. If one of the user's heart rate is faster than their goal and the other user's is slower then the faster user's audio will seem to come from in front of the slower user increasing the desire to perform better.


Methods - To test the effectiveness of the system, 17 participants went on 14 runs using Jogging over a Distance. To evaluate the system's performance on the subjects, extensive interviews were conducted with all participants after use and analyzed. These interviews consisted of note taking during and after the sessions, discussing heart rate data to better unravel the events that took place, and using this data to form affinity diagrams. Because of this style of data collection and analyses, the researchers were developing only qualitative results to better discuss what the system did right and wrong rather than crunch some numbers that may be misleading due to the social nature of this project.


Results - Communication integration proved to be extremely valuable in getting users involved during the activity by creating a goal of side-by-side conversation that was only possible if target heart rates were being met. This also helped users feel safer and led to many users jogging further than they would have alone. Effort comprehension also proved valuable to participants as they learned about how their bodies actually exercise in a relative sense rather than a competitive sense because users were at different levels of fitness meaning just because one person is slower than another doesn't mean that the slower person is putting in any less effort. Keeping this fact in mind, participants said that working hard and staying right beside their partner led to greater satisfaction from the fact that both parties knew that equal effort was being exerted. The virtual mapping of the participants based on heart rates was overall viewed positively because some of the participants couldn't run with their partners before because they would be too fast or slow but Jogging over a Distance fixed that. Their were still negatives when a user entered in a bad baseline value or external factors got in the way.


Conclusion - The researchers conclude by saying they hope this study paves the way for more research and application regarding social integration with physical exertion.


Discussion


I think the researchers achieved their goal of showing how social integration with physical exertion can encourage people to exercise more. Putting myself in this situation has me convinced that this would motivate me to exert more if someone sounded distantly in front of me. I found this study especially interesting as a first step in what could be a great field in the future as group exercise is brought into the space. The only fault of this work that I could find was the lack of concrete data but it makes sense to take a qualitative approach because numbers are not as important in something that is attempting to increase participation and encouragement of physical activity. However, this approach could have been improved if the participants had been studied for a longer period of time to see if habits developed or not based on the system.

Deliverable #1: Prior Assessments and First Results

Pro-Gamers group
Neal Roessler
Zach Casull
Chandler Cell


Prior Perception  

            My group decided to study the world of professional gaming instead of a single group of people. Expanding to this larger scale allows for a more comprehensive glimpse into a lifestyle and does not limit us to a specific set of people. In this paper I will describe my expectations, perceptions, and future plans regarding this field.

            My initial expectation of what I will see is a stereotypically geeky culture that is predominately male and focused on what they do. These people will probably tend to be fairly private individuals that have been doing this kind of thing for years probably on the same game. I expect my interactions with them to be minimal as I do not foresee long, drawn out responses to questions or prolonged conversations occurring.

            Integration into this group may prove to be fairly difficult as I, or anyone else in the group, is anywhere near as skilled, or even familiar with, most of the games we will see. We will have to take the position of the audience for a majority of the study and will probably have only 1 or 2 people able to participate to any competitive degree in any of the events we plan to attend. We plan on attending gaming competitions and following some people that we have connections to during the study.

            I hope that I, and the group, will gain some meaningful insight into this broad area and be able to contribute during the project.

Initial Results

            The first interaction we had with professional gaming consisted of an informal meeting with one of our connections that is a semi-professional. We only met with this one person, Tyler, so as to give us an entry point into this field.

            Our first meeting consisted of a meeting with Tyler at his residence on Tuesday, introducing ourselves and what we would like to do, and a small interview with Tyler consisting of some broad questions we had about the area and his personal experience as well. On first arrival, his apartment appeared how I expected it to be. Minimal, kind of messy, and what little furniture he did have related to his technology in some way (TV stand, consoles, etc.). We proceeded to introduce ourselves and describe the project in a general sense as not to influence anything Tyler would do or say later. Through questions we learned that about the top 1% of gamers in a particular game can be considered professional which usually equates to around 100+ players. An interesting made by Tyler was that at a certain point, a single gamer cannot get any better and they must begin working with a team to become truly professional which is why competitions usually consist of only team contests. In addition to being on teams, professional gamers must have a coach for their teams to act as an anchor and keep the team on task and off each other’s backs.

            Overall, we gained a lot of insight from Tyler and hope to continue our study with him and others in the future. We are currently working on plans to attend as many professional gaming events as we can find and maybe even find one that Tyler can enter in.

Paper Reading #8: Gesture search

Gesture search: a tool for fast mobile data access

Yang Li

Yang Li is a researcher at Google and earned his PhD from the Chinese Academy of Sciences.

This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.


Summary


In this paper, the researcher creates a program that recognizes gestures and searches data on mobile devices based on said gestures. The hypothesis is that using such a system can improve access time and makes navigating the ever increasingly complex mobile interfaces quicker and easier than ever. 



Next, the researcher discusses the basics of how Gesture Search works. A user gestures a letter and search results begin appearing ranked in order of most commonly accessed. The user can then continue entering more gestures to refine the search or select the desired data and perform a task on it such as calling a contact or opening an application.


The researcher then described many of the interactions with the device such as discerning GUI input (scrolling, tap selection) from gesture input, searching from a gesture query, and using the history to display better results. Telling GUI input apart from gesture input was difficult at first but resolved after a study was done involving users that helped develop a model for recognizing scrolling vs. gesture input quickly. Searching using a gesture query is extremely similar to any other searching done on mobile devices as search results change based on new input entered in real time. Using access frequency consists of storing the frequency in a local container and then applying a ranking algorithm that takes this information into account when displaying results.


Lastly, the researcher performed a study using employees at a specific company and had them use the app in their daily lives. The study found that a majority data access was in the form of contacts and applications and that most queries consisted of between 1 and 3 characters to get to the desired data. The survey conducted at the end of the study showed that the participants enjoyed the product and found it useful.


Discussion


I think the researcher achieved his goal of providing a quicker way to access data using gestures. I think letter recognition may not be the best way to do this though because, typing the first 3 letters of an application can be just as fast if not faster than any kind of gesture recognition that has a lot of wait time, in my experience, between letters. The aspect of this product that interested me the most was user-defined gestures which were barely mentioned because those could lead to consistent one-gesture use cases.

Paper Reading #7: Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop

Swype

Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop


Jochen Rick


Jochen Rick is an associate professor at Saarland University and is a previous research fellow at The Open University after earning his PhD from Georgia Institute of Technology.


This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.


Summary


In this paper, the researcher explores the validity of the QWERTY keyboard as a viable way to handle input in a touch based world and develops a new model that supports stroke-based input that increases efficiency.


The researcher explains the history of keyboard layouts beginning in 1874 with the QWERTY keyboard after altering a alphabetic design that resulted in keys close together being pressed and jamming the typewriter. Since this keyboard became dominant, many people and companies have tried to improve upon it in both physical and digital environments to no real success.


Next, the researcher creates a model to be used in later steps by performing a study of 8 participants as they perform many tasks, that consist of connecting 4 randomly placed points in a specific order and recording the time it took to go certain distances in certain directions. The stroke was separated into three phases (beginning, middle, end) to further study. The results of the study led the researcher to create a model that is based on a Shannon formulation of Fitts's law.


The researcher then applied this model to the many keyboard designs introduced earlier in the paper and calculated the words per minute rate that could be possible, for stroke-based and tapping input, for each layout. The words used for testing came from the Project Gutenburg library and has shown to be very accurate for these kinds of tests. The results shows the OPTI II and GAG II designs working the fastest while the QWERTY and other more popular layouts performed poorly.


The researcher then applies a simulated annealing process to produce a square-key and hexagon-key layout that result in a faster calculated wpm than the best layouts in the previous analysis.


The researcher concludes by discussing why the QWERTY keyboard has remained a standard for so long despite being shown slower than some other models and explaining that adoption of a new typing layout could prove to be a difficult endeavor but worthwhile as we explore touch.


Discussion


The researcher did achieve his goal of developing an optimized stroke-based keyboard layout for touch input but the real question is does it matter? On this account, I think not because this entire study revolves around tabletop surfaces only which I don't believe will be in common use for the next 5-10 years if even then. It does however point out that keyboard input does need to be fixed for touch based devices.


There is a future in this field for research on a mobile level and in other areas as well. A big segment that this kind of technology would be useful is for disabled people that have to use keyboards such as Swype to track their eye movements and respond in a manner similar to those discussed in this paper. Using an optimized keyboard, this task could be simplified and made quicker.

Paper Reading #6: TurKit

TurKit: Human Computation Algorithms on Mechanical Turk


Greg Little, Lydia B. Chilton, Max Goldman, Robert C. Miller


Greg Little is a PhD student at MIT working in the User Interface Design group.
Lydia Chilton is a graduate student at the University of Washington and attended MIT prior to that.
Max Goldman is a graduate student at MIT focusing on user interfaces and software development.
Rob Miller is an associate professor at MIT and leads the User Interface Design Group there.


This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.


Summary


In this paper, researchers propose a way to improve upon the Mechanical Turk application used for human computation by providing a toolkit that provides a framework upon which to build more complex algorithms. The hypothesis is that TurKit will make human computation more effective than it currently is by enabling complex algorithms.


The first item presented by the researchers is that TurKit Script which extends javascript's ability to communicate with MTurk. TurKit Script adds support for the functions waitForHIT, prompt, vote, and sort as well as fork and join to support concurrency. 


The next thing discussed is crash-and-rerun programming which TurKit Script is based on. Crash-and-rerun programming involves recording all successfully executed lines of code, crashing, and rerunning all recorded lines. TurKit Script had to adjust this to fit their model due to some of the functions being called costing actual money so they developed a once function that tells the script to look for an output to this line before running it to see if it had already been run. 


TurKit was written in Java and used Rhino and E4X for additional support. Implementation of the once function took the most work because a persistent data store had to be created to hold the values after the line is executed so that on the next run it can be retrieved without making unnecessary calls.


The researchers provide examples of where TurKit has been used to great affect and shows what cannot currently be done in standard MTurk. These include iterative writing, text recognition, and decision theory experiments. All of these examples take advantage of the crash-and-rerun nature of TurKit and allow users to build off of other users' answers.


The paper concludes by noting the many uses of TurKit that have already been put into practice as well as note new ways it can be used like for experimental replication or just exploring new algorithms using the crash-and-rerun framework.


Discussion


I think the researchers did not necessarily prove their hypothesis with this paper alone however, I feel that they did still convince me of the success of the project by the examples of people already using TurKit. They explored a new territory in human computation by supporting iterative improvement allowing users to build off of other users to get maximum feedback.


I think this paper was interesting because of the crash-and-rerun programming that they decided to use in TurKit. This approach supports the claim that the simplest solution is usually good enough although not optimal. Recording line executions that succeed is very smart but also very costly especially when trying to scale so there is a very niche audience that I think this is aimed and they better be willing to make some sacrifice. But as mentioned above, people are willing to make these sacrifices and are already using it so it sounds like a success.