Paper Reading #31: Identifying emotional states using keystroke dynamics

Identifying Emotional States using Keystroke Dynamics


Authors - Clayton Epp, Michael Lippold, and Regan L. Mandryk


Authors Bios - Clayton Epp is a graduate student at the University of Saskatchewan and focuses on affective computing using unobstrusive technologies.
Michael Lippold is a master's student at the University of Saskatchewan and has an undergraduate degree from the University of Calgary.
Regan L. Mandryk is an Assistant Professor at the University of Saskatchewan and has a PhD from Simon Fraser University.


Venue - This paper was presented at the CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems.


Summary


Hypothesis - In this paper, researchers note that systems don't react to contextual input in a reliable way in the current world and the ones that do require expensive sensors and are often invasive. The researchers hypothesize that they can build a reliable program that accurately detects emotion through keystroke dynamics and can use this information to respond appropriately.


Methods - The researchers wanted to gather daily use cases so rather than perform laboratory studies, they had users install software that recorded keystroke dynamics and periodically asked for users to fill out a questionnaire regarding their current emotional state. The software also featured a set of control (fixed) text as users had to type the same excerpts from a book from time to time. The study consisted of 26 participants, although only 12 were used in the results due to participation requirements not being met, who installed the above software and used their computer regularly for a period of about 4 weeks. The researchers gathered:

• Keystroke Features - This includes keystroke duration, latency, and more.
• Content Features - This includes the characters being typed such as number of capital letters or numbers.
• Emotional State Classes - The users expressed how they felt by answering a questionnaire that consisted of 15 questions regarding the users' emotional state on a Likert scale.
• Additional Data Points - An example of this is process name that was running while data was being typed.

In building their machine that will detect emotional states, the researchers used a machine learning algorithm that was trained on the data collected in the study. Models that detect a certain state with high accuracy will be shown.

Results - The top performing models were those that detected confidence, hesitance, nervousness, relaxation, sadness, and tired emotional states. Anger and excitement are models that have potential but were very skewed because people are usually not angered or excited for long periods of time making it difficult to get a solid read on necessary features. Each model only needed about 7 features to detect an emotional state accurately although the models varied on which features they needed meaning a system that used these models would still need a lot of features to be recorded.


Content - It should be noted that only models using the fixed text input were able to accurately detect emotion meaning that when the users were typing normally, the models were not able to pick up on what the user was feeling. The researchers propose that this is because the free text entries varied too much in length and did not provide enough data to be reliable. Another interesting note about this study is that the data was aggregated before analysis meaning individual differences were not catered to, so the researchers propose that forming a model on each user's data could provide better results for that user.


Conclusion - The researchers conclude by stating that this research is crucial in creating emotionally-aware computers in the future and can help in the work place by taking more precautions in one state while moving more efficiently in another.


Discussion


I think the researchers supported their hypothesis and convinced me that they had built a system capable of determining emotional states based on keystroke dynamics. I think the concept of using something as common as keystrokes to detect information is a great idea as it adds nothing to a user's workload but can form contextual information that can be used to help. I would like to see a system use this data to improve performance and reduce errors.