Surfnet logo

Machine Recognition of Touch Gestures and Emotion States via Fabric Pressure/Location Sensor

This is an ongoing project.

UBC’s SPIN Lab is advancing techniques to access user emotion parameters through tactile sensing and machine learning. We are using a custom low-cost, flexible and large-area pressure sensitive touch sensor. Our touch sensor can be applied to a vast range of touch-based input devices, making it generally accessible as a surface interface tool. We are using the touch sensor on our fleet of custom affective haptic robots, “Cuddlebots”. We are expanding and deepening our touch estimation and recognition capabilities to estimate a user’s emotion state and gestural intent. We are working towards applying our approach to continuous, less controlled collection contexts, while improving the accuracy of recognizing individuals from touch. Simultaneous application development is providing use cases for therapeutic and consumer device contexts.

We are developing the tools to recognize emotional content directly, focusing on within-individual variation of affect valence and arousal as emotional states change. We are using advanced collection and training paradigms, sampling of multiple emotion contexts for individuals studied, cross-validation, and deployment of sophisticated numerical analysis. We are developing architectures for continuous sampling and delineation of informative gesture or feature sequences. Exploiting our existing surface touch sensor platform, tactile sensor data is collected from volunteers as they pet a Cuddlebot while emotion state is triangulated with physiological sensors and self-reports.

Our larger goal is to provide cost-effective, non-invasive access to a user’s emotional state via touch. Our work is fully focused on exploiting unique possibilities for surface sensing, through sensing on large, flexible and compliant surfaces, and unintrusive inference of a user’s affective state and identity. Both are highly novel and impactful advances that can open new possibilities for surface-sensed applications, including roles in health, learning, expert systems monitoring as well as interaction more broadly.