Areas of Interest

1. Understanding how groups make decisions.

Many real life decisions don’t have right or wrong answers but often depend on personal values (e.g. health, education, family, environment) and preferences (e.g. food tastes, music tastes). Using a group decision making paradigm with real outcomes, I investigate how these values and preferences are revealed, modified, and integrated in group discussions and also leverage facial expressions and body pose data to evaluate how non-verbal signals can be indicative of people’s roles, influence, and satisfaction with decision outcomes.

2. Shared affective experiences and social connection.

Why do we feel connected to another person after we watch a movie together, eat together, or take a same bus ride together? To answer this, we created a shared experience in which participants watched a TV show either alone or with another participant. By recording and analyzing their facial expressions elicited during the show, we find that more synchronized facial expressions both in time and in shape lead to greater social affiliation. This shows interpersonal metrics based on dynamics of facial expressions can be used to gauge the development of social relationships.

3. Building facial expression models of complex emotions.

Complex emotions such as regret, guilt, pain, and empathy are difficult to elicit and under investigated. Using decision-making paradigms with real interpersonal outcomes such as painful thermal stimulations applied to oneself or to others, we elicited these emotions and built facial expression models to pioneer new ways to study how experiencing these emotions impact social interactions and judgments of others.

In the process, I have authored and contributed to research tools that may be also useful to others.

  • FaceSync: Python toolbox to synchronize videos based on audio.

  • FEAT (Facial Expression Analysis Toolbox): Python toolbox for preprocessing & analyzing facial expressions.

  • Py-PAT (Python Pose Analysis Toolbox): Python toolbox for preprocessing, feature extracting, and analyzing human pose data.

  • Affectiva-API-APP: Web tool to extract facial expressions from videos using Affectiva’s JavaScript API.

  • PaperWiki: Free online summaries for scientific articles, created and edited by readers around the world.

  • nltools: Python toolbox designed for streamlined analysis of neuroimaging data.

  • pyMedoc: Python class to remotely communicate with Medoc Pathway Thermode over WiFi.