Emotions matter. They are at the core of human experience, shape our lives in the profoundest of ways and trigger our decision-making and social behaviour. Evolutionary theories indicated that emotions have an important survival functions and helped us solve certain challenges during our evolution. In social and business sciences, emotions are associated with the idea of human irrationality and non-logical behaviour. Affective computing deals with the study and development of digital systems and devices that can recognize, interpret, process, and simulate human affects. The new interplay of research findings from consumer behaviour, psychology, neuroscience and computer science opens up a new research universe and is attracting more and more scientists. Measuring human emotions with cutting-edge and AI-based technology support researchers on their mission to further explore these new worlds and set up experimental projects.
Effect of nudging in
regarding the consumption of fast food
Nudges are nowadays commonly used by companies and governments to prompt consumer choices in online and offline settings. However, the reasons for the effectiveness of specific characteristics in nudges is still scarce. The aim of the research is to understand the effectiveness and the impact of nudges to reduce caloric intake. The avatar-based nudges displayed either a face of dissatisfaction when the participant chooses a meal over the calorie intake goal, or a facial expression of satisfaction when the calories stay within the recommended goal. During the whole experiment the participants facial expressions were recorded and the triggered user emotions were measured with TAWNY technology to answer the question of what made the avatar more effective than the other nudge options.
Deep Flow: Detecting Optimal User Experience From Physiological Data
Using Deep Neural Networks
The affective state called flow is described as a state of optimal experience, total immersion and high productivity. As an important metric for various scenarios ranging from (professional) sports to work environments to user experience evaluations, it is extensively studied using traditional questionnaires. In order to make flow measurement accessible for online, real-time environments TAWNY technology was used in an experimental setting to automatically estimate a user’s flow state based on physiological signals measured with a wearable device. We conducted a study of subjects playing the game Tetris in varying difficulty levels, leading to boredom, stress, and flow. Using a convolutional neural network, we achieve an accuracy of 70% in recognizing flow-inducing levels. In the future, we expect flow to be a potential reward signal for human-in-the-loop reinforcement learning systems.
How to Stay in Flow at Work –
An Experimental Approach
To investigate the occurrence of flow during work as well as its connection to performance and physiological parameters, 59 subjects performed a simulated work task in an experiment. The participants had to retyped different texts on a laptop. These texts had three different difficulty levels: simple (children’s stories), medium and hard (chemistry science text), with the goal to induce three different performance and flow levels as well as physiological activations. Next to video data of the test persons the heart rate, R-R intervals and skin conductance were measured with the TAWNY setup.
A test and control group (n=33) was shown a trailer with/without commercial breaks. Arousal and valence of the test persons were measured with the Tawny emotion scan. Significant interaction effects between the ad and the environment were found. This brings the discussion about the advertising environment back to the foreground, which has been neglected for a long time but is gaining relevance through the common practice of programmatic buying for the automated placement of advertising media in online environments.
"There are a number of complex lab settings on the market which I used for my research on affective computing and digital empathy. TAWNY now offers a very convenient, scalable, cost-efficient, valid and reliable new research tool. I use the SaaS platform to organize all my experiments and directly share and discuss it with my research fellows. I also offered it for project work to my students and they love it. Finally, I also use it in applied research projects with partners from industry in academia."
Prof. Dr. Alexander Hahn
(Digital Marketing & Affective Computing)
TAWNY opens up a whole new universe for research projects and proposals in the area of consumer behaviour and affective computing!
Bartl, Füller (2020): The Rise of Emotion AI: Decoding Flow Experiences in Sports, In: 21st Century Sports: How New Technologies Change Sports in the Digital Age, Springer.
Bartl M. (2019): The Missing Piece in New Work: Decoding Human Flow at the Workplace, In: The Making-of Innovation, November 2020. https://michaelbartl.com/wp-content/uploads/2019_MoI_New-work-Flow_eng.pdf
Bartl M. (2019): The Rise of Emotion AI, In: Handbuch Künstliche Intelligenz.
Bartl M. (2018): Von der künstlichen zur emotionalen Intelligenz und was das für die Marktfoschung bedeutet, In: marktforschung.de 27.06.2018
Hahn A., Bartl M., Klug K. (2020): Digital Empathy: Wie Künstliche Intelligenz und Affective Computing die Marktforschung verändern, In: Dem Konsumenten auf der Spur: Erfolgreiches Marketing durch zeitgemäße Marktforschung. Pusler M. (Hrsg.) Freiburg: Haufe.
Richter D. and Bartl M. (2018): Affective Computing Applied as a Recipe Recommender System, In: Computational Social Science in the Age of Big Data. Herbert von Halem Verlag Cologne.
Maier, M., Elsner, D., Marouane, C., Zehnle, M., and Fuchs, C. (2019): DeepFlow: Detecting Optimal User Experience From Physiological Data Using Deep Neural Networks, In: International Joint Conference on Artificial Intelligence (IJCAI 2019).
ARTE TV (2019): Helena. Die Künstliche Intelligenz – Gefühle. https://www.arte.tv/de/videos/086131-002-A/helena-die-kuenstliche-intelligenz/
ARD TV (2020): Arbeiten im Flow. https://www.daserste.de/information/wissen-kultur/w-wie-wissen/arbeitswelt-110.html