RHUMBO · modelling and pRedicting Human decision-making Using Measures of subconscious Brain processes through mixed reality interfaces and biOmetric signals

modelling and pRedicting Human decision-making Using Measures of subconscious Brain processes through mixed reality interfaces and biOmetric signals

RHUMBO, a pioneering research project, aims to revolutionize business practices amidst the evolving technological landscape. In this changing world, companies no longer wield full control over media and messaging. Simultaneously, consumers have become increasingly discerning, informed, and proactive, losing trust in traditional approaches. To address these challenges, RHUMBO has developed a suite of tools and models that utilize mixed reality technologies (MRT) and biometric signals, supported by artificial intelligence, to predict human decision-making in dynamic business environments. The primary focus has been on understanding consumer behavior and psychological states during complex, realistic situations. The research yielded remarkable outcomes and insights: 1. Digital biomarkers of consumer behavior were successfully developed, using metrics such as eye tracking, navigation, and neuroscientific measures like EEG, ECG, and GSR. These biomarkers could predict consumer facets like demographics and personality traits. 2. Contextual information's influence on consumers' cognitive processes in virtual retailing was analyzed, including factors like product layout and banners affecting stress, confusion, and joy levels. 3. Social cues within virtual stores were found to influence consumer behavior, with avatars adapting based on shoppers' stress levels to enhance the shopping experience. 4. Novel EEG metrics were explored to understand narrative cognition in video advertisements and paintings, leading to increased engagement and shared understanding across subjects. 5. Nonlinear dynamical systems theory was employed to assess cognitive states through observed signals from modalities like fMRI, fNIRS, and EEG, enabling classification of cognitive states and exploring neural substrates. 6. The brain-heart interplay and its role in cognition and emotions were studied, providing insights into emotional responses' physiological basis and opportunities for neuromarketing. 7. Functional near-infrared spectroscopy (fNIRS) was used to estimate respiratory rate during physical activity, demonstrating potential applications in stress assessment and sports studies. 8. The uHD EEG system was developed, enabling high-resolution brain activity measurement for various applications like sports science and medical diagnostics. 9. Real-time applications of the uHD EEG system were explored, with wireless streaming allowing for brain activity-based control mechanisms in virtual reality environments, revolutionizing fields like gaming and rehabilitation. 10. More reliable methods to measure emotional and cognitive responses in virtual store settings were developed using EEG-based metrics and facial electromyography, contributing to understanding and measuring consumer responses. RHUMBO's achievements span scientific, educational, and training domains, nurturing young scientists and disseminating neuroscience-based business methods to various stakeholders. These advancements have the potential to reshape consumer behavior analysis, marketing strategies, and virtual reality interactions.

Data: CORDIS, © European Union