Leader: Antonio Lanatà (UNIFI); Other collaborator(s):
In this task innovative wearable textile sensors for physiological monitoring along with intelligent platform integration will be implemented employing spatially distributed systems model to ensure security and reliability of the information. This task includes innovative signal processing and artificial intelligence tools for psychophysiological assessment and physical body activity analysis. Developed methods will enable the investigation of user engagement, cognitive loading, emotional response and mood dynamics to prevent physical and mental dangerous conditions. Furthermore, the dynamics of emotional face expression and eye gaze pattern will be processed for quantitative index extraction emotional wellness.
During the specified period (November 2023 - March 2024), our activity focused on different experimental designs for the psychophysiological characterization of young and adult subjects by analysing multivariate physiological signals obtained from biosensors and imaging techniques. In particular, a public database of EEG and fMRI data relating to an adult population considered at risk of developing Alzheimer Disease (AD) was preliminarily explored, along with a genotypic and phenotypic investigation. In experimental designs proposed to a healthy population, electrophysiological characterization of the participants was carried out through analysis of Event-Related Potentials (ERP) on words and pseudowords aloud reading task, a clinical protocol widely adopted by speech therapists and laryngologists to diagnose speech and language disorders. The ERP analysis was also proposed in an experimental design to understand subjects' implicit attitudes through the Implicit Association Test (IAT). In this experimental paradigm, eye gaze, blinking activity and variation of the pupillary diameter were investigated. Bibliographical research focused on rehabilitation and treatment aspects of elderly population with neurodegenerative disorders has been carried out. Particular attention has been paid to the study of the effect of non-invasive brain stimulation (tES and rTMS) in experimental contexts related to multi-sensory stimulation integrated into contexts of extended reality (XR) or serious games (SG). Finally, new acquisitions utilizing wearable sensors were performed on an experimental design to investigate possible different emotional and cognitive responses on real or AIgenerated human faces. Additional questionnaires to establish the participants' memory abilities and to understand the adopted strategy during the proposed task have been implemented.
Please see the next reporting period.
This report details activities conducted from July to September 2024, focusing on clustering analysis of vocal properties of 287 patients diagnosed with benign lesions of the vocal folds (BLVF) and unilateral vocal fold paralysis (UVFP), which are among the highest causes of dysphonia. The research aimed to identify if these patients constituted separate vocal subtypes of dysphonia and to understand whether misclustered data could depend on a specific diagnosis and age. This was done primarily to identify straightforward differences in vocal fold motor dynamics that generally are assessed with visual inspection through high- resolution endoscopy, which is not always available. Also, a particular emphasis was placed on misclustered observations to understand the role of confounding factors (i.e., the BLVF subtypes and age) in clustering analysis. Data collection regarded 287 patients recruited at the Ospedale Maggiore Policlinico di Milano. The diagnosis was made by perceptual voice evaluation (GRB scale) and video- laryngostroboscopic assessment. 136 patients presented UVFP and 151 BLVF. For the acoustic analysis, patients were asked to utter the sustained vowel /a/ for at least 3s. Acoustic parameters were extracted, accounting for age, gender and type of vocal emission. Specifically, fundamental frequency F0, local jitter, adaptive normalised noise energy, the first three formants, the duration of voiced and unvoiced parts, and the median, standard deviation, minimum and maximum values were used for further analysis. The k-means algorithm was applied to find clusters on the event space by iteratively assigning each point to the nearest candidate cluster centre and updating the cluster’s centres until convergence. The silhouette score offered a means for assessing the clustering quality by comparing inter-cluster distances with intra-cluster ones. Unaware and aware analyses were performed for clustering, whilst a misclustered data analysis was additionally performed to provide interpretable results for clinicians. Key findings for the female dataset in the unaware condition report that the best results were obtained with the principal component analysis (PCA). Also, a higher silhouette score for the UVFP class could be helpful in clinical practice for better recognising the most severe of the considered pathologies, possibly reducing its misdiagnosis. Results from the male dataset remain similar between the unaware and aware conditions. Moreover, in both genders, misclustered observations seem to be independent of a specific pathology (and its subtypes). Finally, the PCA weight analysis highlighted that phonation parameters were the most contributive. In conclusion, the results showed valuable preliminary data on voice disorders with clustering techniques on voice features, offering a new perspective for clinical practice and professionals.
During October-December 2024, we undertook several significant initiatives to advance research in physiological signal acquisition and analysis in clinical and experimental contexts. Below is a summary of the key activities conducted during this period:
These studies are relevant to ageing and healthy ageing, exploring the physiological underpinnings of emotional and social interactions. Emotional regulation and social interactions are critical for maintaining mental health and cognitive function in older adults, underscoring the broader implications of this research. Future efforts will further refine these methodologies and expand collaborations to amplify the impact of our findings.
Scientific publications
From January to March 2025, Task 5.5 focused on advancing the understanding of psychophysiological and neurocognitive dynamics in social, affective, and clinical contexts through a series of experimental studies; the results have been consolidated in four peer-reviewed conference papers.
A major line of investigation explored the role of electrodermal activity (EDA) in characterising emotional responses within dyads exposed to affective stimuli. In a controlled setting involving 22 participants (11 dyads), variations in EDA patterns were found to reflect both intra- and inter-subject dynamics as arousal levels increased, thereby supporting the use of EDA as a reliable marker in hyperscanning protocols to assess emotional synchrony and co-regulation in social contexts.
In parallel, a second study introduced a novel approach to intersubject neurophysiological monitoring by integrating shared biofeedback mechanisms in dyadic affective tasks. EEG and ECG signals were jointly acquired using a custom-developed hyperscanning system in 24 dyads, enabling real-time cross-subject monitoring. Notably, heart rate variability (HRV) metrics, especially cross-entropy indices, were found to vary across baseline and stimulation phases, suggesting that intersubject biofeedback may influence autonomic synchrony and warrants further exploration as a modulator of interpersonal dynamics during joint cognitive-affective processing.
The task also contributed to the clinical voice assessment field by implementing a multimodal evaluation framework that combined acoustic analysis with sentiment scoring of transcribed free speech. In a cohort of 26 dysphonic patients, this approach effectively captured psychophysiological changes before and after surgical intervention. Findings indicated that male patients experienced measurable improvements in vocal stability. In contrast, female patients exhibited a strong correlation between pitch and positive sentiment, suggesting an intertwined evolution of vocal and emotional recovery. This methodology provides an efficient, automatic, and gender-independent tool to assess therapeutic outcomes beyond traditional self-report measures.
Finally, in serious games (SG), a study evaluated the effects of auditory cues on memory and visual attention using a gamified version of the Corsi Block-Tapping Test. Although no direct enhancement in memory performance was observed with auditory cueing, uniform auditory stimuli significantly altered eye-tracking metrics, indicating a shift in attentional processing. Specifically, fixation and saccade parameters were modulated in opposing directions depending on auditory condition, pointing to perceptual rather than memory-related effects of auditory cues.
These studies reflect Task 5.5’s continued commitment to advancing multimodal and multisystemic approaches for investigating human behavior in emotionally, cognitively, and clinically relevant contexts.
Scientific publications
EMBC 2025 (accepted contributions) – Copenhagen, Denmark
GEM 2025 (submitted contribution) - Kaohsiung, Taiwan
Mustafa Can Gursesli, Pietro Tarchi, Andrea Guazzini, Mirko Duradoni, Arda Tunc Yaras, Emir Akinci, Ulas Yurdakul, Federico Calà, Alessandro Tonacci, Daniele Vilone, Antonio Lanatà