Leader: Gabriella Cortellessa (CNR-ISTC); Other collaborator(s): Pirrelli, Maggi ,Marzi, Ferro (CNR), Cincotti (Sapienza), Pilotto (UNIBA), Ferrarese and Bellelli (UNIMIB), Riccardi (INRCA), Marcello Ferro (CNR), Alessandro Leone (CNR-IMM), Francesca Gasparini (UNIMIB), Patrizia Ribino (CNR-ICAR), Claudia Di Napoli (CNR-ICAR), Aurora Saibene (UNIMIB), RTD-A (Sapienza)
The task aims to develop novel ICT-based services integrating sensors (wearable, environmental) and AI technologies to support interventions while monitoring emotional, cognitive and functional status of individuals (including delirium detection). These services will be designed to support personalized and situated multicomponent interventions considering abstractions of sensors data, reasoning on knowledge graphs and inferring the state of both individuals (bedridden and non-bedridden) and environmental contexts to monitor onset/evolution of age-related diseases. A cognitive architecture inspired by the Dual process theory will be developed to provide assistance services (e.g., coaching, cognitive/physical training, situated dialogue, diet), delivered through different interaction devices (e.g., apps, social robots, smartTV). The task aims to design, develop and test an integrated multifunctional software-hardware platform for collection, storage, transcription, time-alignment, classification and quantitative analysis of digitised speech productions of people aged 65-85, engaged in a variety of language tasks: from elicited picture-based narratives to the oral reading of a connected text. The platform intends to support the assessment of a subject’s cognitive condition, as well as multicomponent clinical interventions, through the continual monitoring of the onset/evolution of age-related cognitive decline. Based on the use of an Internet-connected tablet frontend for data recording, and powered with remote, online AI services for language data processing and statistical analysis, the platform is designed to implement a minimally invasive, flexible protocol that can be adapted to different conditions, needs and settings. In addition, the battery of language-based services can be combined with other cognitive tests and biomarkers for a multicomponent clinical assessment of the prodromal phases and the longitudinal progress of dementia. Cognitive training (traditional or computer-based) aimed at improving cognitive functioning (memory, attention, processing speed and executive functioning) in elderly populations can result in long term improved functional outcomes. Neurofeedback training and training programs based on hybrid Brain-Computer interfaces have the potential to offer personalized feedback based on the real-time detection of the user’s brain activity. Clinical needs arised from WP1-3 will be taken into account.
Aims of this task are: (i) the development of a modular proof of concept solution (e.g. portable device for home use or more immersive stimulation setting for use in clinical settings) for the personalization of the cognitive training intervention and for its use even remotely; (ii) a feasibility assessment on a small sample of users.
Brief description of the activities and of the intermediate results
The research activities in the frameword of Task 5.3 are related to different contribution from CNR istitutes. CNR-ISTC is active in The COmbined deduction and abduCtiOn logic reasoner (COCO), a cognitive architecture inspired by the dual process theories which integrates a rule-based system with a timeline-based planner to plan, execute activities, and react to dynamic changes; preliminary results include personalized planning, integration of reactive and deliberative components either without or with embodiment through robots, tablets or smartphones, depending on the needs and available resources to be possibly tested in WP3. CNR-ILC adapted and tested an existing ICT infrastructure (Readlet) with a view to screening cohorts of elderly subjects (aged 65-80 years) on a battery of linguistic and cognitive tasks. In the first phase of the present activity (March-September 2023), ILC assessed the suitability of the current protocol for our purposes, by testing the reading performance of a cohort of about 60 normally ageing adults aged 65-75 (40 women, mean age: 68.6, and 21 men, mean age: 69.3), with an above-average schooling background (mostly high school graduates), hosted in long-term care facilities in the province of Milan. CNR-IMM investigated:
The Sapienza University in Rome is involved in:
CNR-IMM Activities:
CNR-ILC Activities:
CNR-ISTC Activities
Sapienza Activities
EEG data collected from 17 healthy participants were analyzed in the framework of the development of training and monitoring of cognitive functions in elderly persons based on Brain-Computer Interface (BCI) technology. As for the spectral EEG indices defined in the literature as descriptor of pathological condition (mild cognitive impairment, dementia), our results showed that those indices changed even as we get older and, therefore, can be monitored to catch signs of cognitive decline. As for the delta to alpha ratio, a moderate correlation between the value of the index and the age resulted in the parietal areas (Pz, p=0.05, r= 0.48; P7, p=0.008, r= 0.68). As for the theta to alpha ratio (TAR), a moderate correlation between the value of the index and the age was observed in the parietal and occipital areas (P3, p=0.05, r= 0.48). As for the alpha and beta frontal asymmetry, our results did not highlight correlation between the alpha frontal asymmetry and the age (p>0.05) but revealed an increased beta frontal asymmetry as function of the age.
CNR-IMM Activities:
1. Design/implementation (C#) of fall detection algorithms by using 3-axial IMU device integrated in commercial e-shirts (Smartex WWS). Design and implementation of the first release of the GUI for end-user interaction.
2. Debugging of Fall Detection algorithms ported in Python for Google FitBit Versa 4 smartwatch usage in several “controlled” conditions (simulated falls through imbalance provided by tilting platform).
3. Integration of new “rules” in the clinical protocol for sarcopenia evaluation by using commercial sEMG devices. In particular the usage of arms has been foreseen, extending the previous approach in which only legs was monitored.
CNR-ISTC Activities:
From October to December 2024, CNR-ISTC focused on refining the ontology classes of the CoCo system to enable preliminary experimentation with low-TRL technologies in controlled environments. These efforts resulted in significant improvements to the system's stability and robustness. To further enhance the system's capabilities, we are planning to initiate a collaboration with the Neuroscience Institute (IN) to improve the planning of activities, aiming to make the CoCo system more proactive in responding to users' needs. Additionally, a collaboration with the Institute of Computational Linguistics (ILC) is being planned to integrate reading monitoring tools, which will serve as a mechanism for assessing cognitive decline. Complementing these efforts, activities have begun to integrate a social robot into the CoCo system. This robot will enable social interactions with users, driven by the system's reactive and deliberative capabilities, further advancing the system’s ability to support and engage with users in a dynamic and interactive manner. Additionally, in collaboration with Milano Bicocca, CNR-ISTC also worked at the definition of objectives and identification of potential user types for a platform designed to manage and integrate data collected from various types of sensors (wearables, environmental sensors, etc.). The latest activities focus on structuring the platform's content and creating initial wireframes to outline how users will access and navigate information related to doctors, elderly individuals (or specific subgroups of the elderly), sensors, and activity scenarios. The platform design also considers the connections between these elements, aiming to provide a comprehensive and intuitive framework that will support effective data analysis and meaningful insights for doctors and researchers.
CNR-ILC Activities:
Publications