Leader: Federico Carpi (UNIFI); Other collaborator(s):
This task includes the design, development and implementation of novel, advanced materials-based technologies, enabling unprecedented cognitive and motor rehabilitation programmes to be performed at home, specifically conceived for the elders. The technologies will especially include advanced materials-based sensing and actuation components, with special design and implementation, in order to obtain fully wearable, easy-to-use and low-cost rehabilitation systems. Furthermore, the sensing and actuation technologies will be integrated with simple and bespoke virtual-reality environments, in order to enable innovative and engaging rehabilitation programs.
Brief description of the activities and of the intermediate results:
During the concerned period, the research activities have covered the following topics:
- Completion of a second batch of psychophysical tests on the wearable tactile display technology and analysis of the data; the tests have compared how the softness of different objects is perceived within a virtual environment; a research manuscript has been submitted and is currently under revision.
- Design of a solution to control the contact area between the finger and the tactile display’s surface, so as to possibly improve the realism of the tactile perceptual experience.
- Completion of the first investigation with functional Near Infrared Spectroscopy (fNIRS), on healthy subjects, on the tactile perceptual response elicited by the tactile displays.
- Preliminary technical verifications with the company Tech4care on the possibility of integrating their visor-based virtual-reality technology with a wireless version of the tactile display.
Main policy, industrial and scientific implications:
The research activity concerned by this Task could generate a novel wearable technology for an unprecedented immersive cognitive experience in virtual reality systems. Indeed, it might engage not only vision, but also, for the first time, the sense of touch. This may improve the efficacy of virtual reality-based (serious games) cognitive rehabilitation of elders with Mild Cognitive Impairment, at home, in front of a computer screen.
Please see the next reporting period.
During the concerned period, the research activities have covered the following topics:
- Design and preliminary prototyping and testing of a wearable vibrotactile device integrated with the tactile display of softness, so as to generate tactile feedback that combines information on both softness and roughness of explored virtual objects; this is expected to significantly broaden the tactile display’s scope, enabling the possibility of mimicking more physical properties of objects and thereby making a virtual cognitive experience more realistic. In the long term, increasing realism is considered beneficial, as a way to potentially improve the cognitive rehabilitative performance of the system. Psychophysical tests are currently in progress.
- Completion of the design, fabrication and preliminary testing of the first device that allows for controlling the contact area between the finger and the tactile display’s surface, so as to possibly improve the realism of the tactile perceptual experience. Indeed, according to neurophysiological evidence, controlling the contact area is required, in order to effectively render the softness of virtual objects. A research manuscript has been submitted and is currently under revision.
- Data processing and statistical analysis of the results of the first investigation with functional Near Infrared Spectroscopy (fNIRS) on the tactile perceptual response elicited in healthy subjects by the tactile display. In particular, this activity intends to investigate how the user’s tactile perception (quantified by cortical activations detected via fNIRS) is affected by an interplay between the level of pressure generated by the tactile display and the level of attention from the user. The results are currently under evaluation.
During the concerned period, the research activities covered the following topics:
- Development of wearable tactile displays of softness: The activities were focused on: i) Testing of a wearable vibrotactile device (designed during the previous reporting period) integrated with the tactile display of softness, in order to combine feedback on both softness and roughness of explored virtual objects; the aim to broaden the tactile display’s scope, enabling the possibility of mimicking more physical properties of objects and thereby making a virtual cognitive experience more realistic. The long-term goal is to increase realism, to potentially improve the cognitive rehabilitative outcome. ii) Publication of a scientific paper that used the tactile display developed in this project to show how the tactile sensitivity to softness in virtual reality can increase when visual expectation and tactile feedback contradict each other. iii) Publication of a scientific paper on the first device that allows for controlling the contact area between the finger and the tactile display’s surface, so as to possibly improve the realism of the tactile perceptual experience. iv) Completion of the data analysis and beginning of the preparation of the first manuscript on the investigation, with functional Near Infrared Spectroscopy (fNIRS), of the tactile perceptual response elicited in healthy subjects by the tactile display. The data have been analyzed to clarify how the user’s tactile perception (quantified by cortical activations detected via fNIRS) is affected by an interplay between the level of pressure generated by the tactile display and the level of attention from the user. The manuscript is expected to be ready for submission during the next reporting period.
- Development of wearable sensors for monitoring the posture of subjects affected by senile cognitive-motor decline: During this reporting period an additional activity within Task 2.2 was initiated. It concerns the development of wearable sensors for monitoring the posture of the human body in subjects with senile cognitive-motor decline, especially elderly people within the home environment. In particular, the objective is the creation of monitoring systems based on capacitive and resistive elastomeric sensors. Strategies based on both concentrated sensors and distributed sensors will be designed. The research activity will include the development of appropriate sensor reading electronics, with wireless data transmission. During this period, the activity was focused on reviewing the state of the art.
During the concerned period, the research activities covered the following topics:
1) Development of wearable tactile displays of softness for cognitive rehabilitation in virtual reality: The activities were focused on:
- Completion of the first set of tests on the prototype wearable vibrotactile device developed during the previous reporting period. The device was integrated with the previously developed tactile display of softness, so as to enrich a virtual reality experience, potentially useful for cognitive rehabilitation of MCI patients. The integration of the two devices provides users with feedback on both softness and roughness of explored virtual objects. This strategy is motivated by the intention of expanding the tactile display’s functions, by using such a wearable interface to mimic more physical properties of virtual objects and, thereby, making a virtual cognitive experience more realistic. To assess the perceptual performance of the new device, psychophysical tests were performed, whose results are being evaluated at present, to be described in the next report.
- Preparation of a manuscript for a joint review paper, co-authored by partners involved in Spoke 9 - WP2, of the AgeIT project. The paper is intended to belong to a broader special issue on the AgeIT project, in preparation for the Journal of Gerontology. The tentative title of the review paper is ‘Digital Biomarkers for Robotic and Virtual Reality-Based Diagnostics and Rehabilitation in Ageing: a Review’. The aim is to describe the state of the art of the use of digital biomarkers as indicators of physiological and behavioral processes, collected from digital devices, for cognitive and physical assessment in older adults.
- Preparation of the first manuscript on the investigation, with functional Near Infrared Spectroscopy (fNIRS), of tactile perceptual responses stimulated in healthy subjects by the developed wearable tactile display. Cortical activations detected by fNIRS are used to investigate whether the combination of the specific performance of fNIRS (in terms of accuracy) with the specific performance of the novel tactile display (in terms of tuneability of the tactile pressure on fingers) is adequate to differentiate responses that are elicited by different conditions (stimulation with different pressure intensities, stimulation of the dominant or non-dominant hand, measurements from the right or left hemisphere). This analysis was found to be more complex than expected, due to confounding factors, which require improvements in statistical analysis procedures. The final results are expected to be available by the end of the next reporting period.
2) Development of wearable sensors for monitoring the posture of subjects affected by senile cognitive-motor decline: This additional activity (which was initiated during the previous reporting period) concerns the development of wearable sensors for monitoring the posture of the human body. During this period, the activity was focused on completing a detailed revision of the state of the art and selecting the most suitable sensing strategies to design systems that can easily be used to monitor elderly people within the home environment. As expected (according to a preliminary early evaluation), the review confirmed that the most suitable technologies are based on capacitive and resistive elastomeric (stretchable) sensors, to be arranged on wearable substrates, and to be combined with inertial measurement units (IMU). Furthermore, electromyographic recordings via portable low-cost instrumentation were selected as a useful complementary technology, to gather additional information about muscular activations, so as to enrich the evaluation of the user’s state. The next goal is to develop such sensors, integrate them into a unified monitoring system and enable their use with suitable reading electronics, capable of wireless data transmission for all the sensors at the same time.
Scientific publications
- Frediani, G., Carpi, F. (2024). Tactile sensitivity to softness in virtual reality can increase when visual expectation and tactile feedback contradict each other, Journal of Neural Engineering, 21, 066041. https://doi.org/10.1088/1741-2552/ada0e8
- Frediani, G., Carpi, F. (2024). How to make the skin contact area controllable by optical calibration in wearable tactile displays of softness. Sensors, 24, 6770. https://doi.org/10.3390/s24206770
- F. Carpi, “Wearable pneumatic tactile display of softness for virtual reality”, Proc. of EuroEAP 2024 – Twelfth International Conference on Soft Transducers and Electromechanically Active Polymers, Stuttgart, Germany, 11-13 June 2024.
- Carpi, F., Valles, M.C., Frediani, G., Toci, T. Grippo, A. (2023)."EEG investigation on the tactile perceptual performance of a pneumatic wearable display of softness." Actuators, 12, 431. DOI: 10.3390/act12120431.
- G. Frediani, F. Carpi, “Wearable tactile display of softness for virtual reality”, Proc. of GNB 2023 - VIII Congress of the National Bioengineering Group, Padova, Italy, 21-13 June 2023.
- F. Carpi, “Wearable pneumatic tactile display of softness for virtual reality”, Proc. of IEEE World Haptics 2023, Delft, The Netherlands, 10 July 2023.
- G. Frediani, F. Carpi, “Wearable pneumatic tactile displays of softness for virtual reality”, Proc. of EuroEAP 2023 – Eleventh International Conference on Soft Transducers and Electromechanically Active Polymers, Bristol, United Kingdom, 6-8 June, 2023.
Dissemination Events
- F. Carpi, Keynote Lecture, “Wearable pneumatic tactile display to render the softness of virtual reality objects”, Eurohaptics 2024 Conference, ‘Enabling artificial agents to communicate with humans through touch’ Workshop, Lille, France, 30 Giugno 2024.
- F. Carpi, Keynote Lecture, “Wearable pneumatic tactile displays of softness for virtual reality”, IEEE Haptics Symposium 2024, Long Beach, California, USA, 7 April 2024.