Leader: Andrea Corvi (UNIFI); Other collaborator(s): Pietro Benvenuti (UNIFI)
We propose an innovative clinically-inspired AI-based system encompassing wearable inertial and physiological sensors. The idea is to combine motor data with information coming from brain dynamics to identify cues of subtle decline in subjects' performance, suggesting the onset of neurodegenerative syndromes. This task aims to develop an integrated platform for at-home self-assessment and self-monitoring, which rely on trustworthy and cutting-edge AI algorithms and architectures for biomechanical and neurophysiological parameters extraction and analysis. Notably, the data will be gathered through gold-standard clinical protocols and groundbreaking tasks, which allows to describe the motor actity as an emergent property of the brain.
During the specified timeframe (November 2023 - March 2024), our research endeavors focused on developing an algorithm aimed at systematically exploiting quaternion data derived from inertial sensors (XSens), for integration into the OpenSim musculoskeletal modeling software. Our primary objective was to explore a methodology that leverages Principal Component Analysis (PCA) in conjunction with the Denavit Hartenberg principle to effectively utilize quaternions for reconstructing movement kinematics. In parallel, we conducted a comparative analysis of four distinct models available within the open-source repository of OpenSim. This comparison was centered on evaluating the models across various parameters including degrees of freedom, muscle count, and the articulation of the shoulder joint. Our goal in this comparative study was to identify the most suitable model for simulating a reach-to-grasp movement accurately. Furthermore, during this period, significant efforts were directed towards defining an experimental setup tailored for conducting reach-to-grasp movements. This setup was meticulously designed to facilitate the execution of precise and controlled experimental trials aimed at gathering empirical data essential for validating our modeling approaches and assessing their real-world applicability.
Main policy, industrial and scientific implications:
In a collaborative effort with KU Leuven University, we delved into a comprehensive exploration of various features offered by the open-source musculoskeletal modeling software OpenSim.
Please see the next reporting period.
During the last three months, the research activities of this Task focused on biomechanics simulation using musculoskeletal modeling software, with a particular interest in the reach-to-grasp movement. This research aims to explore motor control planning and improve the accuracy of movement simulation. Accurate simulations of these movements, especially when interacting with objects, require precise measurements of the forces exchanged between the person and the object. External force data plays a crucial role in solving the inverse dynamics problem, allowing for the estimation of internal forces, such as muscle activations. This is particularly relevant in applications involving object manipulation like reach to grasp movement or manual interactions, such as passive movements conducted by clinicians to assess joint stiffness.
In this period, the main task was focused on developing a flexible force sensor capable of measuring the reactions that occur during interactions with objects or between individuals. To achieve this, we explored a novel approach to create flexible sensors using textile-based electronics, known as e-textiles. This method aims to integrate flexibility and durability, allowing for the development of sensors that could adapt to dynamic and deformable surfaces. Two distinct force sensors were designed, developed, and tested: a resistive force sensor (FSR) and a capacitive force sensor (FSC). The FSR was created using an embroidery machine with conductive threads, while the FSC was assembled using a plastic foam dielectric and conductive textile with conductive silicone. Both sensors underwent compressive mechanical tests using an Instron machine and an LCR meter, with design parameters adjusted to optimize sensor performance. These tests aimed to identify the optimal sensor parameters to accurately detect changes in applied force, according to each sensor's operating principle. We are now facing the problem of replicating the sensor characteristic measure on a portable microcontroller.
During the last three months, the research activity of task 5.4 was concentrated on addressing the muscle activation optimization problem in musculoskeletal modeling software, with a specific focus on Reach-to-Grasp (RG) movements. This work is pivotal for advancing the understanding of motor planning and control, particularly in aging and pathological contexts. RG movements are highly complex, involving multiple degrees of freedom and the redundancy of muscle activation, which necessitates solving an optimization problem to estimate internal forces and muscle activations accurately.
A significant aspect of this period was the development and implementation of a custom optimization framework to resolve the muscle redundancy problem. Unlike traditional static optimization approaches, which aim to minimize the sum of squared activations without explicitly considering experimental EMG data, the proposed method incorporates both musculoskeletal models and experimental data from IMUs and EMG signals. The optimization framework was designed to align simulated muscle activations with recorded EMG signals while minimizing the sum of muscle activations.
This custom optimization approach was mathematically formalized to minimize muscle activation discrepancies while meeting the biomechanical constraints of the motion. The cost function included terms to minimize both muscle activations and the error between simulated activations and recorded EMG signals. It was applied to 10 participants performing RG movements, specifically the "drink gesture," repeated 10 times per subject.
The results of this approach were compared with standard static optimization methods provided by OpenSim. Key performance metrics, such as correlation coefficients and RMSE, were evaluated between the simulated activations and the experimental EMG data. The custom optimization consistently demonstrated higher correlation values (ranging from 0.641 to 0.966) and lower RMSE for most subjects, indicating a significant improvement in accuracy. However, Subject 8 showed a slight negative correlation but maintained a lower RMSE, suggesting minor alignment issues that require further investigation.
During this period, I also had the opportunity to present these findings at the poster session of Florence Ageing Research Center held in Florence on December 19 2024.
During this trimester, significant progress was achieved in developing a sensor-to-segment calibration method and in formulating optimization functions for estimating muscle activations. These efforts culminated in an experiment involving 20 healthy adult subjects (13 males, 7 females; 29.8 ± 3.5 years old) aimed at validating the kinematic reconstruction of upper-body movements. The experiment integrated three inertial measurement unit (IMU) sensors placed on torso, bicep and forearm, with a motion capture (MOCap) system that employed eight infrared cameras and twelve reflective markers.
In parallel, the gathered data is being leveraged to refine an optimization function designed to estimate muscle activity via five electromyography (EMG) sensors positioned on the pectoral, biceps, triceps, deltoid, and dorsal muscles.
Participants were seated and instructed to perform three types of reach-to-grasp tasks:
For each task, subjects completed 10 repetitions with standardized positions and distances.
A preliminary analysis is currently in progress, focusing on metrics such as the Root Mean Square Error (RMSE), Range of Motion (ROM), and R² values to compare the movement reconstructions derived from the IMU and MOCap systems. This will validate the kinematic reconstruction made with wearable sensors against the gold standard of motion capture systems. The next phase will focus on finalizing the optimization framework and further validating its effectiveness in replicating muscle activation patterns during RG movements.
Over the past quarter, our work has centered on two complementary streams: a custom muscle‑activation optimization and comparative evaluation of upper‑limb musculoskeletal models.
Personalized Muscle‑Activation Optimization
We built a MATLAB pipeline to infer individual muscle activation profiles from the torques produced by inverse‐dynamics (via OpenSim), enforcing that the net moment at each joint equals the modeled muscular contribution. To accommodate modeling errors and measurement noise, we introduced nonnegative “slack” variables that allow slight violations of exact equilibrium when strictly necessary but are heavily penalized to ensure that deviations remain minimal.
The optimization unfolds in two stages. First, a box‑constrained least‑squares solver produces an initial estimate of muscle activations (each constrained between zero and full activation) by minimizing the squared difference between modeled torques and those predicted by the current activation guess. The residual between those torques drives the initial values of the slack variables, capturing any imbalance. Second, we refine this solution with a nonlinear optimizer that jointly adjusts activations and slack variables to minimize the sum of squared muscle activations plus a weighted penalty on any slack usage, while respecting the original equilibrium and bound constraints.
We validated this approach against surface EMG recordings (pectoralis, biceps, triceps, deltoid, and dorsal muscles) collected from 20 healthy subjects performing a variety of upper‑limb tasks. The pipeline achieved an average root‑mean‑square error (RMSE) of just 0.26 % between estimated activation levels and normalized EMG signals. A manuscript about this work is currently in preparation.
Comparative Evaluation of Upper‑Limb Musculoskeletal Models
In parallel, we recruited 16 healthy adults to perform four isolated joint movements (shoulder adduction, flexion, rotation; elbow flexion) and two functional reach‐to‐grasp tasks (“Place in Cup” and “Pass to Partner”). The setup combined four IMUs (mounted on thorax, scapula, upper arm, forearm) with an optical motion‐capture system (eight cameras, 15 markers) as reference.
We tested three OpenSim models (Wu, Seth, Holzbaur), driving each model with IMU‐derived orientations to compute joint angles. Model outputs were compared to the optical reference using RMSE, Pearson’s correlation, range‐of‐motion error (ΔROM), and relative error. A voting scheme across these metrics identified the best‐performing model for each degree of freedom and task. The Holzbaur model prevailed in 12 of 18 RMSE votes, 11 of 18 correlation votes, 11 of 18 ΔROM votes, and 12 of 18 relative‐error votes. It achieved sub‑7° RMSE and <10 % relative error in all shoulder degrees of freedom, and sub‑4° RMSE with correlation >0.97 in elbow flexion. Notably , comparable performance was maintained even when omitting the scapular IMU—using only three sensors—which simplifies the setup and improves wearer comfort. The manuscript “Comparative Evaluat ion of Upper Limb Musculoskeletal Models for IMU‑Based Reach‑to‑Grasp Protocols” is in advanced publication stage.
Future Developments
Successful discrimination would demonstrate the combined MSK+IMU system’s sensitivity in clinical‐rehabilitation protocols, enabling objective, quantitative assessment of patient performance in reach‐to‐grasp activities.
Scientific publications