Research
Ultrasound-Based Intent Modeling and Control Framework for Neurorehabilitation
Funded by NSF CAREER Award # 2126570
Robotic therapies aim to improve limb function in individuals with incomplete spinal cord injury (iSCI). Modulation of robotic assistance in many of these therapies is achieved by measuring extant volitional strength of limb muscles. However, current sensing techniques are often unable to correctly measure voluntary strength of a targeted muscle. The difficulty is due to their inability to remove ambiguity caused by interference from activities of neighboring muscles. These discrepancies in the measurement can cause the robot to provide inadequate assistance or over-assistance. Improper robotic assistance slows function recovery, and can potentially lead to falls during robot-assisted walking. An ultrasound (US) imaging is proposed as a sensing technique to predict extant volitional strength of individuals with iSCI during walking. US imaging allows direct visualization and measurement of the muscle activity, thus gets minimal interference from neighboring muscles. The proposal facilitates a breakthrough rehabilitation therapy that uses non-invasive wearable ultrasound sensors to collect, monitor, and control muscle activity of persons with iSCI.
Relevant Publications
- Q. Zhang, A. Iyer, K. Lambeth, K. Kim, N. Sharma, “Ultrasound Echogenicity as an Indicator of Muscle Fatigue during Functional Electrical Stimulation, Sensors, 22, 335. 2022
- Q. Zhang, W. Clark, J. Franz, and N. Sharma, “Personalized Fusion of Ultrasound and Electromyography-derived Neuromuscular Features Increases Prediction Accuracy of Ankle Moment during Plantarflexion,” Biomedical Signal Processing and Control, 71, 103100, 2022
- Q. Zhang, A. Iyer, Z. Sun, K. Kim, and N. Sharma, “A Dual-modal Approach Using Electromyography and Sonomyography Improves Prediction of Dynamic Ankle Movement: A Case Study,” IEEE Transactions on Neural System and Rehabilitation Engineering, vol. 29, pp. 1944-1954, 2021
- Q. Zhang, K. Kim, and N. Sharma “Evaluation of Noninvasive Ankle Joint Effort Prediction Methods for Use in Neurorehabilitation using Electromyography and Ultrasound imaging,” IEEE Transactions on Biomedical Engineering, vol. 68, no. 3, pp. 1044-1055, March 2021
- Q. Zhang, K. Kim, and N. Sharma “Prediction of Ankle Dorsiflexion Moment by Combined Ultrasound Sonography and Electromyography,” IEEE Transactions on Neural Systems and Rehabilitation Engineering,” vol. 28, no. 1, pp. 318-327, 2020.
Ultrasound Imaging Based Modeling and Muscle Fatigue Detection during Functional Electrical Stimulation
Funded by NSF Award # 1646009
Hybrid neuroprostheses integrate functional electrical stimulation of leg muscles with electrical motors to generate torque in the lower extremities. Evaluation of FES-induced muscle fatigue is required to properly coordinate these two systems. Traditionally, this fatigue estimate is obtained from fatigue governing equations, which are highly dependent on offline force measurement during isometric contraction experiments. Ultrasound imaging technology potentially provides an alternative online method for fatigue estimation which is a more direct, accurate, and natural way to characterize system behavior in a real time hybrid neuroprosthetic control system.
Relevant Publications
- Q. Zhang, A. Iyer, K. Lambeth, K. Kim, N. Sharma, “Ultrasound Echogenicity as an Indicator of Muscle Fatigue during Functional Electrical Stimulation, Sensors, 22, 335. 2022
- V. Molazadeh, Q. Zhang, X. Bao, N. Sharma, “An Iterative Learning Controller for a Switched Cooperative Allocation Strategy during Sit-to-Stand Tasks with a Hybrid Exoskeleton,” IEEE Transactions on Control Systems Technology, 2022.
- V. Molazadeh, Q. Zhang, X. Bao, B. Dicianno, N. Sharma, “Shared Control of a Powered Exoskeleton and Functional Electrical Stimulation using Iterative Learning and Fatigue Optimization,” Frontiers in Robotics and AI, 8, 329, 2021.
- Z. Sheng, Z. Sun, V. Molazadeh, and N. Sharma, “Switched Control of an N-Degree-of-Freedom Input Delayed Wearable Robotic System,” Automatica, 125, 109455, 2021.
- Z. Sheng, N. Sharma, and K. Kim, “Ultra-high-frame-rate ultrasound monitoring of muscle contractility changes due to neuromuscular electrical stimulation,” Annals of Biomedical Engineering, vol. 49, pp. 262–275, 2021.
- Z. Sheng, N. Sharma, and K. Kim, “Quantitative Assessment of Changes in Muscle Contractility Due to Fatigue During NMES: An Ultrasound Imaging Approach,” IEEE Transactions on Biomedical Engineering,” vol. 67, no. 3, pp. 832-841, 2020.
Modeling, Control, and Sensing of a Micro-Swimming Robot Driven by Ultrasound
Funded by NSF Award # 1637815
The project aims to develop a wireless 3D-maneuverable, micro swimming drone to perform in vivo biomedical investigations or operations such as bio-sensing, drug delivery, microsurgery, bio-imaging, etc. The strategy consists of three parts: (1) biocompatible micro swimming drone which is propelled by microstreaming flow from ultrasound-activated bubble oscillation (2) real-time ultrasound (US) imaging and actuating system for in vivo performance (3) feedback control algorithms and state estimation algorithms for 3-D maneuverability.
Relevant Publications
- Q. Chen, F. Liu, Z. Xiao, N. Sharma, S. K. Cho, and K. Kim, “Ultrasound Tracking of the Acoustically Actuated Microswimmer,” IEEE Transactions on Biomedical Engineering,” vol. 66, no. 11, pp. 3231-3237, 2019.
Robust Model Predictive Control of a Hybrid Neuroprosthesis
Funded by NIH R03HD086529
Functional electrical stimulation (FES) and a powered exoskeleton are among some of the technologies that aim to restore walking in individuals with paraplegia. FES can be used to obtain desired muscle contractions in the lower limbs through external application of low-level repetitive electrical currents. A powered exoskeleton uses electric motor drives to move the lower-limb joints. Alone, each has limitations. We aim to combine these two technologies to create a hybrid neuroprosthesis that is more advantageous than an FES-based walking system or a powered exoskeleton alone. Specifically, we propose to design and evaluate a robust model predictive control (MPC) method for controlling a hybrid neuroprosthesis that can optimally allocate the control efforts between the motors and the FES while ensuring the control performance. The advantage of using MPC is that the controller optimizes both FES and motor despite system constraints. However, its implementation requires model knowledge and it is very challenging to overcome uncertainties in the musculoskeletal dynamics due to day-to-day parametric variations. Therefore, we are looking into enhancing the robustness of the MPC method.
A person with spinal cord injury participating in the experiments
Relevant Publications
- X. Bao, Z. Sheng, B. Dicianno, and N. Sharma, “A Tube-based Model Predictive Control Method to Regulate a Knee Joint with Functional Electrical Stimulation and Electric Motor Assist,” IEEE Transactions on Control Systems Technology, vol. 29, no. 5, pp. 2180-2191, Sept. 2021.
- X. Bao, V. Molazadeh, A. Dodson, Brad E. Dicianno, and Nitin Sharma, “Using Person-Specific Muscle Fatigue Characteristics to Optimally Allocate Control in a Hybrid Exoskeleton,” IEEE Transactions on Medical Robotics and Bionics, vol. 2, no.2, pp. 226-235, 2020.
- X. Bao, N. Kirsch, A. Dodson, and N. Sharma, “Model Predictive Control of a Feedback Linearized Hybrid Neuroprosthetic System with a Barrier Penalty,” Journal of Computational and Nonlinear Dynamics, vol. 14., no. 10, 2019.
- N. Kirsch, X. Bao, Brad E. Dicianno, and N. Sharma, “Model-based Dynamic Control Allocation in a Hybrid Neuroprosthesis,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 1, pp. 224-232,2018.
- N. Kirsch, N. Alibeji, and N. Sharma, “Nonlinear Model Predictive Control of Functional Electrical Stimulation,” Control Engineering Practice, vol. 58, pp 319-331, 2017.
- B. Doll, N. Kirsch, X. Bao, B.E. Dicianno, and N. Sharma, “Dynamic Optimization of Stimulation Frequency to Reduce Isometric Muscle Fatigue Using a Modified Hill-Huxley Model“, Muscle & Nerve, 2017.
Optimal and Adaptive Control of a Single Joint Hybrid Neuroprosthesis
Funded by NSF Award # 1511139
The overall objective of this project is to investigate a new class of optimal adaptive automatic controllers that control FES and an electric motor, both of which can act together or alone to achieve limb movement. These controllers are based on reinforcement learning principles that compute sub-optimal solutions in real-time. Particularly, an optimal adaptive control framework will be provided for the simultaneous control of FES and an electric motor. This control framework will be able to overcome the challenges in the control of a hybrid exoskeleton such as dissimilar dynamics of its effectors and decreasing control gain due to the onset of muscle fatigue – a major limitation in FES-based rehabilitation interventions.
Relevant Publications
- V. Molazadeh, Q. Zhang, X. Bao, N. Sharma, “An Iterative Learning Controller for a Switched Cooperative Allocation Strategy during Sit-to-Stand Tasks with a Hybrid Exoskeleton,” IEEE Transactions on Control Systems Technology, 2022.
- V. Molazadeh, Q. Zhang, X. Bao, B. Dicianno, N. Sharma, “Shared Control of a Powered Exoskeleton and Functional Electrical Stimulation using Iterative Learning and Fatigue Optimization,” Frontiers in Robotics and AI, 8, 329, 2021.
- X. Bao, Z.H. Mao, P. Munro, Z. Sun, and N. Sharma, “Sub-optimally Allocating FES-Assist in a Hybrid Neuroprosthetic System with a Multi-layer Neural Network Structure, International Journal of Intelligent Robotics and Applications, vol. 3, no. 3., pp. 298-313, 2019.
- N. Alibeji, B. E. Dicianno, and N. Sharma, “Bilateral Control of Functional Electrical Stimulation and Robotics-based Telerehabilitation,” International Journal of Intelligent Robotics and Applications, vol. 1, no. 1, pp. 6–18, 2017.
- X. Bao, Z. Sun, and N. Sharma, “A Recurrent Neural Network-based MPC for a Hybrid Neuroprosthesis System“, IEEE Conference on Decision and Control, Melbourne, Australia 2017, pp. 4715-4720.
- N. Sharma, N. Kirsch, N. Alibeji, and W. E. Dixon “A Nonlinear Control Method to Compensate for Muscle Fatigue During Neuromuscular Electrical Stimulation,” Frontiers in Robotics and AI, vol. 4, pp. 68, 2017.
Input Dimensionality Reduction in a Hybrid Walking Neuroprosthesis by Using Control Approaches Inspired from Muscle-Synergies
Funded by NSF Award # 1462876
A hybrid walking neuroprosthesis that combines functional electrical stimulation (FES) with a powered lower limb exoskeleton provides therapeutic benefits of FES and torque reliability of the powered exoskeleton. Moreover, by harnessing metabolic power of muscles via FES the hybrid combination has a potential of lowering power consumption and reducing actuator size for the powered exoskeleton. Its control design, however, must overcome the challenges of actuator redundancy due to the combined use of FES and electric motor. Further, to maintain stability and control performance when disparate dynamics of FES and electric motor are combined, dynamical disturbances such as electromechanical delay (EMD) and muscle fatigue must be considered during the control design process.
We have developed a general framework to coordinate FES of multiple gait-governing muscles with electric motors is presented. A muscle synergy-inspired control framework is used and is motivated mainly to address the actuator redundancy issue. The synergies between FES of the muscles and the electric motors are artificially generated through optimizations. These synergies were used in the feedforward path of the control system. A dynamic surface control technique, modified with a delay compensation term, is used as the feedback controller to address model uncertainty, the cascaded muscle activation dynamics, and EMD.
Relevant Publications
- N. Alibeji, V. Molazadeh, F. Moore-Clingenpeel, and N. Sharma, “A Muscle Synergy Inspired Control Design to Coordinate Functional Electrical Stimulation and a Powered Exoskeleton,” IEEE Control Systems, 38 (6), pp. 35-60, 2018
- N. Alibeji, V. Molazadeh, B. Dicianno, and N. Sharma, “A Control Scheme that uses Dynamic Postural Synergies to Coordinate a Hybrid Walking Neuroprosthesis: Theory and Experiments,” Frontiers in Neuroscience, section: Neural Prostheses for Locomotion, vol. 12, pp. 159, 2018.
- N. Alibeji, N. Kirsch, and N. Sharma, “An Adaptive Low-Dimensional Control to Compensate for Effector Redundancy and FES-Induced Muscle Fatigue in a Hybrid Neuroprosthesis,” Control Engineering Practice, vol. 59, pp. 204–2019, 2017.
- N. Alibeji, N. Kirsch, and N. Sharma, ” A Muscle Synergy-inspired Adaptive Control Scheme for a Hybrid Walking Neuroprosthesis,” Frontiers in Bioengineering and Biotechnology, section Bionics and Biomimetics, vol. 3, no. 203, 2015.