Вопросы радиоэлектроники. 2019; : 64-75
ПРИМЕНЕНИЕ МЕТОДОВ ГЛУБОКОГО МАШИННОГО ОБУЧЕНИЯ ИСКУССТВЕННЫХ НЕЙРОННЫХ СЕТЕЙ ДЛЯ ПРОЕКТИРОВАНИЯ АЛГОРИТМОВ РАСПОЗНАВАНИЯ ЭЛЕКТРОМИОГРАФИЧЕСКИХ СИГНАЛОВ В БИОНИЧЕСКИХ ПРОТЕЗАХ
https://doi.org/10.21778/2218-5453-2019-5-64-75Аннотация
Для полноценного применения научных и прикладных достижений в области бионического протезирования необходимо предоставить конечному пользователю удобный и естественный интерфейс управления протезом. В статье рассматриваются способы и подходы в сфере анализа сигналов, полученных посредством регистрации электрической активности мышц с поверхности кожи (поверхностная электромиография). Подобный сигнал по своей природе – нестационарный и нелинейный, зависящий от многих факторов. Интерфейс на основе поверхностной электромиографии в настоящее время имеет несколько нерешенных проблем, таких как недостаточная точность распознавания (ложные срабатывания), а также сильная задержка, связанная с обработкой и распознаванием сигнала. Поэтому актуально применение методов глубокого машинного обучения для повышения качества распознавания электромиографических сигналов. В ходе исследований был спроектирован и собран аппаратный комплекс, позволяющий регистрировать электрическую активность мышц, разработана система сбора данных, а также написаны алгоритмы распознавания жестов. В итоге удовлетворительный результат удалось получить с использованием технологий сверточной нейронной сети с двумерным входом, так как данный подход обладает явной трансляционной ориентацией. В дальнейшем планируется модификация архитектуры нейронной сети и алгоритмов обучения, а также эксперименты со структурой входных данных.
Список литературы
1. Brose S. W., Weber D. J., Salatin B. A., et al. The role of assistive robotics in the lives of persons with disability // American Journal of Physical Medicine Rehabilitation. 2010. Vol. 89. No. 6. P. 509–521.
2. Smith C. Bionic eyes, arms and spines are no longer science fiction. The Science Show, 2017. URL: http://www.abc.net.au/news/2017-02-16/bionic-eyes-arms-and-spines-no-longer-science-fiction/8266238 (дата обращения: 22.01.2018).
3. Merletti R., Farina D. Surface EMG processing: introduction to the special issue // Biomedical Signal Processing and Control. 2008. Vol. 3. Iss. 2. P. 115–117.
4. Николаев С. Г. Практикум по клинической электромиографии. 2-е изд. Иваново: Ивановская государственная меди- цинская академия, 2003. 264 с.
5. Shin Y. K., Proctor R. W., Capaldi E. J. A review of contemporary ideomotor theory // Psychological Bulletin. 2010. Vol. 136 (6). P. 943–974.
6. Ortiz-Catalan M., Håkansson B., Brånemark R. An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs // Science translational medicine. 2014. Vol. 6. Iss. 257. P. 257re6.
7. Ortiz-Catalan M., Håkansson B., Brånemark R., Delbeke J. On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion // BioMedical Engineering OnLine. 2012. Vol. 11. P. 33.
8. Grill W., Mortimer J. Stability of the input-output properties of chronically implanted multiple contact nerve cuff stimulating electrodes. IEEE transactions on rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society. 1998. Vol. 6 (4). P. 364–373.
9. Struijk J., Thornsen M., Lorsen J., Sinkjaer T. Cuff electrodes for long-term recording of natural sensory information // IEEE Engineering in Medicine and Biology Magazine. 1999. Vol. 18. Iss. 3. P. 91–99.
10. Loeb G., Peck R. Cuff electrodes for chronic stimulation and recording of peripheral nerve activity // Journal of Neuroscience Methods. 1996. Vol. 64. Iss. 1. P. 95–103.
11. Waters R., McNeal D., Faloon W., Clifford B. Functional electrical stimulation of the peroneal nerve for hemiplegia. Long-term clinical follow-up // The Journal of Bone and Joint Surgery. 1985. Vol. 67. Iss. 5. P. 792–793.
12. Tyler D. J., Durand D. M. Functionally selective peripheral nerve stimulation with a flat interface nerve electrode // IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2002. Vol. 10. Iss. 4. P. 294–303.
13. Yoo P. B., Durand D. M. Selective recording of the canine hypoglossal nerve using a multicontact flat interface nerve electrode // IEEE Transactions on Biomedical Engineering. 2005. Vol. 52. Iss. 8. P. 1461–1469.
14. Park H., Durand D. M. Motion control of the rabbit ankle joint using a flat interface nerve electrode // Muscle Nerve. 2015. Vol. 52. P. 1088–1095.
15. Horch K., Meek S., Taylor T. G., Hutchinson D. T. Object discrimination with an artificial hand using electrical stimulation of peripheral tactile and proprioceptive pathways with intrafascicular electrodes. IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society. 2011. Vol. 19. Iss. 5. P. 483–489.
16. Boretius T., Badia J., Pascual-Font A., et al. A transverse intrafascicular multichannel electrode (TIME) to interface with the peripheral nerve // Biosens Bioelectron. 2010. Vol. 26. P. 62–69.
17. Sharma A., Rieth L., Tathireddy P., et al. Long term in vitro functional stability and recording longevity of fully integrated wireless neural interfaces based on the Utah Slant Electrode Array // Journal of Neural Engineering. 2011. Vol. 8. Iss. 4. P. 1–6.
18. Nobre M. E., Lopes F., Cordeiro L., et al. Inspiratory muscle endurance testing: pulmonary ventilation and electromyographic analysis // Respiratory Physiology Neurobiology. 2007. Vol. 155. Iss. 1. P. 41–48.
19. Winter D. Biomechanics and motor control of human movement. 4th ed. New York: John Wiley, 2009. 384 p.
20. Amorim C. F., Hirata T. Behavior analysis of electromyographic activity of the masseter muscle in sleep bruxers // Journal of bodywork and movement therapies. 2010. Vol. 14. Iss. 3. P. 234–238.
21. Weiss J., Weiss L., Silver J. Easy EMG. A guide to performing nerve conduction studies and electromyography. Elsevier, 2016. 296 p.
22. Feldmann E., Grisold W., Russell J., Zifko U. Atlas of neuromuscular diseases: a practical guideline. Springer, 2005. 320 p.
23. Wisotzky E., Tseng V., Pohlman D. Pocket EMG. New York: Demos Medical, 2015. 176 p.
24. Kasman G., Wolf S. Surface EMG made easy. Scottsdale: Noraxon, 2002. 178 p.
25. Fall C. L., Turgeon P., Campeau-Lecours A., et al. Intuitive wireless control of a robotic arm for people living with an upper body disability. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015. P. 4399–4402.
26. Saponas S., Tan D., Morris D., Turner J. Making muscle-computer interfaces more practical. CHI ’10 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2010. P. 851–854.
27. Tan D., Saponas S., Morris D., Turner J., inventors. Wearable electromyography-based controllers for human-computer interface. United States patent US8170656. 2008.
28. Wolf M., Assad C., Vernacchia M., Fromm J., Jethani H. Gesture-based robot control with variable autonomy from the JPL BioSleeve. IEEE International Conference on Robotics and Automation. 2013. P. 1160–1165.
29. Lake S., Bailey M., Grant A., inventors. Method and apparatus for analyzing capacitive EMG and IMU sensor signals for gesture control. United States patent US9299248. 2013.
30. Brown W. F. The physiological and technical basis of electromyography. Butterworth-Heinemann, 1984. 524 p.
31. Юсевич Ю. С. Электромиография в клинике нервных болезней. М.: Медгиз, 1958. 128 с.
32. Englehart K., Hudgins B. A robust, real-time control scheme for multifunction myoelectric control // IEEE Transaction on Biomedical Engineering. 2003. Vol. 50. No. 7. P. 848–854.
33. Phinyomark A., Hirunviriya S., Limsakul C., Phukpattaranont P. Evaluation of EMG feature extraction for hand movement recognition based on euclidean distance and standard deviation. IEEE International Conference on Computer Telecommunications and Information Technology. 2010. P. 856–860.
34. Xueyan T., Yunhui L., Congyi L., Dong S. Hand motion classification using a multichannel surface electromyography sensor // Sensors. 2012. Vol. 12. Iss. 2. P. 1130–1147.
35. Hargrove L., Englehart K., Hudgins B. A comparison of surface and intramuscular myoelectric signal classification // IEEE Transaction on Biomedical Engineering. 2007. Vol. 54. No. 5. P. 847–853.
36. Englehart K., Hudgins B., Parker P. A wavelet-based continuous classification scheme for multifunction myoelectric control // IEEE Transaction on Biomedical Engineering. 2001. Vol. 48. No. 3. P. 302–311.
37. Oskoei M., Hu H. Myoelectric control systems a survey // Biomedical Signal Processing and control. 2007. Vol. 2. P. 275–294.
38. Boschmann A., Platzner M. Towards robust HD EMG pattern recognition: reducing electrode displacement effect using structural similarity. 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014. P. 4547–4550.
39. Stegeman D. F., Kleine B. U., Lapatki B. G., Van Dijk J. P. High-density surface EMG: Techniques and applications at a motor unit level // Biocybernetics and Biomedical Engineering. 2012. Vol. 32. No. 3. P. 3–27.
40. Sainath T. N., Mohamed A. R., Kingsbury B., Ramabhadran B. Deep convolutional neural networks for LVCSR. IEEE International Conference on Acoustics, Speech and Signal Processing. 2013. P. 8614–8618.
41. Cecotti H., Graeser A. Convolutional neural network with embedded Fourier transform for EEG classification. 19th International Conference on Pattern Recognition. 2008. P. 1–4.
42. Farina D., Jiang N., Rehbaum H., et al. The extraction of neural information from the surface EMG for the control of upper- limb prostheses: Emerging avenues and challenges // IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2014. Vol. 22. No. 4. P. 797–809.
43. Qiu J., Wang J., Yao S., et al. Going deeper with embedded FPGA platform for convolutional neural network. Proceedings of the 2016ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, USA. 2016. P. 26–35.
44. Chen Y., Krishna T., Emer J., Sze V. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks // IEEE Journal of Solid-State Circuits. 2017. Vol. 52. Iss. 1. P. 127–138.
Issues of radio electronics. 2019; : 64-75
USE OF DEEP MACHINE LEARNING METHODS OF ARTIFICIAL NEURAL NETWORKS FOR DESIGNING ALGORITHMS OF ELECTROMYOGRAPHY SIGNAL RECOGNITION IN BIONIC PROSTHESIS
https://doi.org/10.21778/2218-5453-2019-5-64-75Abstract
For sterling application of scientific and engineered achievements in field of bionic prosthesis it’s required to provide comfortable and natural human‑prosthesis interface for an end‑user. In this article we are looking into ways and methods of analysis of the signal collected through electromyography activity of muscles on the skin surface. Such signal is nonstationary and unstable by its nature, dependent on various factors. sEMG based interface has several unsolved problem at the moment, such as insufficient accuracy of recognition and noticeable delay caused by signal recognition and processing. Article is dedicated to application of deep machine learning required to provide decent recognition of electromyography signals. In the course of the research hardware was developed to register muscle activity. Data collecting system and algorithms of gesture recognition have been designed as well. In conclusion decent results were achieved by using convolutional neural network, with two‑dimensional input, since data stream has obvious translational orientation. In the future, modification of neural network architecture, learning algorithms and experiments with structure of data are planned.
References
1. Brose S. W., Weber D. J., Salatin B. A., et al. The role of assistive robotics in the lives of persons with disability // American Journal of Physical Medicine Rehabilitation. 2010. Vol. 89. No. 6. P. 509–521.
2. Smith C. Bionic eyes, arms and spines are no longer science fiction. The Science Show, 2017. URL: http://www.abc.net.au/news/2017-02-16/bionic-eyes-arms-and-spines-no-longer-science-fiction/8266238 (data obrashcheniya: 22.01.2018).
3. Merletti R., Farina D. Surface EMG processing: introduction to the special issue // Biomedical Signal Processing and Control. 2008. Vol. 3. Iss. 2. P. 115–117.
4. Nikolaev S. G. Praktikum po klinicheskoi elektromiografii. 2-e izd. Ivanovo: Ivanovskaya gosudarstvennaya medi- tsinskaya akademiya, 2003. 264 s.
5. Shin Y. K., Proctor R. W., Capaldi E. J. A review of contemporary ideomotor theory // Psychological Bulletin. 2010. Vol. 136 (6). P. 943–974.
6. Ortiz-Catalan M., Håkansson B., Brånemark R. An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs // Science translational medicine. 2014. Vol. 6. Iss. 257. P. 257re6.
7. Ortiz-Catalan M., Håkansson B., Brånemark R., Delbeke J. On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion // BioMedical Engineering OnLine. 2012. Vol. 11. P. 33.
8. Grill W., Mortimer J. Stability of the input-output properties of chronically implanted multiple contact nerve cuff stimulating electrodes. IEEE transactions on rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society. 1998. Vol. 6 (4). P. 364–373.
9. Struijk J., Thornsen M., Lorsen J., Sinkjaer T. Cuff electrodes for long-term recording of natural sensory information // IEEE Engineering in Medicine and Biology Magazine. 1999. Vol. 18. Iss. 3. P. 91–99.
10. Loeb G., Peck R. Cuff electrodes for chronic stimulation and recording of peripheral nerve activity // Journal of Neuroscience Methods. 1996. Vol. 64. Iss. 1. P. 95–103.
11. Waters R., McNeal D., Faloon W., Clifford B. Functional electrical stimulation of the peroneal nerve for hemiplegia. Long-term clinical follow-up // The Journal of Bone and Joint Surgery. 1985. Vol. 67. Iss. 5. P. 792–793.
12. Tyler D. J., Durand D. M. Functionally selective peripheral nerve stimulation with a flat interface nerve electrode // IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2002. Vol. 10. Iss. 4. P. 294–303.
13. Yoo P. B., Durand D. M. Selective recording of the canine hypoglossal nerve using a multicontact flat interface nerve electrode // IEEE Transactions on Biomedical Engineering. 2005. Vol. 52. Iss. 8. P. 1461–1469.
14. Park H., Durand D. M. Motion control of the rabbit ankle joint using a flat interface nerve electrode // Muscle Nerve. 2015. Vol. 52. P. 1088–1095.
15. Horch K., Meek S., Taylor T. G., Hutchinson D. T. Object discrimination with an artificial hand using electrical stimulation of peripheral tactile and proprioceptive pathways with intrafascicular electrodes. IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society. 2011. Vol. 19. Iss. 5. P. 483–489.
16. Boretius T., Badia J., Pascual-Font A., et al. A transverse intrafascicular multichannel electrode (TIME) to interface with the peripheral nerve // Biosens Bioelectron. 2010. Vol. 26. P. 62–69.
17. Sharma A., Rieth L., Tathireddy P., et al. Long term in vitro functional stability and recording longevity of fully integrated wireless neural interfaces based on the Utah Slant Electrode Array // Journal of Neural Engineering. 2011. Vol. 8. Iss. 4. P. 1–6.
18. Nobre M. E., Lopes F., Cordeiro L., et al. Inspiratory muscle endurance testing: pulmonary ventilation and electromyographic analysis // Respiratory Physiology Neurobiology. 2007. Vol. 155. Iss. 1. P. 41–48.
19. Winter D. Biomechanics and motor control of human movement. 4th ed. New York: John Wiley, 2009. 384 p.
20. Amorim C. F., Hirata T. Behavior analysis of electromyographic activity of the masseter muscle in sleep bruxers // Journal of bodywork and movement therapies. 2010. Vol. 14. Iss. 3. P. 234–238.
21. Weiss J., Weiss L., Silver J. Easy EMG. A guide to performing nerve conduction studies and electromyography. Elsevier, 2016. 296 p.
22. Feldmann E., Grisold W., Russell J., Zifko U. Atlas of neuromuscular diseases: a practical guideline. Springer, 2005. 320 p.
23. Wisotzky E., Tseng V., Pohlman D. Pocket EMG. New York: Demos Medical, 2015. 176 p.
24. Kasman G., Wolf S. Surface EMG made easy. Scottsdale: Noraxon, 2002. 178 p.
25. Fall C. L., Turgeon P., Campeau-Lecours A., et al. Intuitive wireless control of a robotic arm for people living with an upper body disability. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015. P. 4399–4402.
26. Saponas S., Tan D., Morris D., Turner J. Making muscle-computer interfaces more practical. CHI ’10 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2010. P. 851–854.
27. Tan D., Saponas S., Morris D., Turner J., inventors. Wearable electromyography-based controllers for human-computer interface. United States patent US8170656. 2008.
28. Wolf M., Assad C., Vernacchia M., Fromm J., Jethani H. Gesture-based robot control with variable autonomy from the JPL BioSleeve. IEEE International Conference on Robotics and Automation. 2013. P. 1160–1165.
29. Lake S., Bailey M., Grant A., inventors. Method and apparatus for analyzing capacitive EMG and IMU sensor signals for gesture control. United States patent US9299248. 2013.
30. Brown W. F. The physiological and technical basis of electromyography. Butterworth-Heinemann, 1984. 524 p.
31. Yusevich Yu. S. Elektromiografiya v klinike nervnykh boleznei. M.: Medgiz, 1958. 128 s.
32. Englehart K., Hudgins B. A robust, real-time control scheme for multifunction myoelectric control // IEEE Transaction on Biomedical Engineering. 2003. Vol. 50. No. 7. P. 848–854.
33. Phinyomark A., Hirunviriya S., Limsakul C., Phukpattaranont P. Evaluation of EMG feature extraction for hand movement recognition based on euclidean distance and standard deviation. IEEE International Conference on Computer Telecommunications and Information Technology. 2010. P. 856–860.
34. Xueyan T., Yunhui L., Congyi L., Dong S. Hand motion classification using a multichannel surface electromyography sensor // Sensors. 2012. Vol. 12. Iss. 2. P. 1130–1147.
35. Hargrove L., Englehart K., Hudgins B. A comparison of surface and intramuscular myoelectric signal classification // IEEE Transaction on Biomedical Engineering. 2007. Vol. 54. No. 5. P. 847–853.
36. Englehart K., Hudgins B., Parker P. A wavelet-based continuous classification scheme for multifunction myoelectric control // IEEE Transaction on Biomedical Engineering. 2001. Vol. 48. No. 3. P. 302–311.
37. Oskoei M., Hu H. Myoelectric control systems a survey // Biomedical Signal Processing and control. 2007. Vol. 2. P. 275–294.
38. Boschmann A., Platzner M. Towards robust HD EMG pattern recognition: reducing electrode displacement effect using structural similarity. 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014. P. 4547–4550.
39. Stegeman D. F., Kleine B. U., Lapatki B. G., Van Dijk J. P. High-density surface EMG: Techniques and applications at a motor unit level // Biocybernetics and Biomedical Engineering. 2012. Vol. 32. No. 3. P. 3–27.
40. Sainath T. N., Mohamed A. R., Kingsbury B., Ramabhadran B. Deep convolutional neural networks for LVCSR. IEEE International Conference on Acoustics, Speech and Signal Processing. 2013. P. 8614–8618.
41. Cecotti H., Graeser A. Convolutional neural network with embedded Fourier transform for EEG classification. 19th International Conference on Pattern Recognition. 2008. P. 1–4.
42. Farina D., Jiang N., Rehbaum H., et al. The extraction of neural information from the surface EMG for the control of upper- limb prostheses: Emerging avenues and challenges // IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2014. Vol. 22. No. 4. P. 797–809.
43. Qiu J., Wang J., Yao S., et al. Going deeper with embedded FPGA platform for convolutional neural network. Proceedings of the 2016ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, USA. 2016. P. 26–35.
44. Chen Y., Krishna T., Emer J., Sze V. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks // IEEE Journal of Solid-State Circuits. 2017. Vol. 52. Iss. 1. P. 127–138.
