An Ensemble Method for EEG-based Texture Discrimination during Open Eyes Active Touch
Received: 29 September 2023 | Revised: 12 October 2023 | Accepted: 15 October 2023 | Online: 12 December 2023
Corresponding author: Alexandros Tzallas
Abstract
Touch sensation is a key modality that allows humans to understand and interact with their environment. More often than not, touch sensation depends on vision to accumulate and validate the received information. The ability to distinguish between materials and surfaces through active touch consists of a complex of neurophysiological operations. To unveil the functionality of these operations, neuroimaging and neurophysiological research tools are employed, with electroencephalography being the most used. In this paper, we attempt to distinguish between brain states when touching different natural textures (smooth, rough, and liquid). Recordings were obtained with a commercially available EEG wearable device. Time and frequency-based features were extracted, transformed with PCA decomposition, and an ensemble classifier combining Random Forest, Support Vector Machine, and Neural Network was utilized. High accuracy scores of 79.64% for the four-class problem and 89.34% for the three-class problem (Null-Rough-Water) were accordingly achieved. Thus, the methodology's robustness indicates its ability to classify different brain states under haptic stimuli.
Keywords:
multisensory, machine learning, PCA, ensemble methodDownloads
References
G. Robles-De-La-Torre, "The importance of the sense of touch in virtual and real environments," IEEE MultiMedia, vol. 13, no. 3, pp. 24–30, Jul. 2006.
C. Reed and M. Ziat, "Haptic Perception: From the Skin to the Brain," in Reference Module in Neuroscience and Biobehavioral Psychology, Amsterdam, Netherlands: Elsevier, 2018, pp. 545–556.
A. Gallace and C. Spence, "Touch and the body: The role of the somatosensory cortex in tactile awareness," Psyche: An Interdisciplinary Journal of Research on Consciousness, vol. 16, no. 1, pp. 30–67, 2010.
M. A. Symmons, B. L. Richardson, D. B. Wuillemin, and G. H. VanDoorn, "Active versus passive touch in three dimensions," in First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference, Pisa, Italy, Mar. 2005, pp. 108–113.
C. Simoes‐Franklin, T. A. Whitaker, and F. N. Newell, "Active and passive touch differentially activate somatosensory cortex in texture perception," Human Brain Mapping, vol. 32, no. 7, pp. 1067–1080, Jul. 2010.
S. Eldeeb, J. Ting, D. Erdogmus, D. Weber, and M. Akcakaya, "EEG-Based Texture Classification During Active Touch," in 29th International Workshop on Machine Learning for Signal Processing, Pittsburgh, PA, USA, Oct. 2019, pp. 1–6.
A. Moungou, E. Vezzoli, C. Lombart, B. Lemaire-Semail, J.-L. Thonnard, and A. Mouraux, "A novel method using EEG to characterize the cortical processes involved in active and passive touch," in Haptics Symposium, Philadelphia, PA, USA, Apr. 2016, pp. 205–210.
G. Anuradha and D. N. Jamal, "Classification of Dementia in EEG with a Two-Layered Feed Forward Artificial Neural Network," Engineering, Technology & Applied Science Research, vol. 11, no. 3, pp. 7135–7139, Jun. 2021.
S. Lederman and R. Klatzky, "TuTorial Review: Haptic Perception: A Tutorial," Attention, perception & psychophysics, vol. 71, no. 7, pp. 1439–1459, Oct. 2009.
A. Greco, A. Guidi, M. Bianchi, A. Lanata, G. Valenza, and E. P. Scilingo, "Brain Dynamics Induced by Pleasant/Unpleasant Tactile Stimuli Conveyed by Different Fabrics," IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 6, pp. 2417–2427, Aug. 2019.
F. McGlone, J. Wessberg, and H. Olausson, "Discriminative and Affective Touch: Sensing and Feeling," Neuron, vol. 82, no. 4, pp. 737–755, May 2014.
H. Singh et al., "The brain’s response to pleasant touch: an EEG investigation of tactile caressing," Frontiers in Human Neuroscience, vol. 8, 2014, Art. no. 893.
H. Alsuradi, W. Park, and M. Eid, "EEG-Based Neurohaptics Research: A Literature Review," IEEE Access, vol. 8, pp. 49313–49328, 2020.
M. T. Fairhurst, F. McGlone, and I. Croy, "Affective touch: a communication channel for social exchange," Current Opinion in Behavioral Sciences, vol. 43, pp. 54–61, Feb. 2022.
N. V. Kimmatkar and B. V. Babu, "Human Emotion Detection with Electroencephalography Signals and Accuracy Analysis Using Feature Fusion Techniques and a Multimodal Approach for Multiclass Classification," Engineering, Technology & Applied Science Research, vol. 12, no. 4, pp. 9012–9017, Aug. 2022.
A. Gutierrez, D. Sepulveda-Munoz, A. Gil-Agudo, and A. de los Reyes Guzman, "Serious Game Platform with Haptic Feedback and EMG Monitoring for Upper Limb Rehabilitation and Smoothness Quantification on Spinal Cord Injury Patients," Applied Sciences, vol. 10, no. 3, Jan. 2020, Art. no. 963.
A. C. Voos, K. A. Pelphrey, and M. D. Kaiser, "Autistic traits are associated with diminished neural response to affective touch," Social Cognitive and Affective Neuroscience, vol. 8, no. 4, pp. 378–386, Apr. 2013.
M. Grunwald, C. Ettrich, B. Assmann, A. Dahne, and W. Krause, "Deficits in haptic perception and right parietal theta power changes in patients with anorexia nervosa before and after weight gain," International Journal of Eating Disorders, vol. 29, no. 4, pp. 417–428, 2001.
X. Hou and O. Sourina, "Emotion-enabled haptic-based serious game for post stroke rehabilitation," in 19th ACM Symposium on Virtual Reality Software and Technology, Singapore, Asia, Oct. 2013, pp. 31–34.
B. Hughes, J. Wang, D. Rosic, and K. Palmer, "Texture Gradients and Perceptual Constancy under Haptic Exploration," in Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Tsukuba, Japan, Mar. 2007, pp. 66–71.
S. Biswas and Y. Visell, "Haptic Perception, Mechanics, and Material Technologies for Virtual Reality," Advanced Functional Materials, vol. 31, Feb. 2021, Art. no. 2008186.
D. Prattichizzo, C. Pacchierotti, S. Cenci, K. Minamizawa, and G. Rosati, "Using a Fingertip Tactile Device to Substitute Kinesthetic Feedback in Haptic Interaction," in International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, Amsterdam, Netherlands, Jul. 2010, pp. 125–130.
B. Petreca, N. Bianchi-Berthouze, S. Baurley, P. Watkins, and D. Atkinson, "An Embodiment Perspective of Affective Touch Behaviour in Experiencing Digital Textiles," in Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland, Sep. 2013, pp. 770–775.
A. Miltiadous, E. Gionanidis, K. D. Tzimourta, N. Giannakeas, and A. T. Tzallas, "DICE-Net: A Novel Convolution-Transformer Architecture for Alzheimer Detection in EEG Signals," IEEE Access, vol. 11, pp. 71840–71858, 2023.
S. Ballesteros, F. Munoz, M. Sebastian, B. Garcia, and J. M. Reales, "ERP evidence of tactile texture processing: Effects of roughness and movement," in Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Salt Lake City, UT, USA, Mar. 2009, pp. 166–171.
C. Genna, C. Oddo, C. Fanciullacci, C. Chisari, S. Micera, and F. Artoni, "Bilateral cortical representation of tactile roughness," Brain Research, vol. 1699, pp. 79–88, Nov. 2018.
C. Genna, F. Artoni, C. Fanciullacci, C. Chisari, C. M. Oddo, and S. Micera, "Long-latency components of somatosensory evoked potentials during passive tactile perception of gratings," in 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Orlando, FL, USA, Aug. 2016, pp. 1648–1651.
A. Moungou, J.-L. Thonnard, and A. Mouraux, "Using EEG (SS-EPs) to characterize the brain activity in response to textured stimuli in passive touch," in IEEE World Haptics Conference, Evanston, IL, USA, Jun. 2015, pp. 113–118.
J.-H. Cho, J.-H. Jeong, M.-K. Kim, and S.-W. Lee, "Towards Neurohaptics: Brain-Computer Interfaces for Decoding Intuitive Sense of Touch," in 9th International Winter Conference on Brain-Computer Interface, Gangwon, Korea, Feb. 2021, pp. 1–5.
M.-K. Kim, J.-H. Cho, and J.-H. Jeong, "Classification of Tactile Perception and Attention on Natural Textures from EEG Signals," in 9th International Winter Conference on Brain-Computer Interface, Gangwon, Korea, Feb. 2021, pp. 1–5.
O. Ozdenizci, S. Eldeeb, A. Demir, D. Erdogmus, and M. Akcakaya, "EEG-based texture roughness classification in active tactile exploration with invariant representation learning networks," Biomedical Signal Processing and Control, vol. 67, May 2021, Art. no. 102507.
G. Baghdadi, M. Amiri, E. Falotico, and C. Laschi, "Recurrence quantification analysis of EEG signals for tactile roughness discrimination," International Journal of Machine Learning and Cybernetics, vol. 12, no. 4, pp. 1115–1136, Apr. 2021.
A. Greco, M. Nardelli, M. Bianchi, G. Valenza, and E. P. Scilingo, "Recognition of affective haptic stimuli conveyed by different fabrics sing EEG-based sparse SVM," in 3rd International Forum on Research and Technologies for Society and Industry, Modena, Italy, Sep. 2017, pp. 1–5.
M. A. Becerra et al., "Electroencephalographic Signals and Emotional States for Tactile Pleasantness Classification," in International Workshop on Artificial Intelligence and Pattern Recognition, Havana, Cuba, Sep. 2018, pp. 309–316.
N. Katsuyama, E. Kikuchi-Tachi, N. Usui, H. Yoshizawa, A. Saito, and M. Taira, "Effect of Visual Information on Active Touch During Mirror Visual Feedback," Frontiers in human neuroscience, vol. 12, Jan. 2018, Art. no. 424.
A. Khasnobish, A. Konar, D. N. Tibarewala, S. Bhattacharyya, and R. Janarthanan, "Object Shape Recognition from EEG Signals during Tactile and Visual Exploration," in International Conference on Pattern Recognition and Machine Intelligence, Kolkata, India, Dec. 2013, pp. 459–464.
W. Li, Q. Xu, Y. Li, C. Li, F. Wu, and L. Ji, "EEG characteristics in ‘eyes-open’ versus ‘eyes-closed’ condition during vibrotactile stimulation," Biomedical Signal Processing and Control, vol. 68, Jul. 2021, Art. no. 102759.
J. F. Veale, "Edinburgh Handedness Inventory – Short Form: A revised version based on confirmatory factor analysis," Laterality, vol. 19, no. 2, pp. 164–177, Mar. 2014.
N. S. Williams, G. M. McArthur, B. de Wit, G. Ibrahim, and N. A. Badcock, "A validation of Emotiv EPOC Flex saline for EEG and ERP research," PeerJ, vol. 8, Aug. 2020, Art. no. e9713.
"Follow why does Emotiv EPOC+ stop at 43 Hz in range? Is there a way to expand this somehow?," EMOTIV, Dec. 12, 2018. https://www.emotiv.com/knowledge-base/follow-why-does-emotiv-epoc-stop-at-43-hz-in-range-is-there-a-way-to-expand-this-somehow/.
J. J. Newson and T. C. Thiagarajan, "EEG Frequency Bands in Psychiatric Disorders: A Review of Resting State Studies," Frontiers in Human Neuroscience, vol. 12, Jan. 2019, Art. no. 521.
H. Abdi and L. J. Williams, "Principal component analysis," WIREs Computational Statistics, vol. 2, no. 4, pp. 433–459, 2010.
I. H. Witten, E. Frank, L. E. Trigg, M. A. Hall, G. Holmes, and S. J. Cunningham, "Weka: Practical machine learning tools and techniques with Java implementations," Working Paper, Aug. 1999. [Online]. Available: https://researchcommons.waikato.ac.nz/handle/10289/1040.
T. Head, M. Kumar, H. Nahrstaedt, G. Louppe, and I. Shcherbatyi, "scikit-optimize/scikit-optimize." Zenodo, Jul. 12, 2021.
K. Polat and S. Gunes, "Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform," Applied Mathematics and Computation, vol. 187, no. 2, pp. 1017–1026, Apr. 2007.
S. Ruggieri, "Efficient C4.5 [classification algorithm]," IEEE Transactions on Knowledge and Data Engineering, vol. 14, no. 2, pp. 438–444, Mar. 2002.
L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001.
K. D. Tzimourta et al., "A robust methodology for classification of epileptic seizures in EEG signals," Health and Technology, vol. 9, no. 2, pp. 135–142, Mar. 2019.
M. A. Alsuwaiket, "Feature Extraction of EEG Signals for Seizure Detection Using Machine Learning Algorthims," Engineering, Technology & Applied Science Research, vol. 12, no. 5, pp. 9247–9251, Oct. 2022.
M. Saeedi, A. Saeedi, and A. Maghsoudi, "Major depressive disorder assessment via enhanced k-nearest neighbor method and EEG signals," Physical and Engineering Sciences in Medicine, vol. 43, no. 3, pp. 1007–1018, Sep. 2020.
K. G. Sheela and S. N. Deepa, "Review on Methods to Fix Number of Hidden Neurons in Neural Networks," Mathematical Problems in Engineering, vol. 2013, Jun. 2013, Art. no. e425740.
I. Tsoulos, D. Gavrilis, and E. Glavas, "Neural network construction and training using grammatical evolution," Neurocomputing, vol. 72, no. 1, pp. 269–277, Dec. 2008.
J. Heaton, Artificial Intelligence for Humans: Deep Learning and Neural Networks. Scotts Valley, CA, USA: CreateSpace Independent, 2015.
R. Fu, Y. Tian, T. Bao, Z. Meng, and P. Shi, "Improvement Motor Imagery EEG Classification Based on Regularized Linear Discriminant Analysis," Journal of Medical Systems, vol. 43, no. 6, May 2019, Art. no. 169.
K. D. Tzimourta, L. G. Astrakas, M. G. Tsipouras, N. Giannakeas, A. T. Tzallas, and S. Konitsiotis, "Wavelet Based Classification of Epileptic Seizures in EEG Signals," in 30th International Symposium on Computer-Based Medical Systems, Thessaloniki, Greece, Jun. 2017, pp. 35–39.
J. Breneman, "Kernel Methods for Pattern Analysis," Technometrics, vol. 47, no. 2, pp. 237–238, May 2005.
A. Lahouar and J. Ben Hadj Slama, "Day-ahead load forecast using random forest and expert input selection," Energy Conversion and Management, vol. 103, pp. 1040–1051, Oct. 2015.
R. Chatterjee and T. Bandyopadhyay, "EEG Based Motor Imagery Classification Using SVM and MLP," in 2nd International Conference on Computational Intelligence and Networks, Bhubaneswar, India, Jan. 2016, pp. 84–89.
A. Miltiadous et al., "Alzheimer’s Disease and Frontotemporal Dementia: A Robust Classification Method of EEG Signals and a Comparison of Validation Methods," Diagnostics, vol. 11, no. 8, Aug. 2021, Art. no. 1437.
P. Christodoulides et al., "Classification of EEG signals from young adults with dyslexia combining a Brain Computer Interface device and an Interactive Linguistic Software Tool," Biomedical Signal Processing and Control, vol. 76, Jul. 2022, Art. no. 103646.
V. Aspiotis et al., "Active touch classification using EEG signals," in 6th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference, Preveza, Greece, Sep. 2021, pp. 1–5.
H. Alsuradi and M. Eid, "Trial-based Classification of Haptic Tasks Based on EEG Data," in IEEE World Haptics Conference, Montreal, QC, Canada, Jul. 2021, pp. 37–42.
H. Alsuradi, C. Pawar, W. Park, and M. Eid, "Detection of Tactile Feedback on Touch-screen Devices using EEG Data," in IEEE Haptics Symposium, Crystal City, VA, USA, Mar. 2020, pp. 775–780.
Downloads
How to Cite
License
Copyright (c) 2023 Andreas Miltiadous, Vasileios Aspiotis, Dimitrios Peschos, Katerina D. Tzimourta, Al Husein Sami Abosaleh, Nikolaos Giannakeas, Alexandros Tzallas
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
