[2211.02625] MAEEG: Masked Auto-encoder for EEG Representation Learning

Abstract:Decoding information from bio-signals such as EEG, using machine learning has been a challenge due to the small data-sets and difficulty to obtain labels. We propose a reconstruction-based self-supervised learning model, the masked auto-encoder for EEG (MAEEG), for learning EEG representations by learning to reconstruct the masked EEG features using a transformer architecture. We found that MAEEG can learn representations that significantly improve sleep stage classification (~5% accuracy increase) when only a small number of labels are given. We also found that input sample lengths and different ways of masking during reconstruction-based SSL pretraining have a huge effect on downstream model performance. Specifically, learning to reconstruct a larger proportion and more concentrated masked signal results in better performance on sleep classification. Our findings provide insight into how reconstruction-based SSL could help representation learning for EEG.

# link_ti9s1go2hg.pdf ## Metadata - PDFFormatVersion=1.5 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - CreationDate=D:20221107014453Z - Creator=LaTeX with hyperref - ModDate=D:20221107014453Z - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2 - Producer=pdfTeX-1.40.21 - Trapped=False ## Contents ### Page 1 MAEEG: Masked Auto-encoder for EEG Representation LearningHsiang-Yun Sherry Chien, Hanlin Goh, Christopher M. Sandino, Joseph Y. ChengApple{sherry.ch

saved by

• 6774

related reading

• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• The Annotated JEPA | Elements of a Vector Spaceelonlit.com
• Self-supervised learning - Wikipediaen.wikipedia.org
• Zyphrazyphra.com
• [2007.04871] Subject-Aware Contrastive Learning for Biosignalsarxiv.org
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com
• Self-Supervised Representation Learning | Lil'Loglilianweng.github.io
• Towards a “universal translator” for neural dynamics at single-cell, single-spike resolutionarxiv.org
• The Annotated Transformernlp.seas.harvard.edu
• On the Stepwise Nature of Self-Supervised Learning - Imbuegenerallyintelligent.com
• A Few Words on Representation Learning - Thalles' blogsthalles.github.io
• Frontiers | Decoding EEG Brain Activity for Multi-Modal Natural Language Processingfrontiersin.org