Brain-CLIPLM Reframes EEG-to-Text Decoding Around Compressed Semantic Meaning Rather Than Exact Wording

Summary

Researchers including first author Xiaoli Yang and senior author Shijian Li propose that non-invasive EEG signals are better suited to recovering compressed semantic meaning than reconstructing a sentence's exact wording. Their two-stage framework, Brain-CLIPLM, first uses contrastive learning to align EEG activity with a fixed set of keyword-level "semantic anchors," then uses a language model to reconstruct probable sentence meaning from those anchors. On a standard EEG-language benchmark, the approach retrieved the correct sentence within its top 5 candidates 67.6% of the time, and within its top 25 candidates 85% of the time, with control experiments showing the EEG signal itself — not just the language model's own guesses — was driving the improvement.