Novel computational approaches for neural speech prostheses and causal dynamics of language processing
Project Summary
This project develop computational approaches that will advance our understanding of neural processing for human speech and drive novel clinical applications including speech neural prosthetics, critical for people who are unable to speak; and neurosurgical mapping of language cortex, a necessary procedure in tumor and epilepsy surgery. Our research is framed across three intertwining thrusts:
1) Developing deep-learning models that decode speech from neural signals recorded using intracranial depth (sEEG) and surface (ECoG) electrodes;
2) Developing efficient algorithms for estimating directed connectivity dynamics among a large number of electrode sites, essential for understanding the interaction across brain regions during cognitive processing;
3) Developing deep-learning models for predicting brain regions causally critical for language processing from sEEG/ECoG recordings without electrical stimulation.
Participants
Yao Wang, Principal Investigator, Lab Page
Adeen Flinker, Co-Principal Investigator, Lab Page
Amirhossein Khalilian-Gourtani, Postdoctoral Researcher
Xupeng Chen, PhD student
Nika Emami, PhD student
Chenqian Le, PhD student
Sponsor
This material is based upon work supported by the National Science Foundation under Grant No. 2309057. This research extends the work previously supported by NSF under Grant No. 1912286.
Decoding Speech from Intracranial Electrode Signals
Publications
Chen, X., Wang, R., Khalilian-Gourtani, A., Yu, L., Dugan, P., Friedman, D., Doyle, W., Devinsky, O., Wang, Y. and Flinker, A., 2024. A neural speech decoding framework leveraging deep learning and speech synthesis. Nature Machine Intelligence, pp.1-14. Press release
Wang, R., Chen, X., Khalilian-Gourtani, A., Yu, L., Dugan, P., Friedman, D., Doyle, W., Devinsky, O., Wang, Y. and Flinker, A., 2023. Distributed feedforward and feedback cortical processing supports human speech production. Proceedings of the National Academy of Sciences, 120(42), p.e2300255120. Press release
Chen, J., Chen, X., Wang, R., Le, C., Khalilian-Gourtani, A., Jensen, E., Dugan, P., Doyle, W., Devinsky, O., Friedman, D. and Flinker, A., 2024. Subject-Agnostic Transformer-Based Neural Speech Decoding from Surface and Depth Electrode Signals. bioRxiv.
Demo page
https://xc1490.github.io/swinTW