ECoG Demo

In our work: “Stimulus Speech Decoding from Human Cortex with Generative Adversarial Network Transfer Learning”, we use a transfer learning approach with a pre-trained GAN to predict spectrograms from ECoG signals. We further use a WaveNet Vocoder to generate speech waveforms from predicted spectrograms.

Audio Demo

We compared the ground truth audio, vocoder predicted audio as follows. With our transfer learning GAN and vocoder models, we can precisely reconstruct the audio from ECoG signals.

Ground truth and predicted audio pair of eight words: waldo, yich, game, padge, wish, jazz, waldo, dach.

 

Waveform plot:

Spectrogram: