Speech Isolation for Keyword Spotting
A research paper built around one blunt question: if you scrub the noise out of audio before a model ever sees it, does keyword spotting actually get better?
The question, stated plainly
Keyword spotting works fine in a quiet room. The interesting case is a loud one. The intuition a lot of people reach for is to clean the audio first, then classify. For UVA's Deep Learning course I set out to test whether that intuition holds, and under what conditions it quietly fails.
How we set it up
We trained a CNN and an Audio Spectrogram Transformer on Google Speech Commands v0.02, then tested both under four noise types from the MUSAN corpus at signal-to-noise ratios from 20 dB all the way down to minus 5. Three preprocessing conditions ran against each: no filter at all, a bandpass tuned to the 300 to 3400 Hz speech band, and spectral gating.
My share of the paper covered the speech-recognition history, the dataset section, the proposed method, the experiments and evaluation metrics, and the implementation of spectrogram conversion and spectral gating.
94.76% versus 90.52% on clean audio.The CNN beat the transformer in every single condition, clean or noisy.
What the numbers said
The CNN won everywhere, from clean audio down to minus 5 dB with the best filter, where it held 81.94% against the transformer's 77.40%. The single biggest driver of robustness turned out to be noise-augmented training, more than any preprocessing choice. Bandpass filtering gave small, consistent gains for both models. Spectral gating helped the CNN a little and hurt the transformer badly, dropping it as much as 5.67 points at the noisiest setting, because the subtraction artifacts get amplified at the transformer's finer spectral resolution.
The takeaway is one I keep coming back to: you cannot choose a preprocessing strategy without knowing what runs downstream of it. A filter that helps one architecture can actively hurt another.