A mel-spectrogram is a spectrogram whose frequency axis has been warped to the mel scale, which spaces frequencies the way human hearing perceives them rather than linearly. It compresses the high frequencies and stretches the low ones, giving a speech model a view that lines up with how people actually tell sounds apart. It is the front-end feature most current recognizers expect.
How a mel-spectrogram works
You start from an ordinary spectrogram produced by the short-time Fourier transform. A filterbank of overlapping triangular filters, usually 40 to 128 of them, maps the linear frequency bins onto mel-spaced bands and sums the energy in each.
The mel scale itself is close to linear below about 1 kHz and roughly logarithmic above it, which is why low-frequency detail survives while the highs get grouped. Taking the logarithm of those band energies produces the log-mel spectrogram: compact, perceptually weighted, and numerically well behaved for a neural network to consume.
Why speech models use mel-spectrograms
The mel scale matches human auditory perception, so a mel-spectrogram emphasizes the frequency detail that carries speech and downweights the rest. It also has far fewer dimensions than a raw spectrogram, which makes it cheaper to compute and to train on.
That combination made it the default input for models like Whisper, and it is also why MFCCs, a further-compressed cepstral derivative, dominated feature extraction in the era before end-to-end models arrived.
Why the mel-spectrogram matters
The mel-spectrogram is the actual input to most speech models, so its parameters, the number of mel bands, the window size, and the frequency range, are part of a model's interface rather than a free choice.
Features computed differently at inference than in training degrade accuracy, which is why the mel-spectrogram configuration has to travel with the model. It sits at the boundary between raw audio and the network, alongside related feature extraction steps like the filterbank and cepstral coefficients.
Frequently asked questions
What is the difference between a spectrogram and a mel-spectrogram? A plain spectrogram spaces frequencies evenly; a mel-spectrogram remaps them onto the mel scale so spacing follows human hearing. The mel version is smaller and better suited to speech models.
Why do speech models use the mel scale? Because it mirrors how the ear resolves pitch, giving fine detail at low frequencies where speech lives and coarser bands higher up. That focuses the model on perceptually meaningful information.
What is a log-mel spectrogram? It is a mel-spectrogram with the logarithm applied to each band's energy. The log compresses the huge dynamic range of audio into values a neural network handles well.
How is a mel-spectrogram computed? Run an STFT to get a spectrogram, pass it through a mel filterbank to pool energy into mel-spaced bands, then take the log of each band. Many pipelines add MFCCs as a further step.