Seeing noise?

Some of you may be familiar with synesthesia, a neurological condition where you perceive sensory input from one sense as if it were another sense–with synesthesia the color yellow might taste like root beer, or the sound of a bassoon may feel like bread dough. Even without synesthesia, however, linguists (particularly phoneticians and phonologists) see sound all the time. What does it look like? Something like this:

These, boys and girls and others, are what your speech sounds look like. Spectrograms are one of the most useful tools in the speech scientist’s tool shed. Heck, they’re pretty much a Swiss army shovel. You can spend your entire career basically only looking at data in this one form.

Why? Well, there’s a lot of data in a spectrogram. Big things, like whether a sound’s a ‘b’ or a ‘p’ (there’s a big black bar on the bottom if it’s a ‘b’, but not if it’s a ‘p’), but also really small things that we as humans have have a really hard time hearing. Like, remember what I said earlier about your ears lying to you? Turns out it’s a lot easier to sort out the truth if you can see what you’re hearing. Plus, by looking at spectrogram we can quantify things like average vowel frequencies really quickly and easily. (Turns out, by the way, that you can [maybe, kinda, if you squint just right and have just the right voice sample] judge how tall someone is based on their vowel frequencies.)

But spectrograms aren’t just a serious scientific tool; they’re also pretty fun. Aphex Twin, an ambient musician (I mean, he makes music in the ambient genre, not that he provide background music at canape parties. Sheesh.) uses spectrograms as an art form. This song, for example, has a picture of his face encoded in it’s spectrogram. Give it a listen and see if you can find it!

On a more general note, the study of images made with sound is known as cymatics. I’m just going to leave this video here for the more physics-minded among you: