Korva is a scroll of the internet’s best ideas.
Hip hop hiding inside the blues
Shrinking the blues
The recording was a basic blues with an 11-year-old on drums and a rough guitar solo on top. The test was to find the smallest set of frequencies that would still make a publicly available genre classifier answer "blues." A 30-second clip contains about 240,000 frequency bins, far too many combinations to try one by one.
So the search worked by halving. The frequencies were split at random into chunks, the chunks were tested in combinations, the ones that weren't needed were thrown away, and the survivors were split again, over and over. What remained was a thin, buzzing hum, a bit like the noise some electric cars make at low speed. The classifier called it the blues.
Removing it instead
Then came the reverse. The frequencies that made up a complete explanation were set to zero, and the model stopped saying blues. That left a version where the guitar was quieter and the bass and drums were prominent. The classifier chose hip hop out of its ten genres and scored it at 83%.
Over and over, on many models, it worked the same way. Genre labels were coming from frequency patterns, and a classifier can be sure of itself while a human ear disagrees.
How can a clip hold 240,000 frequency bins?
Audio is turned into frequencies with a Fourier transform, which breaks a waveform into how much energy sits at each pitch. A 30-second clip sampled many thousands of times per second gives a very fine frequency resolution. Each bin is one narrow slice of pitch, and a classifier can take any of them as input. A human ear groups all of that into notes and timbre instead of tracking slices separately.
Why does the search work by halving?
Splitting a large set in half and testing each half is a divide-and-conquer strategy. If removing a chunk leaves the classifier's answer unchanged, that chunk was not needed and can be dropped for good. Software engineers use a close cousin of this idea, called delta debugging, to shrink a failing program input down to the smallest piece that still triggers the bug. Each round of halving cuts the remaining search space roughly in two.
Why can a model be confident and wrong?
A classifier's confidence score is a number computed from how strongly the input matches patterns it learned, not a measure of whether a human would agree. In 2013, Szegedy and colleagues showed that tiny changes to an image could make a network confidently misclassify it. The same gap between score and meaning shows up with audio. Models trained to pick among a fixed list of genres also have to choose one even when none fit.
More from Computerphile
