Information Extraction - Weeknd Edition
At UMBC there’s an introductory course on Information Systems (IS 101). I have not taught that course (as of now I only teach Juniors and Seniors) but if I did I’d have a homework assignment where I’d ask the students to watch the following video:
It’s someone recreating the Weeknd’s “Blinding Lights” on an iPhone. This blows my mind. Ten years ago the software required to create songs would require a laptop or even a workstation. After watching the video I’d ask the students what components were required to make this “information system” work. Decent answers would include the various chips, the circuit boards, the network connection required to download the software, various technical standards to be created (like USB), the operating system, and the GarageBand software itself. I’d also suggest that the human creating the music was the most important part of the information system. They’re talented.
However, the purpose of this article is to talk about information extraction and how it relates to music, using the Weeknd’s “Blinding Lights” as an example.
Regression Analysis
Any kind of “data analysis” (there are many terms that I could use that roughly mean the same thing) involves taking raw data and making some kind of sense of it. We go from a position of “not knowing” (entropy) to “knowing” (information/knowledge). I wrote an article about that.
Many of the subjects we try to study or analyze are natural phenomena but some are human-made. A crime analyst analyzes crime data to find patterns, but a crime is a human activity. Sociology is “the study of the development, structure, and functioning of human society (Wikipedia) and sociologists (like most social scientists) employ statistical techniques like regression (one of my favorites) to understand human-created phenomena. In my IS 425 class I explain that one of the purposes of most statistical methods of machine learning techniques is to “explain away” this phenomena whether it’s natural or created by people. We want to statistically understand the factors (dependent variables) that can explain why something (the independent variable) happened.
Synthesis
So back to music: I’m not a music major or even a musician so I can’t claim to be an expert on the production of music but I do try to understand the process of “creating” in general, so I’ll do my best.
When The Weeknd and his producers (all very talented people) created “Blinding Lights” they took a series of component parts: their creativity, a series of musical instruments to create sound and a series of hardware and software to mix the component sounds created by the instruments to create the final product, a song. That one song is a synthesis of many parts.
Music and Me
I wasn’t always into popular music. When I was in high school I was only interested in classical music and in college I only really listened to heavy metal. As I’ve gotten older I’ve learned to enjoy and appreciate most genres of music, but I have always believed that Country music peaked with Garth Brooks in the early 90s. It’s a bit unfortunate that I didn't take the time to appreciate a lot of music from the mid 90s (a great time for music) in the mid 90s.
People listen to music mostly for the emotional response and sometimes for the intellectual response. Different choices made in the construction of a song by the artists and producers can have varying effects on people.
Deconstruction
Can a computer program analyze the audio of a song and come to conclusions about the effect that song might have on people? Even further, can a computer program analyze that same audio and figure out the artist’s intent? Is there a difference?
Sometimes these are easy to figure out (for both a person and for a computer):
Does a song have a fast tempo or a slow tempo? Does the tempo change during the song?
Is the song in a major or a minor key?
Then it gets harder:
Is it a party song? A sad song? A slow-dance song? A ballad?
Tempo and key might help here, but it’s probably not enough.
What genre is the song? Rap? Heavy Metal?
Can we identify the instruments used in creating the song and does that determine the genre of the song?
Then it gets even harder:
I listen to some songs that just sound “nostalgic” by design.
There are songs that personally remind me of periods of time in my life. I am very uncomfortable listening to anything by Mumford and Sons because I was dating someone when they broke out and their music was on the radio constantly back then. I guess an algorithm wouldn’t know that.
A human with enough skill and talent can listen to a song and deconstruct the components of that song in their head. In the video at the start of this article, someone knew enough about the components of the song to recreate it in GarageBand on their phone. Can human intuition into the structure of music be translated into something computers can use?
Techniques
There are several potential machine learning techniques we could use to analyze music in the ways described earlier. I’m sure people have already attempted and perhaps succeeded at these techniques, but I want to go through some possibilities as an exercise:
A Fourier analysis to deconstruct the sound waves in the music file to determine the pitch and timbre of the instruments used in the song.
Feeding thousands of existing songs that have been tagged with various labels (like “party” or “nostalgic”) into a neural network to create a model of songs and then inputting new songs into that model.
A graph-based model reflecting different musical genres and how they’re connected.
Conclusion
There were a few topics described in this article:
How a song is constructed.
How music affects me personally.
How a computer might deconstruct music.
Some potential AI/ML techniques that could be used to deconstruct that music.
What’s important is that all of these factor into a good analysis of music. It’s not just a matter of picking the right tools. It is important to understand the problem and that means understanding the humans that both created the music and the humans that listen to it. Understanding nature and people is not just true for data analysis, it’s also true for information systems in general.