VIDEO
The sound of the future: Predictive analytics in the music industry
Predictive analytics has had a profound effect on the music industry and it continues to effect the way we interact with music, the way we consume music, and the way the music industry heads into the future.
Transcription of Audio Recording
The sound of the future
I’d like to talk to you tonight about “The Sound of the Future.” The title here refers to the idea that we use predictive analytics in the music industry to understand what it is people want to hear, which is going and already has had a profound effect on the music industry. But it’s going to continue to have a profound effect on the way that we interact with music, the way that we consume music, and the way the music industry heads into the future.
So, many of you, I’m sure, are familiar with the streaming service called Spotify. How many Spotify users have we? Yeah, just about everybody. You may not know that Spotify about a year and a half ago bought a predictive analytics firm called The Echo Nest. The Echo Nest, yeah, you called it, The Echo Nest is a firm based out of…well, it was originally from MIT. It’s Cambridge based. They purchased this predictive analytics firm, which led to some questions. Why did they do that? What were they trying to achieve?
This question becomes even more interesting when one considers that also, in the last two years, Pandora, the number one internet radio station, purchased Next Big Sound. That is a New York-based predictive analytics firm. Additionally, Apple Music purchased the U.K.-based predictive analytics firm Symmetric.
What I found interesting about this is they didn’t just partner with them. They didn’t hire some data scientists. They purchased firms which suggests to me that these music companies believed predictive analytics is essential to their business strategy.
Question in my mind was, “Why?” To answer that question, we first have to look a little bit at what the music industry actually is. So the music industry is a large, complicated thing made up of any players, some of whom we’ve heard of, some of whom we haven’t. I’m gonna break it down into a couple of its most important component pieces.
The music industry is a large, complicated thing made up of any players, some of whom we've heard of, some of whom we haven't. I'm gonna break it down into a couple of its most important component pieces.
Breaking down the music industry
We’ve got songwriters. Got as an example up there Mr. Max Martin. You may know Max Martin. He’s got something like 22 number 1 hits. You may not have heard of him because he sang none of them. He is a songwriter. He’s written most of the most recent Taylor Swift hits. He wrote that song about not being able to feel his face when he’s with you. Win the Grammy’s this year, the number one song Grammy, there were I think five nominees, two of them were written by him.
We have recording artists like the aforementioned, Taylor Swift, who famously pulled her entire catalogue from Spotify about a year and a half ago. We have producers. I’ve chosen to represent the producers by Mr. George Martin, who passed away just a couple of months ago. George Martin was the producer for The Beatles. So the producers are the ones responsible for overseeing the recording efforts of the recording artists who are recording the songs of the songwriters.
Generally speaking, they do this by being financed by labels. We used to have a lot of labels. We still have a lot of labels but they, generally speaking, are all owned by a very small number of companies. In the ’80s, it was the big eight. Now it’s the big three. If you’re not part of Warner Music Group, Universal Music Group, or Sony Music, you are considered an independent or an indie.
There’s one other major financial component to the music industry and that is live performance, which has become a more and more important part of making a living as a recording artist. There’s only really one major piece of the live performance and that’s the concert promoters. And there’s only one large player in the concert promoter space and that is Live Nation Entertainment. Not only are they the largest promoter, they also own a lot of the performing venues. So they are really the dominant player.
So there are many, many other parts to the music industry. Everything from, you know, copywriters to instrument makers but these are the major components. On the left, songwriters, the recording artists, the producers, the labels, they are creating recorded music. Generally speaking, that ends up on a media, some sort of medium. If we look at media, it’s kind of fun to do because it takes us through the history of the recording industry. So let’s do that.
We start with the record. Before the record, how did people consume music? Either through live performance or what they purchased was sheet music and they recreated music in their own homes. With the advent of recorded sound, the record became a huge influence over the changing music industry and gave birth to the recorded music industry. Huge success, lots of companies grew up around the record. It was dominant for a long time.
Then came the cassette tape. The cassette tape was great. It is small, it is lightweight, it is cheap. You can make your own, you can make mixes for people. It, unfortunately, destroyed a lot of the revenue that was being generated out of records because it was smaller and cheaper.
What supplanted that is the CD. The CD, many of us, looking around the room, probably grew up in the age of the CD. You probably purchased CDs in your youth. I don’t know when the last time you purchased a CD was. How many CDs did you purchase in 2015? Maybe zero. The CD was a big deal. This led to the heyday of the recording industry. This was the largest revenue spike the recording industry had ever seen. The recording industry feared the CD as a piece of technology because they were giving their customers the same thing that they had, a high-quality, digital representation of the sound. This made them very nervous understandably. The cassette, if you made a copy, it was low quality. The CD, you make a copy, it is extremely high quality. Those fears were unfounded. They made a ton of money, famously spent it in very profitably good ways.
And what supplanted the CD? Digital downloads. So we all…well, some of us maybe here remember. Whoever used Napster? Whoever used Kaza? Yeah, I remember I had a friend, who was maybe a little naïve, who said, “This is amazing, this Kaza. I can get any music instantly from anywhere.” He then ended up with a whole host of viruses on his computer. But he did have a lot of music. It was very exciting.
Of course, the record industry closed down Napster and launched iTunes. iTunes launched in 2003 giving birth to a huge, new industry of digital music. And completely destroying the revenue that the industry was seeing out of records and out of CDs, excuse me. And the record industry has still not yet recovered.
One of these four media is currently seen…is currently on the upswing. There is an increase of purchase from last year to this year in one of this four. Can you guess? Vinyl. Amazing, right? Now that is an interesting fact. It is culturally interesting. It is not revenue interesting. It’s interesting because it says something about the importance of the tactile sensation of owning something. The warm sound of listening to a record. But it doesn’t make a big impact on the bottom-line for the recording industry.
The big story right now, of course, is what is taking the place of digital download is streaming.
The big story right now, of course, is what is taking the place of digital download is streaming. So streaming, that’s a representation of a dozen streaming services. There is more than that. There are some you probably never even heard of because they’re not available in the U.S. Streaming is the big story. Let’s take a look at the revenues though for the music industry as a whole because there’s some interesting pieces here where streaming fits in.
Here’s global revenue for the music industry in billions of dollars. So, this is 15 years of data. If I told you there was a company that was a $26 billion company 15 years ago and this year, it is a $15 billion company. What would you think about the likelihood that that company was going to fade into irrelevance? It’s not a positive story. You could see a year-on-year loss from… This actually goes back even further. There’s been year-on-year loss since 1995, but you might have to squint. Take a look at 2015, it’s a tiny bit higher. This, by the way, is huge news. This came out two weeks ago. In 2015, the total revenues of the global music industry went up. This is a headline. You heard it here.
So what produced that bump? Well, it wasn’t CDs. Here’s an interesting chart. Here’s 10 years of CD sales. This is units in millions. We went from 650 million CDs sold in 2004 to only 150 million in 2014.
That is, percentage-wise, a significant drop. 2004, what I find interesting is this like a seismic change in our culture. You all bought CDs when you were younger. You bought zero last year. This is a huge cultural shift in the way that we consume music. And 2004 was not that long ago. That was the year of “American Idiot.” You remember “American Idiot.” You remember “Hey Ya!” That was not that long ago.
Okay. So if it was not CDs, is it digital downloads? Is that providing us the boost? Well, I already gave away the end of this story. No. Digital downloads has been doing great, right? 2003 was the advent of iTunes. Here we are. Oh, look at this, it’s amazing. We’re making our money back. We’re gonna recover. Everything is gonna be okay. 2012, we peak and it’s been a downwards decline ever since. And that is certainly going to continue. Okay. So it’s not CDs. It’s not digital downloads.
Here’s an interesting quote, “Music consumption exploded,” I love that, “exploded in 2015.” Remember that tiny little bump? “Music consumption exploded in 2015 helped by a surge in streaming worldwide.” This is from an international body called the International Federation of the Phonographic Universe Industry. It’s one of those old names but a modern org. This was released. This is actually a tweet and a sign of the times from April 12th in conjunction with the release of their 2015 report which has, like I said, happened 2 weeks ago.
You notice the word “consumption.” I think we can all identify with that. Has your consumption increased of music since the days when you had to stick a CD in a machine in order to listen to it? Nowadays, you might be on a subway, you might be listening to music. You could be listening to music right now. You could be listening to music on your way home, when you wake up. You could be listening in places that you don’t even have internet access because maybe you’ve got an offline mode in your Spotify subscription, right?
Here is U.S. streaming revenue in millions. I’m just looking at U.S. data but it’s similar worldwide. This is an interesting story. You remember CDs declining, digital downloads declining, streaming increasing. This is the bright spot right now in the recording industry and this is responsible for that tiny bump that you saw from 2014 to 2015.
Top line is total streaming. The next one down, the gray one, that’s paid subscriptions. The next one down, internet radio. Below that, on-demand ad-supported. This will become important in just a moment. So let’s dig into this a little bit. You notice on-demand ad-supported, the bottom line, it’s flat. Internet radio went up a little bit and then it’s flat. The only one that continues to go up is paid subscription and that’s responsible for that total line going up.
So what are these things? Paid subscriptions, well that’s Spotify if you pay for it. I won’t ask how many of you pay for it but some of you might. TIDAL, no free option, that’s the Jay Z-owned streaming service. Apple Music, that’s also…there was a three month trial, but that’s a paid-for subscription service. Those represent the paid subscriptions. In the internet radio, we have, of course, Pandora, major player and also the way these statistics are compiled. It also includes Sirius Satellite Radio.
And at the bottom, what is on-demand ad-supported? Any guesses? What does it even mean? What’s that?
Thomas: Indeed. It’s you don’t think about it but on-demand means you chose the song. I wanna hear “Uptown Funk,” you then go online and you google “Uptown Funk.” Immediately, what comes up is the Vevo or YouTube link. You click on that and then you watch a 30-second ad for an insurance company and then you get to watch “Uptown Funk.” That is on-demand ad-supported. So that’s YouTube and Vevo. It’s also actually the free version of Spotify when you get to listen to, often it’s not Spotify ads, between songs on Spotify.
That’s an interesting collection there. I’m only giving you representative. There are other players here besides these but that’s important to understand. The important thing here also to remember is on-demand ad-supported is the bottom line. That will become important later.
Here are U.S. revenues from the industry. On the upper right is digital downloads, 34%. The big story here and this is the headline, so you wanna take this one home to tell your friends. Streaming is at 34.3%. That is higher than digital downloads. 2015 was the year we crossed making more money for the industry on streaming than on digital downloads. Physical, i.e., CD sales generally speaking also includes vinyl, it’s still 28% so it’s still healthy. But the digital downloads actually crossed and made more money for the industry than physical several years ago. I think it was two years ago that we saw digital downloads becoming dominant. This is the year streaming became dominant.
The last one, synchronization. Obviously, it plays a small role. It’s any time music is played in conjunction with a moving image, with a moving picture. So commercials, movie placements, things like that. It actually is an important source of revenue for up-and-coming artists but it’s not important to the industry as a whole, as you can see. So this is an important story.
So streaming, if we take for a moment that that’s the most important story, let’s read this quote from our CEO over at Spotify Daniel Ek. This is a response to a Quora interview from just this February, just a few months ago. “The music economy – like so much of the rest of the economy from cars to publishing to news – is changing at it moves from an ownership model to an access model.” So think about that a moment. When you buy something, you expect to own it. When you bought a CD, you owned it.
What we are shifting to is an access model. When you subscribe to Netflix, you don’t expect to own the movies or “Orange is the New Black.” You just have access to them. This is a big shift. It’s much harder for older people. I know lots of older people who say, “I will never use streaming because I like to have it.” Adele, she said, “I not only download the album, I go out and I buy a physical copy of the same album because I know someone else won’t.” She doesn’t strike me as that old but she has an old person’s attitude toward ownership. Young people find this very easy, I pay for access. I only characterize old versus young not to make any of you feel bad. But because the young people’s attitude is the future of the industry.
The music economy - like so much of the rest of the economy from cars to publishing to news - is changing at it moves from an ownership model to an access model.
Spotify and Discover Weekly
So let’s take Daniel Ek at his word and let’s look more deeply into his firm. He is the CEO and founder of Spotify. What I wanna explore, in this middle section of this talk, is Spotify and a tool called Discover Weekly. It’s a feature. It was launched about a year ago and it has quickly become the most popular feature on Spotify.
So before we get into that more deeply, let’s remember…let’s just remind ourselves a little bit about Spotify. You probably know some of these statistics or some of this data about Spotify. But here’s something about the company itself. It was founded in just 10 years ago, almost exactly, in April of 2006. It’s headquartered in Sweden. They have 100 million active users. That is a large number. That is from January. Thirty million of them are paying. That might not have been intuitive to you. That is a large percentage paying, in my opinion. I thought it would be smaller but it says 30 million paying Spotify customers. That’s from last month.
Their revenues, now this is not a public company so we don’t have all the revenue information. But the last time that we had a report on revenues, 2014 revenues were 1.3 billion. Respectable. They are not profitable. Their losses were 197 million that same year. But their last round of funding, which garnered them another 550 million or so in investment, was last June and during that time, they were valued at $8.53 billion.
Now, I know some in our audience are technically minded so here’s a couple of little tidbits for you. Their database contains 30 million songs. They log 1 terabyte of user data every single day. They are running 1700-node Hadoop cluster and they run 10,000 plus Hadoop jobs every single day. It’s a big database. What are they doing with all that data? What are they doing with a terabyte of user data every day?
So let’s dig into that. What is Discovery Weekly? Discover Weekly is actually a pretty cool thing. Every single week on a Sunday morning, you wake up as a Spotify user and you find out that you have a playlist which has been curated for you with 30 songs. I don’t want recommendations of the most popular music in the world. I want something interesting. I want something that will delight me in the said Sunday morning when I wake up. I wanna be excited. I wanna be surprised.
So collaborative filtering tends to put too much emphasis on what’s popular for a simple reason. If there’s a popular song like “Uptown Funk,” it’s gonna appear on a lot of playlists. And if there is an unusual song, it’s gonna appear on a very few number of playlists so there’s very little data about it. So it’s not gonna end up getting recommended through collaborative filtering.
They have another tool in their arsenal. Every user of Spotify, yourselves included, has a taste profile stored in their databases. What is a taste profile? Well, it looks like this. It is a collection of genres. The big bubble means those are the genres you like most. The smaller bubbles are genres that you like, but like less.
Here’s another one. Here’s another one. Now, these might be a little hard for you to read but some of these genres are probably genres you are very familiar with like indietronica, or pop rap, or future garage, wonky. How many people are into wonky? Chillwave. It doesn’t matter that you’ve never heard of these genres. These are the genres you like, it doesn’t matter if you know what they’re called.
How do they do that? Well, that’s where The Echo Nest comes in. The Echo Nest has software that crawls the internet. It reads “Rolling Stone,” someone must. It reads “Rolling Stone.” It reads “Pitchfork.” It reads other music websites. It reads blogs and using natural language processing, which is software that can read English and understand it, they figure out what songs belong to which genres. And in fact, if you make up a genre today, and “Pitchfork” picks it up and starts writing about the genre you made up, this software will then pick that up from “Pitchfork” and figure out which songs “Pitchfork” had described as belonging to that genre and then include them. And then maybe in your playlists you have included those songs and you will be associated with that genre. Amazing. Okay.
So what do they got? They got songs, they got playlists, they got users, they’ve got taste profiles. They do this amazing thing. I know it’s an ugly picture, right? This is the way computer scientists think. Unfortunately, they’re not user interface designers, right? This is actually a design out of social network analysis. But it does the same thing. This is what’s called “projecting into latent space.” What you do is you take objects that have aspects. You create a multi-dimensional empty space and you project those objects into this space. Things closer to each other are like each other. Things that are far apart from each other are not like each other. You then project the user’s taste profile into that same latent space. Songs that are near that user are songs they are likely to like. They do this amazing thing.
All they need to do then is collect the 30 songs that are near you in their latent space that you haven’t listened to on Spotify. And then they send that to you as a list and you get it and you say, “Oh my god, I love these songs,” or at least some of them. It’s a pretty amazing and complicated thing. There’s more to it than this. There is some deep learning. There’s neural network analysis. They actually even analyze…they started analyzing the audio signals of songs themselves which is obviously very complicated to do. It happens right here in New York City. But that all of these aspects are included in this analysis of what songs belong to which.
So they’re using this amazing set of tools to create this Discover Weekly playlist and as you can imagine… Imagine for a moment the experience of waking up on a Sunday and getting a playlist that you listen to that day and you’re like, “That was awesome.” And the next week it happens again, and the next week it happens again. Tell me that you will leave Spotify for another service. You won’t. People have created methods for taking these Discover Weekly playlists and archiving them, they get wiped clean every week, because they’re so great. This is one of the great tools that Spotify uses to make sure they have a high level of weekly engagement every single week from their users.
What does that all mean to the future? What is the future of music industry? It's worth thinking about. It's a pretty important industry, especially to our culture, but it often is the bellwether for other industries.
The future of the music industry
What does that all mean to the future? What is the future of music industry? It’s worth thinking about. It’s a pretty important industry, especially to our culture, but it often is the bellwether for other industries.
Let’s remind ourselves. Music consumption exploded in 2015 helped by a surge in streaming worldwide. Okay. Let’s take a look at ad-supported streaming. Ad-supported streaming produces less revenue than vinyl. Now think about that a moment. How many times have you gone to Vevo? How many times have you used YouTube? How many of you use the free version of Spotify? The amount of revenue generated out of that is less than the recent resurgence of buying physical records.
It’s understandable that Taylor Swift might be a little bit upset about it. Here’s a quote from her from November of 2014. “I’m not willing to contribute my life’s work to an experiment that I don’t feel fairly compensates the creators of this music. And I just don’t agree with perpetuating this perception that music has no value and should be free.” All right, she’s got, you know, a leg to stand on with that argument.
“The music economy is changing as it moves from an ownership model to an access model.”
However, let’s remind ourselves of Mr. Ek’s comment, “The music economy is changing as it moves from an ownership model to an access model.” So while Taylor Swift is right, music should not be free, in my opinion, maybe you disagree, she’s still bucking the trend. We’re moving to an access model. It is no longer going to be an ownership model. And I think that is true, that is the future.
So there are two ways to compete in streaming. If you accept the premise that streaming is the future, let’s assume you all agree with me, how do you compete? How do you become the number one streaming company? There’s two ways.
One is this way, that gentleman, of course, is our own Jay Z, New Yorker. He purchased TIDAL, T-I-D-A-L, streaming service. How does he compete with TIDAL? It’s through exclusive content, right? Kanye West in February broke the internet yet again. Kanye West released “The Life of Pablo” in February as his TIDAL streaming exclusive. Almost immediately, TIDAL was the number one downloaded app on the Apple Store. TIDAL subscribers went through the roof or at least people using the 30-day free trial. And he had an album that was the number one album and no one knew it because TIDAL refused to release the numbers. He did…TIDAL did finally release the numbers just a few weeks ago and it is… It did then get certified as the number one album, but that’s interesting to me. Their number one album in the country no one, or at least Billboard couldn’t certify that it was the number one album. He broke Billboard, so good work, Kanye.
Rihanna, maybe you listen to “ANTI.” It just came out, TIDAL streaming exclusive. Maybe you listened on Saturday Beyoncé. If you’re not a TIDAL customer, did you get to listen to “Lemonade?” Maybe if you have HBO. That’s pretty amazing that these are TIDAL exclusives. There’s lots of other examples. These are just the big ones. Also, by the way, Saturday number one downloaded app on iTunes, it’s the TIDAL, the TIDAL app. Yet again, right now, number one app, it’s the TIDAL app, thanks to Beyoncé.
So that’s one way to compete. In my opinion, it is not gonna be the successful way to compete because there’s a limit here. This is amazing. This is a trio of huge artists who are providing… Also, by the way, where can you hear Prince’s music? TIDAL. Where can you hear Taylor Swift’s music? TIDAL. But still, we’re talking about a limited number of artists who are willing to make that kind of sacrifice and even “The Life of Pablo” now is available now for download, right? It was only TIDAL exclusive for a period of time and I’m sure that’s true of “Lemonade” as well. So, this is a tough strategy to keep going. Jay Z has got something really great here and he’s got a lot of subscribers, 2 million, but still that ain’t nothing compared to 30 million.
So, what’s the other strategy? It’s the Spotify strategy. It’s a little creepy but it’s impressive. We really want to soundtrack every moment of your life. Imagine in waking up and turning on Spotify and Spotify knows you just woke up. So they play the songs that Spotify knows you might like when you just wake up. Before you go to bed, they play the kind of songs they know you like just before you go to bed. Are they right? Yeah, they probably will be. They have Discover Weekly already nailed. What’s the likelihood that they’ll be right five years from now? Nailing exactly what you wanna listen to before you to bed?
You show up the gym, your phone knows you’re at the gym because of the location. Spotify is playing your workout music. How likely is that to happen? Pretty likely. This Discover Weekly is a pretty powerful tool and it, to me, points the direction of this industry. This is data being used as predictive tool to tell what you like and providing it to you before you have ever even heard of it.
One more quote from Daniel Ek to wrap up the evening. “I don’t think people realize that Spotify is a platform company.” Have you ever thought of Spotify as a platform company? Have you ever tried to listen to a snippet of music on some other site and you’re like, “Oh, you know.” Somebody makes reference to some song and there’s a link and you click it. You can only hear 30 seconds. But you pay for Spotify? Why would that ever be true? If you pay for Spotify, shouldn’t it automatically be routing you to Spotify to hear the whole song, maybe the whole album, maybe other songs like it? That’s the vision here of Daniel Ek is that Spotify becomes the music portal for every application that ever requires music. And why shouldn’t it?
So that is the vision. Again, like I said, might be a little creepy but it seems to me like that’s where we’re headed. Young people are willing to give up a great deal of privacy for convenience, for a frictionless experience and that’s what Spotify promises. So even if it seems to creep you out, you might wanna talk to some of your younger colleagues. I’m surprised always at the number of times a 20-something person says to me, “Yeah, listen, if I can buy that thing faster, then you can have all the personal information you want.” Again, maybe a little creepy but this is the future.
Presented by Tom Cunningham at a Pariveda Perspectives™ event in New York
Learn more about how predictive analytics is shaping the music industry
Case Studies
Explore our success stories
INSIGHTS
Article
