Prediction Of Product Success: Explaining Song Popularity .
Prediction of product success: explaining songpopularity by audio features from Spotify dataAuthor: Rutger NijkampUniversity of TwenteP.O. Box 217, 7500AE Enschedethe NetherlandsABSTRACTThis research investigates the relationship between song data - audio features from the Spotify database (e.g. keyand tempo) - and song popularity measured by the number of streams a song has on Spotify. Previous research onthe topic of new product success prediction have identified multiple approaches to asking this question. Especiallyfor products in cultural markets like music, prediction modelling is very complex. A relatively novel approach, theattribute-approach was used to explore whether song attributes have an explanatory power on stream count. Researchin this specific field called Hit Song Science (HSS) has not before measured similar audio features with songpopularity in stream count, which is very important for record companies and which makes this research unique.Furthermore, beneficial implications from HSS can be far-reaching to consumers, record companies and for Spotifyin new value creation.From the Spotify database API, a 1000 songs were analyzed from 10 different genres. By regression, a predictionmodel was built. We can conclude that our results suggest that audio features from Spotify have little to moderateexplanatory power for a higher stream count, with this research design. Some significant relationships however werefound, which lays a promising foundation for the research in prediction with these variables. This researchcontributes to further understanding in the field of HSS and the new product success prediction. Creating effectiveprediction models is an interesting next step to this research and so would be to expand on the variables used.Practical implications include that Spotify can further develop its database and calculations of the variables for inthe future their databases will play an important role in new value creation.Graduation Committee members:Dr M. De VisserDr M.L. EhrenhardKeywordsProduct Success Prediction Music Attributes Popularity Spotify HSSPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.11th IBA Bachelor Thesis Conference, July 10th, 2018, Enschede, The Netherlands.Copyright 2018, University of Twente, The Faculty of Behavioural, Management and Social sciences.
1. INTRODUCTIONBusiness would be easy if we could predict product sales beforethey are released on the market. As the cost of failure in newproduct development is very high, researchers and productdevelopers are looking for good product success/failureprediction models. Research that aims to answer these questionshas established many approaches to creating success predictionmodels. Determinants for success are specified fromorganizational and industry factors, social data and predictionsfrom tests markets to give examples. Predicting the popularity ofelectronic household products however seems a lot morestraight-forward than predicting cultural products such as music.Their success and popularity seems related to taste and moresubjective measurements which makes prediction all the morecomplex.“Historically, neither the creators nor the distributors of culturalproducts have used analytics - data, statistics, predictivemodeling - to determine the likely success of their offerings.Instead, companies relied on the brilliance of tastemakers topredict and shape what people would buy” (Davenport et al.,2009). While tastemakers are still significant influencers ofproducts also in cultural product markets, the way how weconsume is changing by technological developments. Also whatwe consume is changed by the shifting importance of science inart. Unprecedented access and technological advancements makeprediction and recommendation of customer taste easier andmore important. A specific cultural market where prediction andrecommendation capabilities for producers, distributors andconsumers are extremely important is the music industry.Music is one of the most popular types of online information(Casey et al. 2008) and the importance of the music industry canbe expressed in its total revenues. For 2017 they were US 17.3billion, but it is an industry with a future too, as it is a growingindustry; revenues increased by 8.1% in 2017 (IFPI, 2017).Similar to other cultural products such as movies, costs in newproduct development are high. Record companies are estimatedto annually invest 4.5 billion worldwide in artists and repertoire(A&R) combined with marketing.Technological advancements have seen the rise of streamingservices, which revenues grew 41.1% in 2017, making digitalrevenues now account for more than half (54%) of the globalrecorded music market (IFPI, 2017). Streaming services such asSpotify are, even though the controversy on its profitability,having a positive effect on the growing music industry’s revenue(Wlömert et al., 2016) – illustrating its growing importance in thefuture of the music industry.Music streaming services thank its growth to that they are able toreact to new expectations of listeners, who want searchablemusic collections, automatic playlist suggestions, musicrecognition systems and more. (Casey et al. 2008). They can doso because of the (user generated) big data and their digital songdatabase. Because this is an important value proposition forthem, there are much investments made to improve it. In 2014,Spotify acquired ‘music intelligence’ company ‘The Echo Nest’for 49.7m to further develop its service offerings likerecommendation systems 1 . The music database that formedoffers easily manageable data and contains all sorts of data onsongs such as audio features (Tempo, Key) and track information(Artist, Genre).Next to the importance of recommendation, there is theimportance of prediction of music popularity. In the musicindustry too, all parties have an interest in connecting consumerswith content they will like and buy and it remains one of thebiggest mysteries in the industry why some songs becomepopular while other songs fail to do so. Researchers have startedto ask the same question and many approaches to product successprediction have been taken since to predict song popularity, aswill be reviewed in the literature section later.A relatively new approach to success prediction focuses on theattributes of a product and the Spotify database, with free onlineavailable audio features, allows for this approach to be taken.With the increasing availability of digital music, the evolution oftechnology and the ability to retrieve information from music, anew field of research has emerged: Music Information Retrieval(MIR). MIR is a multidisciplinary domain concerned withretrieving and analysing multifaceted information from largemusic databases (Downie, 2003). Success prediction in thisnovel research field has been coined Hit Song Science (HSS),which is, as defined by Pachet (2012), “an emerging field ofinvestigation that aims at predicting the success of songs beforethey are released on the market”.This research and other research in the field have a practicaldimension. The insights gained in this field can provide hugebenefits for the industry and all parties involved in the musiccontent life-cycle. Beneficial examples include that artists canwork reversely the process of HSS and focus on characteristicsthat make their songs more popular and that record companies,aiming at maximum profit, could benefit by selecting the mostpromising works for publication and marketing goals (Karydis etal., 2018). Moreover, music streaming services are struggling todiversify their revenue channels and innovate on their valueproposition, as can be seen by Spotify’s recent IPO. However, itis their rich databases that open up possibilities for new productsuccess prediction models, which can create new valuepropositions. Examples would include they can sell predictionmodels to record companies and artists, but also to use them toimprove their own services to music consumers. It illustrates theimportance of product success prediction and the emergingresearch field of MIR and HSS in the field of business.2. LITERATURE BACKGROUNDThis section will further explain the relevance of this research inexplaining and predicting song popularity by providing aninsight in the existing body of research in product successproduction and specifically for music as a product in a culturalmarket. A gap in the research will be identified and the researchquestion of this study will be defined.2.1 Literature BackgroundResearch in product success prediction models with the use oforganizational- (Lo et al., 2000), industry- (De Vasconcellos etal., 1989) and entrepreneurial factors (Kleinknecht et al., 2012)are exemplary approaches to the subject. In the researchmentioned, success determinants would be identified fromexperiences of developers, expert panels, survey data and thelike. The prediction of success/failure of a new vacuum cleanerhowever is very different from that of a new Kendrick fy-acquiredecho-nest-just-e50m/2
album for example, in that the content (music) does not innovateas radically on the characteristics and attributes as a new productcould. Products in cultural markets, like music, ask for differentdeterminants. There are a variety of relevant approaches thatalready exist in new product success prediction that are relevantfor music too. Moreover, the existing body of research whichdefines many popularity prediction models stresses thecomplexity of the mechanisms of song popularity.Let’s take a look at the variety of relevant approaches that alreadyexist. A determinant for success that can be identified by simpletechnology is the correlation with other items or customers. It isused in cultural markets - Spotify, as well as Netflix and Amazonare known to make use of this technology to recommend contentto consumers. Naturally this method is rather limited, due to theintentions of the costumer and the nature of the shifting culturalproduct market over time. A weakness of this approach to workfor success prediction is also that a substantial amount ofcustomer data is needed for it to work effectively (Davenport etal., 2009).addressed a similar question regarding the prediction ofpopularity by automated labelling of low, medium or highpopularity. They published their research ‘Hit song science is notyet a science’, in the, at that time, still novel field of MIR, aimingat “validating the hypothesis that the popularity of music titlescan be predicted from global acoustic or human features” (Pachet& Roy, 2008). Their research found, as you might have guessed,that their learning machines weren’t able to label popularity aslow medium or high from audio feature sets better than random.10 years later a broad body of research has taken the controversyon its feasibility away. Research from Lee et al. (2015) showsthat it is feasible to predict the popularity metrics of a songsignificantly better than random chance based on its audio signal.Additionally, Ni et al. (2015) also showed that certain audiofeatures such as loudness, duration and harmonic simplicitycorrelate with the evolution of musical trends. Singhi and Brown(2015) propose features from both songs’ lyrics and audiocontent for prediction of hits and also have done research on a hitdetection model based solely on lyrics’ features (2014).Another approach that is implemented by companies and used inresearch for recommendation and prediction is the use of socialnetworks and social data. In HSS, approaches that look for socialpopularity metrics include research on using social media data,for example from Twitter. Zangerla et al. (2016) found that usingTwitter posts is useful to predict future charts, when recent musiccharts are available. Similarly, the research by Kim et al. (2014)shows a high correlation between users’ music listeningbehaviour data from Twitter and music popularity on the charts.Furthermore, Bischoff et al. (2009) propose a music popularityprediction model by social interaction data from Last.fm,showing promising results. A weakness of this approach too isthat it requires a large amount of data.The growing data on listener behaviour, song audio features andmeta-information provided by digital music and specificallystreaming services is of great importance to MIR. Recentresearch is able to use database from Spotify to access realmusical content easily and legally. An exemplary HSS researchthat uses this database is that of Herremans et al. (2014). Theresearch focusses on the dance hit song classification problem.From a database of dance hit songs, including basic musicalfeatures as well as more advanced temporal features (timbre),classifiers were built to create dance hit prediction models. Herresults suggest the possibility to predict whether a song is a ‘top10’ dance hit versus a lower listed position – thus proving thecapabilities of prediction models from audio features.Some popular approaches in product success prediction like theuse of prediction markets (Matzler et al., 2013) are not as relevantfor new content as it is for products. The use of predictionmarkets is beneficial to take away distribution costs that do notexist for musical content (to the same extend).In the field of HSS, popularity prediction is often done in theform of hit prediction and the prediction of chart rankings. Aschart rankings are not directly related to actual popularity, othermeasurements for popularity should be looked for.Above all other approaches, the approach that has seen mostpopularity is found in the majority of the research in the field ofHit Song Science (HSS). It is the approach used in MusicInformation Retrieval (MIR) which focuses on song data - theattributes (audio features) of a song. Technologicaldevelopments and user generated big data by streaming serviceshave made this approach possible. “The underlying assumptionbehind HSS is that popular songs are similar with respect to a setof features that make them appealing to a majority of people.These features could then be exploited by learning machines inorder to predict whether a song will rise to a high position in thechart” (Ni et al., 2015). It is important to note that all approachesdescribed above have a weakness. The attributes approach (ofMIR) requires that attributes are classified and that a lot of datais needed. An enormous amount of song data including manyattributes however is already available on Spotify’s databasewhich shall be used in this research.One of the earliest relevant work in the MIR and HSS field hasbeen done by Dhanaraj and Logan (2005). In their study theyextracted both acoustic and lyric in- formation from songs toseparate hits from non-hits using standard classifiers, specificallySupport Vector Machines and boosting classifiers. Their researchshowed promising results and found that for the features used,lyric-based features are slightly more effective than audio-basedfeatures at distinguishing hits. What is maybe surprisingly, is thatthey found the absence rather than the presence of certainsemantic information in the lyrics mean a song is more likely tobe a hit. Other work followed in 2008, as Pachet and RoyIn an attempt to predict the popularity of a song from Spotify’ssong data, the research of Will Berger (2017) uses (Echo-Nest)audio features similar to this research and uses Spotify’s owncalculated metric “popularity” to measure popularity. This is agiven audio feature on Spotify’s database that is computedsecretly to describe the popularity of a song, while the number ofstreams is not given. The ‘problem’ with this variable howeveris, is that a song can score very high on popularity with only50000 streams and vice versa – Spotify’s popularity metric itdoes not relate to the actual number of streams. It is in thisidentified gap of research, and specifically with a business angle,that this research is unique.2.2 Research QuestionThis paper will take the attributes approach that is data driven totest determinants for song popularity. It will address theidentified gap in the existing product success prediction field ofHSS by analysing stream count on Spotify instead of Spotify’spopularity metric, defining popularity as hits or non-hits or bychart position. Prediction of songs in a particular genre is likelyto be ‘easier’, since each genre has its own popularcharacteristics. This research wants to see whether there aregeneral attributes for song stream count, therefore it will usesongs from the 10 most popular genres as identified by Spotify.To the author’s knowledge, surprisingly no research hasmeasured popularity in the amount of streams a song has – whichis of the greatest importance for record companies and artists.Here is why: Spotify pays loyalties to record companies as afixed amount per stream on Spotify which have seen streaming3
revenues grown to account for 38.4% of total recorded musicrevenue as of 2018 (IFPI, 2018). Moreover, as streamingcompanies and especially Spotify are likely to become one of thebiggest players in the music industry, making research into whatthey can do with their data also seems more relevant than ever.Important to note is the distinction between in-sampleexplanatory power and out-of sample predictive power (Shmueli,2010). This study aims to explore the in-sample explanatorypower of the data already available to Spotify and developers. Itwill question whether the attributes approach is also found to beeffective in explaining stream count on Spotify.Fromcorrelation and regression analysis, we can find if the Spotify’saudio features can explain the stream count. It will be a startingpoint in the research for prediction of song popularity measuredin stream count (out-of sample predictive power). Therefore, theresearch question is:into our relationships and whether our hypothesis hold sometruth.3.1.2.1 AcousticnessAcousticness is an attribute that is calculated by Spotify and is aconfidence measure from 0.0 to 1.0 representing if the track isacoustic. 1.0 represents high confidence the track is acoustic.Looking at the high number of non-acoustic popular songs, andthe array of electronic instruments used in the charts, a negativerelationship between Acousticness and stream count is mostlikely. Hence the following is hypothesized:H1: Acousticness is negatively related to a higher streamcount.3.1.2.2 Danceability“Is the attribute approach based on Spotify’s audio featureseffective in explaining streaming popularity on Spotify?’’Danceability is calculated by Spotify and describes how suitablea track is for dancing based on a combination of musical elementsincluding tempo, rhythm stability, beat strength, and overallregularity. Regularity and whether a track is danceable is likelya characteristic of popularity.3. VARIABLES AND HYPOTHESISH2: Danceability is positively related to a higher streamcount.3.1 Defining the Variables3.1.1 The Dependent VariableSince we are not discriminating between hit and non-hit song buton popularity, the dataset will have to include songs with a bigvariation in stream count. The popularity of a song can bemeasured a posteriori according to statistics such as the numberof times a track has been played – in this context: streamed. Wedefine popularity as the number of streams on Spotify. Thevariable number of streams is easily measurable in the Spotifyapplication, but it has to be done by hand. The number of streamsis a continuous ratio variable put in the dataset as ‘Streams’.3.1.2 The Independent VariablesSince this study is interested i
music databases (Downie, 2003). Success prediction in this novel research field has been coined Hit Song Science (HSS), which is, as defined by Pachet (2012), “an emerging field of investigation that aims at predicting the success of songs before they are released on the market”.
on Explaining the Benefits of Competition Project, 2012). With Explaining the Benefits of Competition to Businesses, the ICN AWG builds upon these findings and Explaining the Benefits of Competition to Government and Legislators, to provide authorities with elements that may be useful in their advocacy activities directed at the private sector.
generic performance capability. The comparative analysis imparts the proposed prediction model results improved GHI prediction than the existing models. The proposed model has enriched GHI prediction with better generalization. Keywords: Ensemble, Improved backpropagation neural network, Global horizontal irradiance, and prediction.
Picturing generosity: Explaining the success of national campaigns in the Netherlands Abstract (148 words) In this study we investigate the success of national campaigns for charitable causes in the Netherlands using historical data from archival sources, including newspapers, and
Prediction models that include all personal, social, psychological and other environmental variables are necessitated for the effective prediction of the performance of the students [15]. The prediction of student performan
A post-prediction process is also proposed to further enhance the prediction results. The framework conducts the prediction in real time. To the best of our knowledge, this is the first study that addresses the potential application of a sequenti
prediction; that is, providing a forecast (or nowcast) of a variable of interest from available data. In some cases, prediction has enabled full automation of tasks – for example, self-driving vehicles where the process of data collection, prediction of behavior and surroundings, a
Crystal structure prediction via particle-swarm optimization, Phys. Rev. B 82, 094116 (2010). Cluster Structure Prediction: Jian Lv, Yanchao Wang, Li Zhu, and Yanming Ma* Particle-Swarm Structure Prediction on Clusters, J. Chem. Phys. 137, 084104 (2012). Two-Dimensional Layer Structure Prediction: 1.
the youth volunteers who participate in the annual Diocesan Pilgrimage to Lourdes. I have been very impressed with their energy, good humour and spirit of service towards those on the pilgrimage who have serious health issues. The young, gathered from around the diocese show a great sense of caring and goodwill to those entrusted to their care over the days of the pilgrimage. Young people have .
