Marketing Mix Modelling From Multiple Regression Perspective - KTH

2 Theoretical BackgroundThis section presents the mathematical background of the common challenges that marketing mixmodellers are facing. It begins with the challenge of choosing the appropriate functional form,continues with the dynamic structure of marketing variables, and nally an approach to accountfor the e ects of seasonality is described.2.1 Methods of selecting functional forms of the modelAn important part of the model building process is deciding upon the functional form that wouldre ect the most appropriate relationship between the variables. The most commonly applied functional form in Marketing Mix Modelling is the linear model. However, it is often the case that thenonlinear functional forms are used, since they take into account such properties as diminishing/increasing returns to scale and threshold e ects. In this section the vector of the model parameters βas well as the vector of the disturbance term ε are named in the same way for di erent speci cations,even though the parameters di er, depending on the functional form.2.1.1 Linear and Multiplicative ModelsLinear models assume constant returns to scale and have the following structure:yt β0 β1 x1t β2 x2t · · · βK xKt εt ,(2.1)where, following the notations in [16]:yt value of the dependent variable in period t (t 1, ., T , where T is the number of observations),xkt value of independent variable k in period t, (k 1, ., K , where K is the number of covariates),andβ0 , β1 , ., βK model parameters.εt the (unobserved) value of the disturbance term.A linear model is often tried rst since the estimation of the coe cients and the interpretationof the results are easy. It shows a good predictive performance and a reasonable approximation toan underlying nonlinear function, but only on a limited range.One drawback of the linearity assumption is that it implies constant returns to scale with respectto each of the covariates, meaning that an increase of one unit in xkt leads to an increase of βkunits in yt . However, the assumption of constant returns to scale is unrealistic in most real lifemarketing applications. Usually a sales response curve exhibits a non-constant behavior. One typeof non-constant behavior is diminishing returns to scale, which happens when the response variablealways increase with increases in the covariates, but each additional unit of xkt brings less in ytthan the previous unit did ([15]). One of the functional forms that re ects this phenomenon is themultiplicative power model (again, following the notations in [16]):yt β0 xβ1t1 εt ,x1t 0,0 β1 1(2.2)Model 2.2 can be linearized by taking the logarithms of both sides:ln yt ln β0 β1 ln x1t ln εt ,3x1t 0,0 β1 1(2.3)

Equation 2.3 is linear in the parameters β0? , β1 , where β0? ln β0 . This model is known as thedouble-logarithmic or the log-log model. The version of the multiplicative model that retains thehighest-order interaction among the variables for K marketing instruments is:(2.4)Kyt β0 xβ1t1 xβ2t2 · · · xβKtεtor more compactly:yt β0 (KY(2.5)xβktk )εtk 1In this setting, if some of the variables are "dummies", the corresponding variables are used asexponents. Besides re ecting the non-constant behavior of the sales response function, anotheradvantage of the multiplicative model over the linear model is that it allows for a speci c form ofinteraction between the various instruments. Taking the rst-order partial derivative of yt withrespect to any of the independent variables xkt , the impact of a change in xkt on yt is a function ofyt itself, which means that it depends not only on the value of xkt but on all the other variables aswell: ytβK β0 βk xβ1t1 xβ2t2 · · · xktk 1 · · · xβKt(2.6) xktWhen sales response function exhibits increasing returns to scale, the exponential model can beused:yt β0 eβ1 x1t εt(2.7)After taking the logarithms of both sides it becomes the semi-logarithmic also known as the loglinear model:ln yt ln β0 β1 x1t ln εt(2.8)When the nonlinear model is log-log or log-linear, an adjustment to the forecasts of yt is required,so that they remain unbiased ([15]). Considering the typical multiplicative speci cation 2.4, whereln εt is N (0, σ 2 ), it can be shown that:K 1/2σE[yt ] β0 xβ1t1 xβ2t2 · · · xβKte2(2.9)The forecasts should be calculated from the expression:K 1/2σ̂eŷt β̂0 xβ̂1t1 xβ̂2t2 · · · xβ̂Kt2(2.10)where hats denote the ordinary least squares (OLS) estimates. A direct re-transformation wouldunder-estimate the forecasts.2.1.2 The Box-Cox transformationOne way to compare between the linear and the multiplicative speci cations is the likelihood ratiotest, using the Box-Cox transformation. It is based on the following transformation of the dependentvariable:ytλ 1 β0 β1 x1t . . . βK xKt εt .(2.11)λTo choose the appropriate functional form, the likelihood ratio test of the model above can be used.The idea behind this method is to compute the likelihood ratio for di erent values of λ and choose4

the value that maximizes the MLE score. The speci cation is then chosen according to the valueof λ reported. If λ 1 then the speci cation is essentially linear. When λ approaches 0, equation2.11 approaches the semi-logarithmic form, since:limλ 0 ytλ 1λ ln yt(2.12)2.2 Marketing DynamicsBecause of the evolving character of markets, the assumption that advertising expenditures have acurrent and immediate impact on sales rarely happens to be realistic. Most often is happens thatparts of the media e ects remain noticeable for several future periods. Thus, sales in some period tare a ected by advertising expenditures in the same period t, but also by expenditures in previousperiods t 1, t 2, . . . . The in uence of current marketing expenditures on sales in future periodsis called the carryover e ect. When the e ect of a marketing variable is distributed over severaltime periods, sales in any period are a function of the current and previous marketing expenditures.In the case of just one explanatory variable the equation for sales is:y t β0 Xβl 1 xt l εt ,(2.13)l 0where xt l , l 0, 1, . . ., are the lagged terms of the independent variable. The model 2.13 is calledthe In nite Distributed Lag (IDL) Model. Assuming that all coe cients of the lagged terms of acovariate have the same sign, equation 2.23 can be rewritten as:y t β0 β Xωl xt l εt .(2.14)l 0Equation 2.14 is called the Geometric Lag Model, whereωl 0 and Xωl 1.(2.15)l 0The omegas can be regarded as probabilities of a discrete-time distribution. As mentioned in [15],the Geometric Distributed Lag (GL) Model is the most commonly used distributed-lag model inmarketing. The maximum impact of marketing expenditures on sales is registered instantaneously,then the in uence declines geometrically to zero. The impact of any past expenditure in subsequentperiods will be a constant fraction of its immediate impact. This constant fraction is called theretention rate. If the retention rate is λ the geometric distribution gives:ωl (1 λ)λll 0, 1, 2 · · ·(2.16)where 0 λ 1. The speci cation of the sales response function becomes:yt β0 β(1 λ) Xλl xt l εt ,(2.17)l 0oryt β0 β1 xt β1 λxt 1 β1 λ2 xt 2 . . . β1 λl xt l . . . εt ,5(2.18)

where β1 β(1 λ). The direct short-term e ect of marketing e ort is β1 β(1 λ), while theretention rate λ measures how much of the advertising e ect in one period is retained in the next.The implied long-term e ect is β β1 /(1 λ). This model is also approximately equivalent to theSimple Decay-E ect Model (Broadbent (1979)):yt β0 β1 at εt ,(2.19)where at f (xt ) is the adstock function at time t, xt is the value of the advertising variable attime t and λ is the decay or lag weight parameter:at f (xt ) xt λat 1 ,t 2, . . . , n(2.20)Recursively substituting and expanding the equation for the adstock function becomes:at xt λxt 1 λ2 xt 2 . . . λn xt n ,(2.21)Since 0 λ 1, λ 0 as n . Moving on to the case with K explanatory variables x1 , . . . , xK ,each with di erent retention rates λ1 , . . . , λK , the model becomes:yt β0 β1 a1t β2 a2t · · · βK aKt εt(2.22)ait f (xit ) xit λi ait 1 ,(2.23)wherei 1, . . . , KTo estimate the marketing variables coe cients, as well as retention rate values, non-linear leastsquares can be used. The algorithm is described more detailed in section 3.2. First the adstock attime t is de ned for each marketing instrument, as in equation 2.23. The estimated sales are then:ŷt β̂0 β̂1 a1t β̂2 a2t . . . β̂K aKt(2.24)Finally, the optimization problem is:minimizeTX(yt ŷt )2t 1subject to 0 λi 1, i 1, . . . , K.For the semi-logarithmic and double-logarithmic models the equation for the predicted sales becomes2.25 and 2.26 respectively:andln ŷt β̂0? β̂1 a1t β̂2 a2t . . . β̂K aKt(2.25)ln ŷt β̂0? β̂1 ln a1t β̂2 ln a2t . . . β̂K ln aKt(2.26)And the optimization problem is:minimizeTX(ln yt ln ŷt )2t 1subject to 0 λi 1, i 1, . . . , K.6

2.3 Modelling trend and seasonalityIn this section the "classical decomposition" is considered:yt mt δit εt ,(2.27)where: mt is a slowly changing function (the "trend component");δit is a function with known period d (the "seasonal component");εt is a stationary time series.In trying to explain sales behavior, a linear trend variable (mt 1, 2, · · · , T for t 1, 2, · · · , T )could be introduced into the sales response function to capture the time-dependent nature of salesgrowth.If a variable follows a systematic pattern within the year, it is said to exhibit seasonality. Todeal with seasonality, s dummy variables could be introduced in the model to express s seasons inthe following way:(1,δit 0,if t is the i'th periodotherwisei 1, · · · , s t 1, · · · , T(2.28)These "dummy" variables for seasons and "time" variable mt for trend could be incorporated intothe linear model 2.1:yt β0 mt δ1t . . . δst β1 x1t β2 x2t . . . βK xKt εt ,(2.29)and also into the multiplicative models, for example into the log-log model 2.4:Kyt β0 e(mt δ1t . δst ) xβ1t1 xβ2t2 . . . xβKtεt(2.30)Equation 2.30 is non-linear. For the purposes of estimation, the model is converted into an additiveform by taking natural logarithms thus:ln yt ln β0 mt δ1t . . . δst β1 ln x1t . . . βK ln xKt ln εt(2.31)Taking into account the dynamic structure, equation 2.31 becomes:ln yt ln β0 mt δ1t . . . δst β1 ln a1t . . . βK ln aKt ln εt(2.32)where a1t , . . . , aKt are the adstock variables de ned in section 2.2. Equation 2.32 is no longerlinear, and was estimated with non-linear least squares, using the Levenberg-Marquardt algorithm,described in section 3.2.7

3 EstimationOnce the appropriate functional form is decided, the parameters of the marketing model must beestimated. A description of the estimation methods for the model parameters is provided in thisChapter.3.1 Ordinary Least SquaresLet us consider the linear model 2.1:yt β0 β1 x1t β2 x2t · · · βK xKt εt ,t 1, . . . , T(3.1)where the notations are de ned in section 2.1.1. Equation 2.1 can be rewritten in the matrix form: 1y1 y2 1 . . . .1yT x11x12.x1T······x21x22.···x2T βε1xK1 0 β1 ε 2 xK2 β2 . . . . . . εTxKTβK(3.2)or:(3.3)y Xβ ε.The OLS estimates of the parameters β̂ β̂0 β̂1 · · · β̂Kthe Residual Sum of Squares (RSS):RSS TX(yt ŷt )2 TX(yt β̂0 t 1t 1 in 3.1 are the values which minimizeKX(3.4)β̂k xkt )2k 1Following the notations in [8], the total sum of squares is de ned as: SStot Tt 1 (yt ȳt )2 . WithRSS and SStot de ned above, the following relationship holds: SStot SSreg RSS , where SSregPis the regression sum of squares : SSreg Tt 1 (ŷt ȳt )2 . It is easy to show that the coe cientestimates β̂ obtained by minimizing the quantity above are:Pβ̂ (X X) 1 X y.(3.5)Assuming that Cov(ε) σ 2 I, the covariance matrix of β̂ is then Cov(β̂) σ 2 (X X) 1 , estimatedas:d β̂) Cov(RSS(X X) 1T K 1An Fα (1, T K 1)-statistic for the hypothesis βk 0 is calculated as:F β̂kSE(β̂k )!2.where the standard error SE(β̂k ) for any k 1, . . . , K is the square root of the correspondingd β̂).diagonal element of Cov(8

3.2 Non-linear Least SquaresTo model the dynamic structure with several explanatory variables, the Levenberg-Marquardt algorithm (LMA) was used. As described in [7], the LMA interpolates between the Gauss-Newtonalgorithm (GNA) and the method of gradient descent. In the current setting, following the notations de ned in the previous sections, the problem is de ned in the following way: given a numberT of observations of independent and dependent variables, (xt , yt ), where xt is a vector of lengthK , containing the K variable measurements corresponding to the observation of the dependentvariable yt , the objective is to optimize K parameters β β0 β1 . . . βK of the model curvef (X, β) such that the sum of the squares of the deviationsS(β) TX[yt f (xt , β)]2(3.6)t 1is minimized.3.2.1 The Gradient Descent MethodThe idea behind the steepest descent method is that it updates parameter estimates in the directionopposite to the gradient of the objective function. The gradient of S with respect to β is f (X, β) S(β) 2(y f (X, β)) (y f (X, β)) 2(y f (X, β)) 2(y f (X, β)) J β β β(3.7)where the Jacobian matrixJ f (X, β) βrepresents the the change of f (X, β) to variation in the parameters β. In each iteration step, theparameter increment δ that moves the parameters β in the direction of steepest descent is given byδgd αJ (y f (X, β))(3.8)The positive scalar α determines the length of the step in the steepest-descent direction.3.2.2 The Gauss-Newton MethodThe Gauss-Newton method assumes that the objective function is approximately quadratic in theparameters near the optimal solution ([7]). The parameter increment δ is found by approximatingthe functions f (xt , β δ) by their linearizationsf (xt , β δ) f (xt , β) Jt δ(3.9)where f (xt , β) βThe above rst-order approximation of f (xt , β δ) givesJt S(β δ) (y f (X, β)) (y f (X, β)) 2(y f (X, β)) Jδ δ J Jδ(3.10)Taking the derivative of S(β δ) with respect to δ and setting the result to zero gives:(J J)δgn J [y f (X, β)]9(3.11)

3.2.3 The Levenberg-Marquardt MethodThe Levenberg-Marquardt algorithm interpolates between the Gauss-Newton method and themethod of gradient descent.(J J λI)δlm J [y f (X, β)](3.12)Small values of the damping parameter λ result in a Gauss-Newton update and large values of λresult in a gradient descent update. In each step the parameter λ is iteratively adjusted, that is λ isincreased S(β δ) S(β), and is decreased otherwise. To avoid slow convergence in the directionof small gradient, Marquardt provided the insight that the values of λ should be scaled to the valuesof J J ([7]):[J J λ diag((J J)]δlm J [y f (X, β)].(3.13)10

4 Validation and TestingThe process of validation and testing of the model begins with testing model's statistical assumptions. This part is called speci cation error analysis (section 4.2). The next step is to test theregression results. This involves tests of signi cance described in section 4.1.4.1 Methods of Model AssessmentIn this section it is assumed that there are no speci cation errors. Linear regression assumes thatthe disturbances are normally distributed: ε N (0, σ 2 I), thus β̂ N (β, σ 2 (X X) 1 ). A teststatistic for the hypothesis that all of the β 's are all equal to zero:H0 : β1 β2 . . . βK 0 vs H1 : at least one βi 6 0is:F SSreg /KRSS/(T K 1)which has an approximate F (K, T K 1) distribution under the null. To determine the amount ofvariation "explained" by the covariates, one looks at a descriptive statistic R2 , called the coe cientof determination or goodness of t.R2 SSregRSS 1 SStotSStot(4.1)There is also an adjusted R2 that considers an adjustment for degrees of freedom:R̄2 1 T 1 RSST K 1 SStot(4.2)To determine which covariates are contributing to the t, one has to examine each covariate separately. The test statistic for the null hypothesis that a coe cient is zero must be calculated asexplained in section 3.1. A common test to determine which covariate should enter the regressionis the Akaike Information Criterion test:AIC T ln(RSS) 2K.(4.3)The model with the lowest AIC is prefered, since it minimizes the information loss ([13]).4.2 Speci cation Error AnalysisTo obtain point estimates of the coe cients and perform statistical inferences based on those pointestimates (for example: tests of signi cance, con dence intervals) the following assumptions mustbe satis ed: E[εt ] 0 for all t; Var[εt ] σ 2 for all t; Cov[εt , εt0 ] 0 for t 6 t0 ;11

εt is normally distributed. The matrix X has full rank, thus X X is non-singular.Table 1 based on [16] is a part of model building strategy from the perspective of violation of assumptions. It presents a short summary of reasons, remedies, and ways to detect possible violationsof each assumption. The Table is adapted to the given problem, and the methods applied in thisthesis.4.2.1 Nonzero expectation of the residualsThe violation of the assumption that the residuals are normally distributed could be a sign ofincorrect functional form, or omitted variables. If the assumed functional form is incorrect, a plotof the residuals et yt ŷt , t 1, . . . , T against each predictor should show a systematic patt

Marketing Mix Modelling is a term that is used to cover statistical methods which are suitable for explanatory and predictive statistical modelling of some ariablev of interest, for example company's sales or market shares. This thesis is focused on modelling sales as a factor of marketing instruments