Extra Sums Of Squares
Extra Sums of SquaresIA topic unique to multiple regressionIAn extra sum of squares measures the marginal decrease inthe error sum of squares when one or several predictorvariables are added to the regression model, given that othervariables are already in the model.IEquivalently-one can view an extra sum of squares asmeasuring the marginal increase in the regression sum ofsquares
ExampleIMultiple regression– Output: Body fat percentage– Input:1. triceps skin fold thickness(X1 )2. thigh circumference (X2 )3. midarm circumference (X3 )IAim–Replace cumbersome and expensive immersion in waterprocedure with model.IGoal– Determine which predictor vafriables provide a good model.
The Data
Regression of Y on X1
Regression of Y on X2
Regression of Y on X1 and X2
Regression of Y on X1 , X2 and X3 .
NotationISSR X1 only denoted by SSR(X1 ) 352.27ISSE X1 only denoted by SSE(X1 ) 143.12IAccordingly,IISSR(X1 , X2 ) 385.44SSE(X1 , X2 ) 109.95
More Powerful Model, Smaller SSEIWhen X1 and X2 are in the model, SSE(X1 , X2 ) 109.95 issmaller than when the model contains only X1IThe difference is called an extra sum of squares and will bedenoted bySSR(X2 X1 ) SSE (X1 ) SSE (X1 , X2 ) 33.17IThe extra sum of squares SSR(X2 X1 ) measure the marginaleffect of adding X2 to the regression model when X1 is alreadyin the model
SSR increase and SSE decreaseThe extra sum of squares SSR(X2 X1 ) can equivalently be viewedas the marginal increase in the regression sum of squares.SSR(X2 X1 ) SSR(X1 , X2 ) SSR(X1 ) 385.44 352.27 33.17
Why does this relationship exist?IRemember SSTO SSR SSEISSTO measures only the variability of the Y’s and does notdepend on the regression model fitted.IAny increase in SSR must be accompanied by a correspondingdecrease in the SSE.
Example relationsSSR(X3 X1 , X2 ) SSE (X1 , X2 ) SSE (X1 , X2 , X3 ) 11.54or SSR(X3 X1 , X2 ) SSR(X1 , X2 , X3 ) SSR(X1 , X2 ) 11.54or with multiple variables included at time–SSR(X2 , X3 X1 ) SSE (X1 ) SSE (X1 , X2 , X3 ) 44.71–or SSR(X2 , X3 X1 ) SSR(X1 , X2 , X3 ) SSR(X1 ) 44.71
Extra sums of squaresAn extra sum of squares always involves the difference between theerror sum of squares for the regression model containing the Xvariables in the model already the error sum of squares for theregression model containing both the original X variables and thenew X variables.
DefinitionsIDefinition–SSR(X1 X2 ) SSE (X2 ) SSE (X1 , X2 )IEquivalently–SSR(X1 X2 ) SSR(X1 , X2 ) SSR(X2 )IWe can switch the order of X1 and X2 in these expressionsIWe can easily generalize these definitions for more than twovariables–SSR(X3 X1 , X2 ) SSE (X1 , X2 ) SSE (X1 , X2 , X3 )–SSR(X3 X1 , X2 ) SSR(X1 , X2 , X3 ) SSR(X1 , X2 )
N! different partitionsFigure:
ANOVA TableVarious software packages can provide extra sums of squares forregression analysis. These are usually provided in the order inwhich the input variables are provided to the system, for instanceFigure:
Why? Who cares?Extra sums of squares are of interest because they occur in avariety of tests about regression coefficients where the question ofconcern is whether certain X variables can be dropped from theregression model.
Test whether a single βk 0IDoes Xk provide statistically significant improvement to theregression model fit?IWe can use the general linear test approachIExample–First order model with three predictor variablesYi β0 β1 Xi1 β2 Xi2 β3 Xi3 i–We want to answer the following hypothesis testH0 : β 3 0H1 : β3 6 0
Test whether a single βk 0IFor the full model we have SSE (F ) SSE (X1 , X2 , X3 )IThe reduced model is Yi β0 β1 Xi1 β2 Xi2 iIAnd for this model we have SSE (R) SSE (X1 , X2 )IWhere there are dfr n 3 degrees of freedom associatedwith the reduced model
Test whether a single βk 0The general linear test statistics isF SSE (R) SSE (F ) SSE (F )/ dfFdfR dfFwhich becomesF SSE (X1 ,X2 ) SSE (X1 ,X2 ,X3 ) SSE (X1 ,X2 ,X3 )/n 4(n 3) (n 4)but SSE (X1 , X2 ) SSE (X1 , X2 , X3 ) SSR(X3 X1 , X2 )
Test whether a single βk 0The general linear test statistics isF SSR(X3 X1 ,X2 ) SSE (X1 ,X2 ,X3 )/1n 4 MSR(X3 X1 ,X2 )MSE (X1 ,X2 ,X3 )Extra sum of squares has one associated degree of freedom.
ExampleBody fat: Can X3 (midarm circumference) be dropped from themodel?Figure:F SSR(X3 X1 ,X2 ) SSE (X1 ,X2 ,X3 )/1n 4 1.88
Example Cont.IFor α .01 we require F (.99; 1, 16) 8.53IWe observe F 1.88IWe conclude H0 : β3 0
Test whether βk 0Another exampleH0 : β2 β3 0H1 : not both β2 and β3 are zeroThe general linear test can be used againF SSE (X1 ) SSE (X1 ,X2 ,X3 ) SSE (X1 ,X2 ,X3 )/n 4(n 2) (n 4)But SSE (X1 ) SSE (X1 , X2 , X3 ) SSR(X2 , X3 X1 )so the expression can be simplified.
Tests concerning regression coefficientsSummary:– General linear test can be used to determine whether or not apredictor variable( or sets of variables) should be included in themodel– The ANOVA SSE’s can be used to compute F test statistics– Some more general tests require fitting the model more thanonce unlike the examples given.
Summary of Tests Concerning Regression CoefficientsITest whether all βk 0ITest whether a single βk 0ITest whether some βk 0Test involving relationships among coefficients, for example,IIIIH0 : β1 β2 vs. Ha : β1 6 β2H0 : β1 3, β2 5 vs. Ha : otherwiseKey point in all tests: form the full model and the reducedmodel
More Powerful Model, Smaller SSE I When X 1 and X 2 are in the model, SSE(X 1;X 2) 109.95 is smaller than when the model contains only X 1 I The di erence is called an extra sum of squares and will be denoted by SSR(X 2jX 1) SSE(X 1) SSE(X 1;X 2) 33:17 I The extra sum of squares SSR(X 2jX 1) measure the marginal
a sum of two squares. More generally, this kind of argument shows that if pis not a sum of two squares, then n pemis not a sum of two squares if eis odd and gcd(p;m) 1. Thus solving the two squares problem for n pwill yield the answer for general n2N, and here is the answer. Theorem 1.1 (Fermat (1640)).
Topic 2 Develop Fluency: Addition and Subtraction Facts within 10 Textbook section IXL skills Lesson 2-1: Count On to Add 1.Addition sentences using number lines - sums up to 10 UWW 2.Addition facts without zero - sums up to 10 RUF Lesson 2-2: Doubles 1.Add doubles with models - sums up to 10 Z8T 2.Add doubles - sums up to 10 ZNG Lesson 2-3: Near Doubles
5.1 Area and Estimating with Finite Sums. Ideas for this section The ideas for this section are Left-Hand Sums Right-Hand Sums Midpoint Rule Average value Velocity problems Distance v. Displacement . Given the function f(x) x2, approximate the area under the curve on the interval [0,4]for n 4 using the Left-Hand Rule and the Right-Hand Rule.
Suppose total sums of squares is 100, and X 1 explains 60. Of the remaining 40, X 2 then explain
Linear Least Squares ! Linear least squares attempts to find a least squares solution for an overdetermined linear system (i.e. a linear system described by an m x n matrix A with more equations than parameters). ! Least squares minimizes the squared Eucliden norm of the residual ! For data fitting on m data points using a linear
5.1 Estimating with Finite Sums Calculus 5 - 2 The Area Problem and the Rectangular Approximation Method (RAM) (a.k.a. Riemann Sums) Suppose we wanted to know the area of the region bounded by a curve, the x – axis, and the lines x a and x b, as shown at the right. The first step is to divide the interval from a to b into subintervals. (The examples below show 4 and 8 subintervals .
Character sums, automorphic forms, equidistribution, and Ramanujan graphs 1 Ching-Li Chai2 and Wen-Ching Winnie Li3 Version 3.1, January 4, 2001 Abstract According to the Langlands program, L-functions arising from families of exponential sums
READING COMPREHENSION PRACTICE EXAM. GENERAL INSTRUCTIONS: You will have 90 minutes for this test. Work rapidly but carefully. Do no spend too much time on any one question. If you have time after you have finished the test, go back to the questions you have left unanswered. The three parts of this test are English Usage, Sentence Correction, and Reading Comprehension. When you have finished .
