3 Curve Fitting: Least Squares Methods - NotesInterpreter

y1427405568Home work problem.3.4 CorrelationIn a bivariate distribution, if the change in one variable affects a change in the othervariable, the variables are said to be correlated.If the two variables deviate in the same direction i.e., if the increase (or decrease) in oneresults in a corresponding increase (or decrease) in the other, correlation is said to be direct orpositive.Fig.1. Positive CorrelationFig.2. Negative Correlatione.g., the correlation between income and expenditure is positive.If the two variables deviate in opposite direction i.e., if the increase (or decrease) in oneresults in a corresponding decrease (or increase) in the other, correlation is said to be inverse ornegative.e.g., the correlation between volume and the pressure of a perfect gas or the correlationbetween the price and demand is negative.Correlation is said to be perfect if the deviation in one variable is a followed by acorresponding proportional deviation in the other.3.4.1 Scatter or dot diagramsIt is the simplest method of the diagrammatic representation of bivariate data. Let( xi , yi ), i 1, 2,3,., n be a bivariate distribution. Let the values of the variables x and y beplotted along the x-axis and y-axis on a suitable scale. Then corresponding to every ordered pair,there corresponds a point or dot in the xy-plane. The diagram of dots so obtained is called a dotor scatter diagram.If the dots are very close to each other and the number of observations is not very large, afairly good correlation is expected. If the dots are widely scattered, a poor correlation isexpected.3.4.2 Coefficient of CorrelationCoefficient of correlation ( r ) lies between -1 and 1, i.e., 1 r 1 .If r is zero; no correlation between two variables, positive correlation ( 0 r 1 ); when bothvariables increase or decrease simultaneously, and negative correlation ( 1 r 0 ); whenincrease in one is associated with decrease in other variable and vice-versa.3.4.3 Karl Pearson Coefficient of Correlation

Coefficient of correlation ( r ) between two variables x and y is defined asCovariance(x, y) XY(remember)r 22Variance(x) Variance(y ) x yXY where X x x , Y y y , x , y are means of x and y data values.1 Cov(x, y ) XY is the covariance between the variables x and y,n x , y y are means of x and y series respectively, alsox nn x X2n Y 2and y nare called the Standard Deviation (SD) of x and y respectively. XY ( x x )( y y ) X 2 Y 2 ( x x )2 ( y y )2n xy x y22n x 2 x n y 2 y Alternate form : r ( x, y ) That is r ( x, y ) Here n is the number of pairs of values of x and y.Example 1. If Cov(x, y ) 10, Var( x) 25, Var( y ) 9 , find coefficient of correlation.Covariance(x, y)1010Solution: r 0.67.Variance(x) Variance(y)25 9 5 3Example 2. Calculate coefficient of correlation from the following data:xy915816714613511412310Solution: Karl Pearson coefficient of correlation ( r ) is given by r where X x x , Y y y ,Here x x 45 5 ,nx987654321 xy1516141311121089 y 45 1089X x x43210-1-2-3-4y x , y are means of x and y data values. y 108 12 .n9Y y y3421-10-2-4-3X216941012916 X2Y291641104169 Y 2XY1212410041212 XY 60 60 572819 XY X 2 Y 2

The Karl Pearson coefficient of correlation is r XY X 2 Y 2 5757 0.95 .60 60 60Example 3. Psychological tests of intelligence and of engineering ability were applied to 10students. Here is a record of ungrouped data showing intelligence ratio (I.R) and engineeringratio (E.R). Calculate the coefficient of correlation.Student ABCDEFGHIJIR104 1051021011009998969392ER101 103100989596104929794Solution : Karl Pearson coefficient of correlation ( r ) is given by r where X x x , Y y y ,Here x x 990 99 ,nStudentABCDEFGHIJ10x , y are means of x and y data values.y I.R (x)1041051021011009998969392 xE.R (y)101103100989596104929794 y 990 980 XY (i) X 2 Y 2 y 980 98 .n10X x x653210-1-3-6-7 XY y y3520-3-26-6-1-4 YX236259410193649 X2Y292540943636116 Y 2XY182560-30-618628 XY 0 170 140 92 0Substituting these values in Eq.(i), we get r XY X 2 Y 2 9292 0.59.170 140 154.3Example 4. Find the coefficient of correlation between the values of x and y (using alternateform):x1357810y81215171820Solution : Here, n 6 .xy13578121517x2192549y264144225289xy83675119

810 x1820 y 34 9064100 x324400 y22 1446 248144200 xy 582Karl Pearson's coefficient of correlation is given byn xy x y6(582) (34)(90)r ( x, y ) 0.98792222226(248) 34 6(1446) 90 n x x n y y Shortcut Method for Karl Pearson Coefficient of Correlation We can also find Karl Pearson Coefficient of Correlation by taking assumed means asshown:If we take X x a, Y y bwhere a and b are assumed means of x and y data values.x a. If x ’s are equispaced with height h , we can take : u hy b.Similarly y ’s are equispaced with height k , we can take : v kn uv u vThen r ( x, y ) .22n u 2 u n v 2 v Example 5. Find the co-efficient of correlation for the following table:x101418222630y18122463036x 22y 24, v Solution : Let u 902226301111130362244422 u v u uv v 3 3 19Here n 6. 19 12The Karl Pearson Coefficient of Correlation : r ( x, y ) i.e., r ( x, y) 6(12) ( 3)( 3)6(19) 3 26(19) 3 2 n uv u vn u 2 u 6363 3 105 105 105 5Therefore, r ( x, y ) 0.6. x 2 y 2 x y 2.Example 6. Establish the formula r 2 x y2n v 2 v 2

where r is the correlation coefficient between x and y. Using the above formula, calculate the"coefficient of correlation” from the following 135Solution: Let z x y , then z x y .Therefore, z z ( x y ) ( x y )z z (x x ) ( y y )Squaring on both sides, we get( z z )2 ( x x ) 2 ( y y ) 2 2( x x )( y y )Operating ‘ ’ on both sides, we get ( z z )2 ( x x )2 ( y y )2 2 ( x x )( y y ) ( z z )2 ( x x )2 ( y y )2 2 ( x x )( y y )nnn22r ( x x )( y y ) n x y x y x y.2 x y2Therefore, r n z x y 2r x y222which is the required result.Home work: Using the above formula, calculate the "coefficient of correlation” from the givendata. Try yourself, submit through the e-mail (nanjundappace@gmail.com)3.5 RegressionRegression is a statistical method used in finance, investing, and other disciplines that attempts todetermine the strength and character of the relationship between one dependent variable (usuallydependent variable y) and a series of other variables (known as independent variable x) and viceversa.Use of Regression Analysis(i) In the field of Business, this tool of statistical analysis is widely used. Businessmen areinterested in predicting future production, consumption, investment, prices, profits and sales etc.(ii) In the field of economic planning and sociological studies, projections of population, birthrates, death rates and other similar variables are of great use.3.5.1 Linear RegressionRegression describes the functional relationship between dependent and independent variables;which helps us to make estimates of one variable from the other. Correlation quantifies theassociation between the two variables; whereas linear regression finds the best line that predicts yfrom x and x also from y. The difference between correlation and regression is illustrated in theadjoining figure.

CorrelationRegression3.5.2 Lines of RegressionA line of regression is the straight line which gives the best fit in the least square sense to thegiven frequency.In case of n pairs ( xi , yi ); i 1, 2,3,., n. from bivariate data, we have no reason orjustification to assume y as dependent variable or x as independent variable. Either of the twomay be estimated for the given values of the other. Thus if we wish to estimate y for given valuesof x, we shall have the regression equation of the form y a bx , called the regression line of yon 'x'. If we wish to estimate x for given values of y, we shall have the regression line of the formx a by , called the regression line x on y.Thus it implies, in general, we always have two lines of regression.3.5.3 Derivation of Lines of Regression3.5.3(i) Line of Regression of y on xTo obtain the line of regression of y on x, we shall assume y as dependent variable and x asindependent variable. Let the equation of regression line of y on x isy a bx (i)The normal equations as derived by the method of least Square are: y na b x .(ii) xy a x b x2 (iii)Solving (ii) and (iii) for 'a' and 'b', we getn xy x yandb 2n x 2 x a y b x y b x .nnSubstituting the values of 'a' in Eq.(i), we gety y b( x x ) (iv)Equation (iv) is called regression line of y on x,. 'b' is called the regression coefficient of y on xand is usually denoted by b yx .Hence Eq.(iv) can be written asy y byx ( x x )is called the regression line y on x.where x , y are the mean values of x and y respectively, while

byx n xy x yn x 2 x 2. (Remember)In equation (iii), shifting the origin to ( x , y ) , we get ( x x )( y y ) a ( x x ) b ( x x ) .(v).We know that ( x x ) 0 ,X 2 ( x x )2 2 ( x x )2 n 22xnxn XY ( x x )( y y ) ( x x )( y y ) n x y X 2 Y 2 ( x x )2 ( y y )2 ( x x )( y y ) rn x yr Equation (v) reduces to ( x x )( y y ) a ( x x ) b ( x x )2 rn x y a (0) b n x2 y x yThat is byx rcalled the regression coefficient (slope of line of regression) y on x. xHere r is the coefficient of correlation, y and x are the standard deviation of x and y seriesTherefore b rrespectively.Note: The regression line of y on x is y y byx ( x x ) (remember) ycalled the regression coefficient (slope of line of regression) y on x. xWhere byx r3.5.3(ii) Line of Regression of x on yProceeding in the same way as 4.5.2(i), we can derive the regression line of x on y asx x bxy ( y y )is called the line of regression of x on yHere bxy is the regression coefficient of x on y and is given bybxy orn xy x yn y 2 y bxy r2 x ywhere the terms have their usual meanings.Here b yx and bxy are known coefficients of regression and are connected by the relation: ybyx bxy r x x r y 2 r .

Note : If r 0 , the two lines of regression become x x and y y which are two straight linesparallel to x and y axes respectively and passing through their means y and x . They aremutually perpendicular.If r 1 , the two lines of regression will coincide.3.5.4 Properties of Regression Coefficients Asbyx bxy r , the coefficient of correlation (r) is the geometric mean between thetwo regression coefficients.byx bxy byx bxy r , arithmetic mean of the two regression coefficients isSince2greater than or equal to the correlation coefficient ( r ).If there is a perfect correlation between the two variables under consideration, thenbyx bxy r ; and the two lines of regression coincide. Converse is also true, i.e. if twolines of regression coincide, then there is a perfect correlation.Since byx bxy r 2 0 , the signs of both regression coefficients b yx and bxy andcoefficient of correlation ( r ) must be same; either all three negative or all positive.Since byx bxy r 2 1 , if one of the regression coefficients is greater than unity, othermust be less than unity.Point of intersection of two lines of regression is ( x , y ), where x and y are the means xand y series respectively.If both lines of regression cut each other at right angle, there is no correlation between thetwo variables; i.e., r 0.3.5.5 Angle between the Lines of RegressionIf be the acute angle between the two regression lines for two variables x and y, then1 r 2 x y tan .r x 2 y 2 Proof: The two lines of regression are given by: y( x x ) .(i) x y and x x r x ( y y ) or y y ( x x ) .(ii) yr xy y rIf m1 and m2 are slopes of lines (i) and (ii), thenr y m m1tan 2, where m1 and m2 y1 m1m2 xr x

y r y 1 y r r x x r x tan 2 r y y y1 1 x r x x 1 r 2 x y Therefore, tan (iii)r x 2 y 2 When r 0,tan 2Therefore, the two lines of regression are perpendicular to each other. When r 1, tan 0 0Therefore, the two lines of regression are coincidentExample 1. The two regression equations of the variables x and y are x 18.13 0.87 y andy 11.64 0.54 x . Find (1) the mean of x’s and y’s, (2) the co-efficient of correlation between xand y.Solution: Given x 18.13 0.87 y .(i)y 11.64 0.54 x (ii)andSince the mean of x’s and y’s lie on the two regression lines, we havex 18.13 0.87 y .(iii)y 11.64 0.54 x .(iv)and(1) On solving the above equations, we get x 15.79 and y 3.74.(2) Regression coefficient y on x from Eq(ii) is byx 0.54 and Regression coefficient x on yfrom Eq(i) is bxy 0.87.Therefore, the coefficient of correlation is the geometric mean between the two regressioncoefficients is given by r byx bxy ( 0.54)( 0.87) 0.66 0.66.(here the -sign is taken, since both the regression coefficients are - sign ).Example 2. In the partially destroyed laboratory record, only the lines of regression of y on xand x on y are available as 4 x 5 y 33 0 and 20 x 9 y 107 respectively. Calculate (ii) xand y , and (ii) the co-efficient of correlation between x and y.Solution: Given 4 x 5 y 33 .(i)20 x 9 y 107 (ii).and(i) Since the mean of x’s and y’s lie on the two regression lines (i) and (ii), we have4 x 5 y 3320 x 9 y 107 .On solving these equations, we have x 13 and y 17.433(ii) The regression line y on x from the Eq(.(i) is y x .(iii)554The regression coefficient y on x is byx .59107y Similarly, the regression line x on y from the Eq(.(ii) is x .(iii)2099The regression coefficient x on y is bxy .20

Therefore, the coefficient of correlation is the geometric mean between the two regression 4 9 coefficients is given by r byx bxy 0.36 0.6 0.6 . 5 20 (here, the sign is taken, since both the regression coefficients are sign ).Example 3. In the following table are recorded data showing the test scores made by salesmenon an intelligence test and their weekly sales :Salesmen12345678910Test Scores x 40 70 50608050 9040 60 60Sales(000) y2.5 6.0 4.55.04.52.0 5.53.0 4.5 3.0Calculate the lines of regression of sales (y) on test scores (x) and estimate the most probableweekly sales volume if a sales man makes a score of 70.Solution: Determine the regression line of sales (y) on test scores (x): y y byx ( x x ) .where byx n xy x yn x x 2From the table, we havex mean of x (test scores) 2. x 600 60n10 y 40.5 4.05 .y mean of y (sales) n10Test scores x Sales(000) y 03.0 y 40.5 x 600Therefore, byx n xy x yn x 2 x 2and 0.06 .The required regression line y on x is y 4.05 (0.06)( x 60)i.e., y 0.06 x 0.45 .When x 70 , y 0.06(70) 0.45 4.65 .Thus the most probable weekly sales volume if a sales man makes a score of 70 is 4.65.Example 4. Following data depicts the statistical values of rainfall and production of wheat in aregion for a specified time period.Mean Standard DeviationProduction of Wheat (kg. per unit area) y108

Rainfall (cm) x82Estimate the production of wheat (y) when rainfall (x) is 9cm if correlation coefficient betweenproduction and rainfall is given to be 0.5.Solution: Let the variables x and y denote rainfall and production respectively.Given that x 8, x 2; y 10, y 8, r 0.5.Now equation of regression of y on x is given by:y y r y(x x ) x(0.5)(8)( x 8)2 y 10 2( x 8) y 2x 6 0 . y 2x 6That is the production of wheat on the rain fall.When rainfall is 9cm (x), production (y) of wheat is estimated to be 2(9) 6 12 kg. per unitarea. y 10 Example 5. Find the coefficient of correlation and the lines of regression for the datagiven below:n 18, x 12, y 18, x2 60, y 2 96, xy 48.Solution: (i) The coefficient of correlation: r ( x, y ) n xy x yn x x 2(ii) Equations of regression lines: y y byx ( x x ) ,Now x x 12 0.67,n18 x2 xn2y 2n y y 2x x bxy ( y y ) . y 18 1.n18 x 16 12 2.9 x 1.7. n 18 18 y 2 y 222296 18 4.33 y 2.08. nn18 18 y 1.7 2.08 bxy r x (0.57) byx r (0.57) 0.47 . 0.7 ; y x 2.08 1.7 y2(i) The coefficient of correlation: r 18(48) (12)(18)18(60) 12 218(96) 18 2 0.57(ii) Equations of regression lines :y y byx ( x x ) , x x bxy ( y y ) y 1 0.7( x 0.67) , x 0.67 0.47( y 1) y 0.7 x 0.53 , x 0.47 y 0.2 .Example 6. Find the correlation coefficient between x and y, when the two lines ofregression are given by: 2 x 9 y 6 0 and x 2 y 1 0 .Solutio

The process of constructing an approximate curve x which fit best to a given discrete set of points ,xyii in., is called curve fitting Principle of Least Squares: The principle of least squares (PLS) is one of the most popular methods for finding the curve of best fit to a given data set ,nii. Let be the equation of the curve to be fitted to .