Lecture 10 Polynomial Regression
Lecture 10Polynomial regressionBIOST 515February 5, 2004BIOST 515, Lecture 10
Polynomial regression modelsy Xβ is a general linear regression model for fitting any relationshipthat is linear in the unknown parameters, β. For example, thefollowing polynomialy β0 β1x1 β2x21 β3x31 β4x2 β5x22 is a linear regression model because y is a linear function of β.BIOST 515, Lecture 101
Polynomial models in one variableA kth order polynomial in one variable is defined asy β0 β1x β2x2 · · · βk xk .Polynomial models are useful in situations where the analyst knows that curvilinear effectsare present in the true response function as approximating functions to unknown and possibly verycomplex nonlinear relationships.We can think of the polynomial model as the Taylor seriesexpansion of the unknown function.BIOST 515, Lecture 102
Important considerations Order of the model Model building strategy Extrapolation Ill-conditioning HierarchyBIOST 515, Lecture 103
Piecewise polynomialsA low-order polynomial may provide a poor fit to the data,and increasing the order of the polynomial may not help. Transformations of x or y may solve this problem, but sometimes wemay prefer to use more flexible approaches. One such approachis to use splines. piecewise polynomials used in curve fitting polynomials within intervals of x that are connected acoressdifferent intervals of xBIOST 515, Lecture 104
The piecewise linear spline function is given byf (x) β0 β1x β2(x a) β3(x b) β4(x c) ,where u, u 0,(u) 0, u 0and a, b and c are referred to as knots.BIOST 515, Lecture 105
105y15Example of piecewise linear spline with knots at 2, 5 and 8.0246810xBIOST 515, Lecture 106
As we increase the number of knots, the piecewise linear polynomial more closely resembles a continuous line.3 knots6 knots21 2 02468 010 2y y2 10 0246xx9 knots25 knots8021 2246xBIOST 515, Lecture 10810 010y 2 y2 10 0246810x7
Cubic splinesAlthough, linear splines may work well, they are not smoothand will not fit highly curved functions well (unless many knotsare used - which requires a lot of data).It is more common for cubic splines to be used in practice.A cubic spline function with k knots is given byf (x) 3Xj 0β0j xj kXβi(x tl)3 ,l 1where tl, l 1, . . . , k are the k knots. We relate x to theoutcome asyi f (xi) i.BIOST 515, Lecture 108
3 knots6 knots21 2 02468 010 2y y2 10 0246xx9 knots25 knots8046xBIOST 515, Lecture 1021 22810 010y 2 y2 10 0246810x9
For estimation purposes, we assume that both the locationsand the number of knots are fixed. Although there are methodsthat allow the number and/or position of the knots to berandom; these models are too complex to be fit using leastsquares.The piecewise cubic splines may give us a more flexiblemodel, but they still may be discontinuous at the knots.BIOST 515, Lecture 1010
Continuous cubic splinesCubic B splines Given k knots at t1, . . . , tk , a cubic B spline function is acubic polynomial on the interval [tj , tj 1], It has continuous first and second derivatives, imposing 3conditions at each knot. With k knots, k 1 parameters are needed to represent thecubic spline.BIOST 515, Lecture 1011
A cubic B-spline function with k knots is given byf (x) k 4Xβk Bk (x),i 1where Bk (x) is the kth B-spline basis function.BIOST 515, Lecture 1012
T 515, Lecture 1013
6810tBIOST 515, Lecture 1014
0.80.60.4q βkBk(t)0.00.2k 10246810tBIOST 515, Lecture 1015
2 knots3 knots420f(x)20f(x) 4 2 4 10 4 8 2 6f(x) 20421 knot0204060xBIOST 515, Lecture 10801000204060x80100020406080100x16
ExampleVenables and Ripley provide a data set, GAGurine, in theMASS library. It is described as follows:Data were collected on the concentration of a chemicalGAG in the urine of 314 children aged from zero to seventeenyears. The aim of the study was to produce a chart tohelp a paediatrican to assess if a child’s GAG concentration is”normal”.BIOST 515, Lecture 1017
50 30 01020GAG40 051015AgeBIOST 515, Lecture 1018
50 1 knot6 knots20 knots30 01020GAG40 051015AgeBIOST 515, Lecture 1019
lmbs1 lm(GAG bs(Age,df 5),data GAGurine)plot(GAGurine Age,GAGurine GAG,col "gray",ylab "GAG", xlab "Age")lines(GAGurine Age,fitted(lmbs1),lwd 2)50 30 01020GAG40 051015AgeBIOST 515, Lecture 1020
Choosing the number and position of knots Knots are usually placed at quantiles of the data or atregularly spaced intervals. Choosing the number, rather than the placement, seems tobe more crucial to the fit. Therefore choose a number of knots that represents thecurvature you believe to be present in the data. This comeswith experience. You may also want to place knots at points in the data whereyou expect significant changes in the relationship betweenthe predictor and the outcome to occur.BIOST 515, Lecture 1021
Polynomial regression models y Xβ is a general linear regression model for fitting any relationship that is linear in the unknown parameters, β. For example, the following polynomial y β 0 β 1x 1 β 2x 2 1 β 3x 3 1 β 4x 2 β 5x 2 2 is a linear regression model because y is a linear
independent variables. Many other procedures can also fit regression models, but they focus on more specialized forms of regression, such as robust regression, generalized linear regression, nonlinear regression, nonparametric regression, quantile regression, regression modeling of survey data, regression modeling of
Introduction of Chemical Reaction Engineering Introduction about Chemical Engineering 0:31:15 0:31:09. Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Lecture 27 Lecture 28 Lecture
3 LECTURE 3 : REGRESSION 10 3 Lecture 3 : Regression This lecture was about regression. It started with formally de ning a regression problem. Then a simple regression model called linear regression was discussed. Di erent methods for learning the parameters in the model were next discussed. It also covered least square solution for the problem
Identify polynomial functions. Graph polynomial functions using tables and end behavior. Polynomial Functions Recall that a monomial is a number, a variable, or the product of a number and one or more variables with whole number exponents. A polynomial is a monomial or a sum of monomials. A polynomial
Polynomial Functions Pages 209–210 Check for Understanding 1. A zero is the value of the variable for which a polynomial function in one variable equals zero. A root is a solution of a polynomial equation in one variable. When a polynomial function is the related function to the polynomial
LINEAR REGRESSION 12-2.1 Test for Significance of Regression 12-2.2 Tests on Individual Regression Coefficients and Subsets of Coefficients 12-3 CONFIDENCE INTERVALS IN MULTIPLE LINEAR REGRESSION 12-3.1 Confidence Intervals on Individual Regression Coefficients 12-3.2 Confidence Interval
Probability & Bayesian Inference CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition J. Elder 3 Linear Regression Topics What is linear regression? Example: polynomial curve fitting Other basis families Solving linear regression problems Regularized regression Multiple linear regression
Artificial Intelligence (AI) is a Cognitive Science and the history of its evolution suggests that it has grown out of the knowledge derived from disciplines such as Science, Mathematics, Philosophy, Sociology, Computing and others. Hence, it is fair for any education system to recognize the importance of integrating AI Readiness to maximize learning across other disciplines. AI is being .
