Ed By [University Of Toronto] At 16:20 23 May 2014 .

Downloaded by [University of Toronto] at 16:20 23 May 2014PrefaceThere are many books on regression and analysis of variance. These books expectdifferent levels of preparedness and place different emphases on the material. This bookis not introductory. It presumes some knowledge of basic statistical theory and practice.Readers are expected to know the essentials of statistical inference such as estimation,hypothesis testing and confidence intervals. A basic knowledge of data analysis ispresumed. Some linear algebra and calculus are also required.The emphasis of this text is on the practice of regression and analysis of variance. Theobjective is to learn what methods are available and more importantly, when they shouldbe applied. Many examples are presented to clarify the use of the techniques and todemonstrate what conclusions can be made. There is relatively less emphasis onmathematical theory, partly because some prior knowledge is assumed and partly becausethe issues are better tackled elsewhere. Theory is important because it guides theapproach we take. I take a wider view of statistical theory. It is not just the formaltheorems. Qualitative statistical concepts are just as important in statistics because theseenable us to actually do it rather than just talk about it. These qualitative principles areharder to learn because they are difficult to state precisely but they guide the successfulexperienced statistician.Data analysis cannot be learned without actually doing it. This means using a statisticalcomputing package. There is a wide choice of such packages. They are designed fordifferent audiences and have different strengths and weaknesses. I have chosen to use R(Ref. Ihaka and Gentleman (1996) and R Development Core Team (2003)). Why have Iused R? There are several reasons.1. Versatility. R is also a programming language, so I am not limited by the procedures thatare preprogrammed by a package. It is relatively easy to program new methods in R.2. Interactivity. Data analysis is inherently interactive. Some older statistical packageswere designed when computing was more expensive and batch processing ofcomputations was the norm. Despite improvements in hardware, the old batchprocessing paradigm lives on in their use. R does one thing at a time, allowing us tomake changes on the basis of what we see during the analysis.3. Freedom. R is based on S from which the commercial package S-plus is derived. Ritself is open-source software and may be obtained free of charge to all. Linux, Macintosh, Windows and other UNIX versions are maintained and can be obtained from theR-project at www.r-project.org. R is mostly compatible with S-plus, meaning that Splus could easily be used for most of the examples provided in this book.4. Popularity. SAS is the most common statistics package in general use but R or S ismost popular with researchers in statistics. A look at common statistical journals confirms this popularity. R is also popular for quantitative applications in finance.Getting Started with RR requires some effort to learn. Such effort will be repaid with increased productivity.You can learn how to obtain R in Appendix A along with instructions on the installationof additional software and data used in this book.

Downloaded by [University of Toronto] at 16:20 23 May 2014xii PrefaceThis book is not an introduction to R. Appendix B provides a brief introduction to thelanguage, but alone is insufficient. I have intentionally included in the text all thecommands used to produce the output seen in this book. This means that you canreproduce these analyses and experiment with changes and variations before fullyunderstanding R. You may choose to start working through this text before learning Rand pick it up as you go. Free introductory guides to R may be obtained from the Rproject Web site at www.r-project.org. Introductory books have been written by Dalgaard(2002) and Maindonald and Braun (2003). Venables and Ripley (2002) also have anintroduction to R along with more advanced material. Fox (2002) is intended as acompanion to a standard regression text. You may also find Becker, Chambers, and Wilks(1998) and Chambers and Hastie (1991) to be useful references to the S language. Ripleyand Venables (2000) wrote a more advanced text on programming in S or R.The Web site for this book is at www.stat.lsa.umich.edu/ faraway/LMR where datadescribed in this book appear. Updates and errata will appear there also.Thanks to the builders of R without whom this book would not have been possible.

CHAPTER 1IntroductionDownloaded by [University of Toronto] at 16:20 23 May 20141.1 Before You StartStatistics starts with a problem, proceeds with the collection of data, continues with thedata analysis and finishes with conclusions. It is a common mistake of inexperiencedstatisticians to plunge into a complex analysis without paying attention to what theobjectives are or even whether the data are appropriate for the proposed analysis. Lookbefore you leap!The formulation of a problem is often more essential than its solution whichmay be merely a matter of mathematical or experimental skill. Albert EinsteinTo formulate the problem correctly, you must:1. Understand the physical background. Statisticians often work in collaboration withothers and need to understand something about the subject area. Regard this as anopportunity to learn something new rather than a chore.2. Understand the objective. Again, often you will be working with a collaborator whomay not be clear about what the objectives are. Beware of “fishing expeditions”—ifyou look hard enough, you will almost always find something, but that something mayjust be a coincidence.3. Make sure you know what the client wants. You can often do quite different analyseson the same dataset. Sometimes statisticians perform an analysis far more complicatedthan the client really needed. You may find that simple descriptive statistics are all thatare needed.4. Put the problem into statistical terms. This is a challenging step and where irreparableerrors are sometimes made. Once the problem is translated into the language ofstatistics, the solution is often routine. Difficulties with this step explain why artificialintelligence techniques have yet to make much impact in application to statistics.Defining the problem is hard to program.That a statistical method can read in and process the data is not enough. The results of aninapt analysis may be meaningless.It is also important to understand how the data were collected. were they obtained via a designed sample survey. How the data were collected has acrucial impact on what conclusions can be made. Is there nonresponse? The data you do not see may be just as important as the datayou do see. Are there missing values? This is a common problem that is troublesome and timeconsuming to handle. How are the data coded? In particular, how are the qualitative variables represented? What are the units of measurement?

2Linear Models with R Beware of data entry errors and other corruption of the data. This problem is all too common —almost a certainty in any real dataset of at least moderate size. Perform some data sanity checks.Downloaded by [University of Toronto] at 16:20 23 May 20141.2 Initial Data AnalysisThis is a critical step that should always be performed. It looks simple but it is vital. Youshould make numerical summaries such as means, standard deviations (SDs), maximumand minimum, correlations and whatever else is appropriate to the specific dataset.Equally important are graphical summaries. There is a wide variety of techniques tochoose from. For one variable at a time, you can make boxplots, histograms, density plotsand more. For two variables, scatterplots are standard while for even more variables,there are numerous good ideas for display including interactive and dynamic graphics. Inthe plots, look for outliers, data-entry errors, skewed or unusual distributionsand structure. Check whether the data are distributed according to prior expectations.Getting data into a form suitable for analysis by cleaning out mistakes and aberrations isoften time consuming. It often takes more time than the data analysis itself. In this course,all the data will be ready to analyze, but you should realize that in practice this is rarely the case.Let’s look at an example. The National Institute of Diabetes and Digestive and KidneyDiseases conducted a study on 768 adult female Pima Indians living near Phoenix. Thefollowing variables were recorded: number of times pregnant, plasma glucoseconcentration at 2 hours in an oral glucose tolerance test, diastolic blood pressure(mmHg), triceps skin fold thickness (mm), 2-hour serum insulin (mu U/ml), body massindex (weight in kg/(height in m2)), diabetes pedigree function, age (years) and a testwhether the patient showed signs of diabetes (coded zero if negative, one if positive). Thedata may be obtained from UCI Repository of machine learning databases atwww.ics.uci.edu/ mlearn/MLRepository.html.Of course, before doing anything else, one should find out the purpose of the study andmore about how the data were collected. However, let’s skip ahead to a look at the data: library(faraway) data (pima) pimapregnant glucose diastolic16148722185663818364 much deleted 76819370triceps insulin bmi diabetes age35033.60.627 5029026.60.351 310023.30.672 3231030.40.31523The library (faraway) command makes the data used in this book available. You need toinstall this package first as explained in Appendix A. We have explicitly written thiscommand here, but in all subsequent chapters, we will assume that you have alreadyissued this command if you plan to use data mentioned in the text. If you get an errormessage about data not being found, it may be that you have forgotten to type this.

Introduction 3Downloaded by [University of Toronto] at 16:20 23 May 2014The command data (pima) calls up this particular dataset. Simply typing the name ofthe data frame, pima, prints out the data. It is too long to show it all here. For a dataset ofthis size, one can just about visually skim over the data for anything out of place, but it iscertainly easier to use summary methods.We start with some numerical summaries:The summary ( ) command is a quick way to get the usual univariate summaryinformation. At this stage, we are looking for anything unusual or unexpected, perhapsindicating a data-entry error. For this purpose, a close look at the minimum andmaximum values of each variable is worthwhile. Starting with pregnant, we see a maximum value of 17. This is large, but not impossible. However, we then see that the nextfive variables have minimum values of zero. No blood pressure is not good for thehealth—something must be wrong. Let’s look at the sorted values:

Downloaded by [University of Toronto] at 16:20 23 May 20144Linear Models with RWe see that the first 35 values are zero. The description that comes with the data saysnothing about it but it seems

Analysis of Failure and Survival Data. Peter J.Smith. The Analysis and Interpretation of Multivariate Data for Social Scientists. David J.Bartholomew, Fiona Steele, Irini Moustaki, and Jane Galbraith. The Analysis of Time Series—An Introduction, Sixth Edition. Chris Chatfield. Applied Bayesian Forecasting