Technical Re Por T - UNC Public Policy
Technical Report:The Impact of Teacher PreparationonStudent LearninginNorth Carolina Public SchoolsApril 2010Carolina Institutefor Public Policy
Technical Report:The Impact of Teacher PreparationonStudent LearninginNorth Carolina Public SchoolsbyGary T. Henry, UNC-Chapel HillCharles L. Thompson, East Carolina UniversityC. Kevin FortnerKelly M. PurtellRebecca A. ZulliDavid C. Kershaw
Table of Contents1.2.3.4.IntroductionOverall Study Design: Value Added ModelsEquations for Estimating Teacher ImpactsMeasures and Study Samplea. Focal Variables into Teachingb. Focal Variables: Teaching Preparation Programsc. Merging Multiple Data Sets to Create the Study Data Based. Outcome Variables: Student Test Scorese. Covariates: Influences on Student Test Scores Controlled in the AnalysisStudent and Family CharacteristicsClassroom and Teacher CovariatesSchool Level Covariatesf. Summary1146677889101111AppendixTable 1: Teacher CharacteristicsTable 2: Universityof North Carolina System School AbbreviationsTable 3: Traditionally Trained UNC Teachers with Less 10 Years ExperienceIncluded in the AnalysisTable 4: Student Roster CountsTable 5: School CharacteristicsTable 6A: Elementary School Route Comparison Models, Standardized EOG Reading ScoresTable 6B: Elementary School Route Comparison Models, Standardized EOG Math ScoresTable 6C: Elementary School Program Comparison Models, Standardized EOG Reading ScoresTable 6D: Elementary School Program Comparison Models, Standardized EOG Math ScoresTable 6E: Middle School Route Comparison Models, Standardized EOG Reading ScoresTable 7F: Middle School Route Comparison Models, Standardized EOG Math ScoresTable 6G: Middle School Program Comparison Models, Standardized EOG Reading ScoresTable 6H: Middle School Program Comparison Models, Standardized EOG Math ScoresTable 6I: High School Route Comparison Models, Standardized EOC ScoresTable 6J: High School Program Comparison Models, Standardized EOC ScoresTable 6K: High School Program Comparison Models, Standardized EOC Scores by Subject 20A.22A.24
1. IntroductionThe main objective of the research reported here was to estimate the average effects of UNCtrained teachers on students’ test scores in North Carolina. The research was carried out byanalyzing more than 1.94 million test scores that we linked to 143,892 classes in elementary,middle, and high schools across the state. We estimated the value added by 12,814 teachersprepared by the University of North Carolina at its 15 institutions. In addition to the averageoverall estimates of the effectiveness of teachers prepared by the UNC system, we made separateimpact estimates for each of the 15 institutions. In our earlier report, The Impact of TeacherPreparation on Student Learning in North Carolina Public Schools, we described the objectivesand findings of the study, commented briefly on our methods, and interpreted the results. In thiscompanion Technical Report, we provide details of the methods used to produce the studyfindings, including our design, sample, and modeling approach.We begin with a discussion of the overall study design, including the rationale for choosing theapproach to modeling. The equations used for estimates are presented in section 3, and the datathat were compiled for the study are described in section 4. The report concludes with tables thatdescribe study data and the findings.2. Overall Study Design: Valued Added ModelsConsistent with most of current research that seeks to estimate teacher or other effects on studentachievement, we used value added models for this study. Value added models includeinformation on students’ prior test performance in order to assess the influences on students’achievement during a year of schooling. By including prior test performance along with a richarray of other control variables, these models separate out the effects of many variables that arebeyond the control of teachers and their preparation programs, including measured differencesbetween students and differences in the resources or experiences provided by their classrooms,schools, families and neighborhoods. In other words, these models are designed to isolate theeffects of a teacher on students’ test score gains in a specific subject and year from all otherinfluences on their achievement, such as their academic ability and motivation, familybackground, and prior educational experiences.In recent years, value added models have become more common as better longitudinal data onstudents’ test scores and other aspects of their family background and schooling have beencompiled from the accountability data that most states collect. The characteristic common to allvalue added models is that they contain information on student prior test performance.Numerous variations in the models have been developed in recent years, but researchers havefrequently implemented three distinct types of value added models to assess the effects ofteachers on student achievement: (1) year-to-year value added models with controls for student,
Technical Report: Impact of Teacher PreparationPage 2 of 12classroom, and school characteristics; (2) year-to-year value added student fixed effects models;and (3) multiple year value added models.(1) The first type of value added model generally uses multilevel modeling to isolate theeffects of individual student, teacher, classroom, school, and school district characteristics.These models always include one or more prior test scores for each student. All students in thestudy population who have test scores available from consecutive years can be included in theestimates of effects.(2) The second type of model controls for all characteristics of students that do notchange during the study period. Instead of comparing students to each other, fixed effects useeach student as his or her own control to examine the effects of changes in resources orexperiences on the student over time. This is done by subtracting each student’s meanstandardized score across years from his or her annual score. Students who do not experiencechanges in the experiences of interest for a particular study are not included in the estimates ofthe effects of those experiences. For the current study, this means that students who always (ornever) experience teachers prepared by the UNC system would be omitted from the estimates ofthe average effects of teacher preparation programs. In addition, only students with threeconsecutive years of test scores could be included in the study.(3) The third type of value added model uses all available test scores to estimate eachteacher’s contribution to the variation in their students’ test scores. These models commonlyestimate the effects of teachers on students beginning in the year the teacher taught the studentand each year thereafter. Individual teachers’ contributions to each of their students’ test scoresare aggregated and then compared to the average effects for teachers in the same grade and thesame subject. These estimates of teachers’ effectiveness are based on the extent to which theirstudents consistently exceed or fall below the averages for their grade and subject. This type ofvalue added model requires at least three test scores for any student to be included, minimizesany problems from missing data, and sets expectations for each student’s gains in the same waywithout regard to differences in their individual characteristics, or the characteristics of theirclassrooms, schools, or school districts.Currently, a great deal of time and effort is being devoted to assessing which type of model is thebest for making educational impact estimates. In addition, new techniques are being developed,such as the analysis of individual student achievement growth over time that may provide betterapproaches in the future. All three types of models have some substantial strengths and somelimitations or weaknesses. Since our purpose was to provide accurate estimates of the effects ofUNC teacher preparation and its 15 individual programs, we evaluated each type of value addedmodel using five criteria: accuracy, fairness, feasibility, consistency, and transparency. In theend, we elected to use year-to-year value added models with extensive controls for severalreasons:
Technical Report: Impact of Teacher PreparationPage 3 of 121. Value added models with student fixed effects, which compare students to only their own testscores in the same subject (either reading or mathematics), can only be estimated in the thirdthrough eighth grade. UNC teacher preparation programs train their graduates for highschool as well as elementary and middle schools. We intended to assess impacts on highschool End-of-Course test scores as well as 3rd-8th grade scores, and for this purpose, theaccuracy and feasibility of fixed effect and multiple year random effects models are muchless well established.2. A fair assessment of program effects should eliminate, to the greatest extent possible, choicesthat are beyond the control of the program that can have an effect on the achievement ofstudents taught by their graduates. In consultation with University of North Carolinaadministrators and Deans of the system’s schools of education, we decided to control for thechoices that their graduates had made about which schools and what students they wouldteach. Thus, estimates of a program’s effectiveness should not be affected if their graduateschoose to teach in underperforming schools with high concentrations of poverty. Thisrequired that we develop and use an extensive set of controls for student, classroom, andschool characteristics. Including these controls does not mean that adjustments for student orschool differences are automatic, nor necessarily set different expectations for differentstudents or schools. It does mean that if the evidence indicates that differences betweenstudents or schools are systematically related to students’ test score gains, then year-to-yearvalue added models with extensive controls will separate out the effects of those differencesfrom the effects of teacher preparation programs.3. In addition, we determined that it might be unfair to credit some teachers with test scoreincreases of their students in the years after they had left their classes, as is done in multipleyear value added models, and not to credit other teachers, such as 8th grade teachers ofreading and mathematics, whose students are not tested in these subjects in the same way inhigh school.For these and other reasons, we elected to use year-to-year multilevel, value added models withextensive controls. We believe they are as accurate, fair, feasible, consistent, and transparent asany currently available. Nevertheless, we will continue to assess our methods, and hope tocontribute to the ongoing research to develop and improve modeling capacities.Year-to-year value added models with extensive controls, which we will refer to as VAM, can beestimated with a variety of techniques. The most common modeling approach, ordinary leastsquares regression (OLS), assumes that all observations within a data set are independent. Inreality, educational data contains student observations that are nested within classrooms, and inturn, those classrooms are nested within schools. This violation of OLS modeling assumptionsleads to under-estimates of standard errors. Failure to adjust the standard errors for clusteringincreases the likelihood that variables included in models will be identified as statisticallysignificant when they should not be. Therefore, to estimate these models, we implementmultilevel models, also referred to as mixed models or hierarchical linear models, which separate
Technical Report: Impact of Teacher PreparationPage 4 of 12the error term at each level to account for the nested nature of educational data. This producescoefficients that more accurately estimate the differences between the effectiveness of teachersfrom different programs, and allows us to correctly test whether the differences are likely due tochance variations. For these models, the tests of significance all assess whether the effectivenessof UNC prepared teachers are reliably different from all sources of teachers other than theUniversity of North Carolina system, including teachers prepared in private colleges anduniversities in North Carolina, colleges and universities in other states, and those that entered theprofession without formal preparation to teach.3. Equations for Estimating Teacher ImpactsIn this study, we compare the performance of teachers who entered NC public schools via tworoutes -- traditional undergraduate teacher education from a University of North Carolina systemschool (Traditional UNC) and a Master of Arts in Teaching (MAT) graduate education programprovided by a UNC system school – to all other sources of teachers. Traditional undergraduateteacher training includes teachers who majored in a teacher preparation degree program, as wellas teachers majoring in another subject who took classes in a school of education to concurrentlygain a teaching credential concurrently with that major. MAT teachers entered a UNC MATtraining program holding a bachelor’s degree that did not include the teacher education coursesrequired for licensure. We limit all analyses to teachers with less than 10 years of experience, asthe influence of one’s initial preparation route on teaching practice is likely to erode over time.The equation used to estimate the effect of the teacher routes is:is a student’s test score, specifically student i in classroom j in school s at time t;Whereis the estimate of the average effect of the teachers from the traditional UNC route on thetest scores;is the estimate of the average effect of the teachers from the UNC MAT route on the testscores;is an indicator variable that equals 1 if the teacher graduated from a UNC institutionand 0 if not;is an indicator variable that equals 1 if the teacher graduated from a UNC MATprogram and 0 if not;represents a prior test score or scores;
Technical Report: Impact of Teacher PreparationPage 5 of 12represents a set of individual student characteristics;is the estimate of the average effects of individual student characteristics;represents a set of classroom characteristics;is the estimate of the average effects of classroom characteristics;represents a set of school characteristics;is the estimate of the average effects of school characteristics;andandare disturbance terms representing unexplained variation at the individual,classroom, and school levels, respectively.We implemented a second type of model as well, one that estimates a separate effect for each ofthe 15 UNC institutions that prepare K-12 teachers. The second type of model, the programmodel, included indicator variables for the 15 UNC programs that prepare teachers for NorthCarolina classrooms. The group to which each of the UNC programs is compared is teachersprepared by all other methods with less than 10 years of teaching experience. The equation forthe teacher training program comparison models is:Whereis a student’s test score, specifically student i in classroom j in school s at time t;provide estimates of the average effect of the teachers from each of the 15 programson the test score;is an indicator variable that equals 1 if the teacher graduated from the first of the 15UNC traditional programs and 0 if not;represents a prior test score or scores;represents a set of individual student characteristics;is the estimate of the average effects of individual student characteristics;represents a set of classroom characteristics;is the estimate of the average effects of classroom characteristics;
Technical Report: Impact of Teacher PreparationPage 6 of 12represents a set of school characteristics;is the estimate of the average effects of school characteristics;andandare disturbance terms representing unexplained variation at the individual,classroom, and school levels, respectively.For elementary grades students, prior scores in both mathematics and reading are included. Themodels differ slightly for high school, however, in that students’ 8th grade test scores in bothmathematics and reading are used as the prior test scores. Three sets of models wereimplemented: a total effects model and two additional models that sequentially added teachercharacteristics to the above equations. First, for the total effects estimates of the routes orprograms, at the teacher/classroom level only the route or program indicator variables andcharacteristics of the classrooms (for example, number of students in the classroom) wereincluded. All student and school level variables were included. Second, to examine the potentialinfluence of other teacher characteristics, indicator variables representing qualifications obtainedafter entering teaching -- such as National Board Certification or masters degrees -- were addedto the first set of teacher controls. Finally, in order to help separate the educative effects of theteacher preparation programs from the academic ability of teachers attending a particularinstitution’s program, we included the teacher’s high school grade point average and their SATscores in the models.4. Measures and Study SampleIn this section we describe the way each of the measures was constructed and the sources of thedata that were used. In addition, this section provides more detailed information on the methodsused to construct the data sets included in models.a. Focal Variables: Routes into TeachingTwo types of models, described above, were used to estimate the effects of UNC routes intoteaching in North Carolina and the effects of the 15 UNC teacher preparation programs onstudent outcomes. The sets of models are further divided by level of schooling into elementary,middle, and high school. The University of North Carolina General Administration provided theinformation on graduates of the system who served as teachers in North Carolina since the 199899 school year, which as indicated above, were merged into the data set for analysis.Two specific routes into the classroom were distinguished for these analyses. First, teacherswere coded as traditionally trained if they graduated from a UNC institution with a major ineducation or were trained as teachers during their undergraduate education while majoring in
Technical Report: Impact of Teacher PreparationPage 7 of 12another field. These designations were made based on Classification of Instructional Program(CIP) codes developed by the National Center for Education Statistics and standard across allinstitutions within the UNC system. A second route into the teaching field identified in this studyis a graduate training program known as a Master of Arts in Teaching (MAT). MAT teachershold a bachelors degree in a non-teaching field and complete an MAT program to simultaneouslyobtain teacher licensure and a Masters degree before entering the classroom as teachers. Thesetwo groups of teachers were compared to all other teachers in the state. The data on teachertraining available from the UNC General Administration is limited to those teachers thatgraduated from a UNC system institution since 1998. Based on the expectation that the effects ofteacher training diminish over time, we limit the analysis to all teachers with less than 10 yearsof experience. In Table 1 in the Appendix, we display the descriptive characteristics of allteachers in the North Carolina public schools in 2004-05 through 2007-08 (the years for whichwe have student test scores that we matched to their teachers using roster information providedby all NC public schools), all teachers in North Carolina public schools with less than 10 years ofexperience in 2004-05 through 2007-08, and teachers with less than 10 years of experience whotaught tested subjects in 2004-05 through 2007-08.b. Focal Variables: Teacher
trained teachers on students’ test scores in North Carolina. The research was carried out by analyzing more than 1.94 million test scores that we linked to 143,892 classes in elementary, middle, and high schools across the state. We estimated the value added by 12,814 teachers prepared by the University of North Carolina at its 15 institutions.
UNC C UNC CH UNC G UNC GA UNC P UNC SA WCU WSSU UNC W-45% -35% -25% -15% -5% 5% 15% 25% 35% 45% Millions Btu/gsf greater than target with use decreasing Btu/gsf greater than target with use INCREASING Btu/gsf less than target with use decreasing Btu/gsf less than target with use increasing kWH 46.0% NG 15.6% 2Oil 2.5% 6Oil 0.6% Propane 0.2 .
School of Medicine, Gillings School of Global Public Health, UNC Medical Center, or other relevant UNC-CH affiliations (the search strategy is available at: https://go.unc.edu/UNC- . Marsico Lung Institute UNC Hussman School of Journalism and Media NC Translational and Clinical Sciences
8 200 3/4” - 10 unc 4.5 114 12 3.5 89 4 28 2.5 63 4 24 10 250 7/8” - 9 unc 5 127 20 3.5 89 4 44 2.5 63 4 40 12 300 7/8” - 9 unc 5 127 20 3.75 95 4 44 2.75 70 4 40 14 350 1” - 8 unc 5.5 140 20 4.25 108 4 44 3 76 4 40 16 400 1” - 8 unc 5.75 146 28 4.5 114 4 60 3.25 82 4 56 18 450 1 1/8” - 8
the 4.1-meter diameter SOAR telescope.4-6 For the WMAP cosmology, this redshift corresponds to 12.8 billion years ago, when the universe was only 6% of its current age. . UNC-Asheville, UNC-Charlotte, UNC-Greensboro, UNC-Pembroke, and Western Carolina University), (3) UNC-CH’s Morehead Planetarium and Science Center (MPSC), and (4) the US
Crossley, Kayla UNC-Charlotte Chancellor's Scholarship UNC Charlotte Tuition Scholarship University of North Carolina at Charlotte Cruz, Benicio Sanchez The Arrupe Scholarship . Stump, Madison UNC-Wilmington SOAR Ambassador Scholarship UNC-Wilmington Hubert Anthony Simpson Scholarship UNC-Wilmington Sykes, Rebecca Appalachian State University .
Hospitals Radiation Therapy Program is located in the UNC Department of Radiation Oncology in Chapel Hill, NC. The UNC Department of Radiation Oncology was formed in 1987 from the UNC Division of Radiation Therapy. The UNC Division of Radiation Therapy began in 1969 with the purchase of a Cobalt60 unit.
UNC Hospitals clinical medical dosimetry setting is recognized by the JRCERT. The education program has no external clinical sites (JRCERT Standard 3.1). The UNC Department of Radiation Oncology has the following student groups/education programs: 1) UNC Hospitals radiation therapy students, 2) UNC Hospitals medical dosimetry Hospitals.
2013-14 UNC Pembroke Utility Cost Distribution kWH 45.2% NG 11.9% 2Oil 2.0% 6Oil Propane 0.2% 0.1% Steam 22.1% CHW 9.4% Wtr Sewer 9.0% 2013-14 UNC System Summary Utility Cost Distribution Page 2 of 8. UNC Pembroke UNC Pemproke Btu by Fuel Type 2013-2014
