COMPASS - Files.eric.ed.gov
COMPASSCollaborative Organizational Model to Promote Aligned SupportStructuresFinal Evaluation ReportUS Department of EducationInvesting in Innovation (i3) FundSubmitted by Lisa Holliday and Joel PhilptoThe Iredell-Statesville School DistrictIredell, NCOctober, 2015
COMPASSCollaborative Organizational Model to Promote AlignedSupport StructuresFinal Evaluation ReportLead Agency:Iredell-Statesville School District410 Garfield St,Statesville NC 28677Project Directors:Sherrard Martin and Brie BeaneThird-Party Evaluators:Contact information:Lisa Holliday, Ph.D. (Implementation Evaluation)Joel Philp, Ph.D. (Impact Evaluation)The Evaluation Group Martin:704-832-2549; sherrard martin@iss.k12.nc.us Beane: 704-832-2524; brianne beane@iss.k12.nc.us Holliday: 704-996-5291; lisa@evaluationgroup.com Philp: 803-719-5102; joel@evaluationgroup.comGrant Period:Funder:PR Award #i3 Cohort Year:i3 ID01/01/2010 – 01/01/2015US Department of Education, Office of Innovation and Improvement (OII),Investing in Innovation (i3) FundU396C1001052010DEV19Funding for this report came from the U.S. Department of Education under its Investing in Innovation (i3) initiativewithin the Office of Innovation and Improvement through Grant U396C100105 to the Iredell-Statesville SchoolDistrict. The i3 grant award required an independent evaluation of the implementation and impacts of the program.The opinions, findings and conclusions in this report do not necessarily represent the official positions or policies ofthe funders. Correspondence regarding this document should be addressed to Lisa Holliday (ImplementationEvaluation) or Joel Philp (Impact Evaluation), The Evaluation Group, 169 Laurelhurst Ave., Columbia SC 29206.Phone: (803) 719-5102; email Joel@evaluationgroup.com. For information about The Evaluation Group, see ourwebsite: www.evaluationgroup.com. Copyright 2015 by TEG . All rights reserved.COMPASS Final Report, page 1
Table of ContentsList of Tables and FiguresExecutive SummaryAcknowledgments1.Implementation Evaluation . 61.1Program Description . 61.2Deployment . 91.3Fidelity of Implementation . 91.3.11.4Method . 101.4.1Selection Criteria . 101.4.2Fidelity Metric . 101.4.3Critical Component Scores . 111.52.Critical Components . 9Overall Findings . 121.5.1Implementation of the Educative Component . 131.5.2Implementation of Procedural/Pedagogical Component. 13Impact Evaluation . 142.1Counterfactual . 142.2Research questions . 142.2.1Confirmatory Research Question: . 142.2.2Exploratory Research Questions: . 142.3Design . 152.3.1Sample Eligibility . 152.3.2Sample Selection . 152.4Measure . 182.4.1Conversion of School-level Standard Scores to Z-Scores . 182.4.2Data Collection . 192.5Analysis . 192.5.1Baseline Sample Size . 202.5.2Analytic Sample Size . 202.5.3Matching Blocks as Covariates . 202.5.4Analytic Approach . 202.6Findings . 222.6.1Baseline Equivalence Testing . 222.6.2Baseline Equivalence Results . 232.6.3Impact Results. 232.7Discussion . 24COMPASS Final Report, page 2
List of Tables and FiguresFigure 1. COMPASS Logic Model . 8Figure 2: Percentage of COMPASS Schools Meeting Fidelity Targets, by Year. . 12Table 1: COMPASS Critical Components and Indicators . 10Table 2: Fidelity Metric . 11Table 3: Critical Components Measured, Year by Phase . 11Table 4: Program Fidelity Thresholds . 11Table 5: School-Level Educational and Procedural/Pedagogical Scores, by Year. . 12Table 6: COMPASS, ERPD, and RtI Training Attendance in Years 2-4, by Phase. . 13Table 7: Leadership Team and PLC Meeting Minutes Noting Support, by Phase. . 13Table 8: Support Requests, Years 3-4, by Phase. . 13Table 9: Propensity Scores on the 21 COMPASS Schools . 16Table 11: Matching by Implementation Phase. . 18Table 12: Treatment Years and Pre-treatment Years COMPASS and Comparison Schools . 19Table 13: Characteristics of the Sample Schools at Baseline. . 20Table 14: Baseline Equivalence of the COMPASS and Comparison Schools, by Contrast. . 23Table 15: Summary of Impact Results, by Contrast. . 23Table 16: Difference in Scaled Scores in COMPASS Schools v Comparison Schools, by Contrast. . 24COMPASS Final Report, page 3
Executive SummaryIn 2010, Iredell-Statesville Schools was awarded an Investing in Innovation grant (i3) from the Office ofInnovation and Improvement within the Federal Department of Education. Collaborative OrganizationalModel to Promote Aligned Support Structures (COMPASS) is a development grant that seeks to meet theneeds of students with disabilities, academically struggling high-needs students, and students with limitedEnglish proficiency by providing timely and targeted professional development to teachers through thealignment of support structures.The Evaluation Group conducted a fidelity of implementation (FOI) study to determine the extent to whichthe program was delivered as intended. We also conducted an impact study to determine the effect ofthe program on reading achievement in grades 3-8.The implementation evaluation assessed both educative and procedural components. Educativecomponents included delivery and attendance at early release professional development sessions,Response to Intervention (RtI) sessions, and specialized COMPASS trainings to targeted support staff.Assessing the fidelity of the procedural/pedagogical components included evaluating the alignment of theprogram with district leadership teams and professional learning communities (PLCs). COMPASS waswell- implemented. The district met its fidelity targets for all years of the evaluation. Fidelity scores rangedfrom 86%-100%.COMPASS was found to have a positive impact overall and at some but not all grade levels. The impactstudy used a short interrupted time-series with comparison group design (C-SITS) to examine the effectof COMPASS on school-level standardized test scores.Reading outcomes were measured at theschool level before and after implementation of the program in COMPASS schools, and at the same timepoints in comparison schools. Effects were assessed for grades 3-8 combined, and for each of grades 3through 8 separately using two-level and three-level hierarchical linear modelling. School-level ELAscaled scores were converted to Z-scores with a mean of 0 and a sd of 1 to ensure comparability acrossgrades and years. The effect of COMPASS, combined across grades 3-8, produced a significant impactestimate of 0.39, which translates into a gain of almost 4 scale score points over schools in thecomparison group. We also found positive and statistically significant impacts within grade 4, grade 5,and grade 7, with estimates ranging from 0.42 (grade 4) to 0.64 (grade 5). Impact estimates for grade 3and grade 6 were in the intended direction but did not reach statistical significance. The impact estimatefor the eighth grade was in the negative direction (that is, COMPASS achievement was lower thancomparison schools) but the estimate of -.17, was not statistically different from zero. It should be notedthat this is school-level data and the impact at the individual student level is likely to be somewhatsmaller.Because all COMPASS schools reside in the same district, this study suffers from an N 1 confound. Thisoccurs when a program can only be implemented in one classroom (or one school or one district) and theeffects cannot be disentangled from other factors that may be operating within that classroom (or schoolor district). This opens up the possibility that the change in test scores may be due to other influenceswithin Iredell-Statesville Schools, including other interventions being implemented, and not solely a resultof the effects of COMPASS.However, due to the high degree of fidelity ( 80%) maintained by the district in all years ofimplementation, it is reasonable (but certainly not definitive) to conclude that COMPASS is a validexplanation for the improvement in test scores. COMPASS had an overall effect (grades 3-8) and aneffect in grades 4, 5, and 7.COMPASS Final Report, page 4
AcknowledgmentsA number of people have made substantial contributions to this report. At the Iredell-Statesville SchoolDistrict, we wish to thank Sherrard Martin and Brie Beane, COMPASS Project Directors whoenthusiastically offered their cooperation and assistance throughout the process; Laura Elliot, Director ofTesting, for her assistance with identifying data sources and providing timely information; and MelanieTaylor, Deputy Superintendent of Curriculum and Instruction, for providing leadership and oversight tothe entire project. At Abt, the Nei3 Technical Assistance providers, we would also like to thank our TAliaison, Linda Caswell, for her unwavering support, insights, and acumen throughout the evaluation,particularly as it relates to fidelity of implementation. We also wish to express our deep gratitude to ChrisPrice for his patience and gentle guidance in helping us to navigate the complexities of the comparativeinterrupted times series design and the associated impact analysis.COMPASS Final Report, page 5
1. Implementation Evaluation1.1Program DescriptionIn 2010, Iredell-Statesville Schools was awarded an Investing in Innovation grant (i3). The CollaborativeOrganizational Model to Promote Aligned Support Structures (COMPASS) is a development grant thatseeks to meet the needs of students with disabilities, academically struggling high-needs students, andstudents with limited English proficiency by providing timely and targeted professional development to theteachers through the alignment of support structures.The long-term goal of COMPASS is to increase the academic achievement of all students with a focus onstudents with high-needs, disabilities, and limited English proficiency. To this end, COMPASS providestraining to school-based support staff, their Executive Directors, and teachers. By increasing theexpertise of support structure staff, COMPASS aims to provide higher quality support to teachers. In turn,this will increase the knowledge, skills, and performance of teachers, which will improve studentperformance.In the COMPASS model, presented in Figure 1, alignment is the process of bringing together supportstructure staff through training and the creation of an online request system through which support staffassistance can be requested. Support structures provide appropriate professional development toteachers as they work to improve the academic achievement of all students, and specifically thosestudents who are struggling or are at-risk for academic failure.In order to align the support structures, the COMPASS management team developed a series ofprofessional development sessions that target the school-level support structures, their ExecutiveDirectors, and faculty. These sessions focused on areas identified by district leadership, including theCommon Core, Positive Behavioral Support, SIOP, RtI, AIMSweb, and Progress Monitoring.Training was offered through several media. North Carolina requires that schools hold six Early ReleaseProfessional Development (ERPD) days per year. During this time, the COMPASS management providedprofessional development. During Year Two, there were a series of COMPASS training sessions offered.These were targeted at support staff and designed to provide them with detailed information on thefollowing content areas: Positive Behavioral Support Overview, SIOP/ESL, RtI Overview, AIMSweb,Curriculum Based Measures, Interventions, and Progress Monitoring. Lastly, in Years Two and Three, thestate of North Carolina offered five eight-hour training sessions on Responsiveness to Intervention (RtI).The COMPASS management team added district-specific RtI training beginning in Years 4-5, i3 C3 RtISuccess! Workshops. These trainings provided a refresher on information covered by state training.After the support staff had been trained, they then began to provide support to teachers. This includedleading COMPASS training sessions at schools, facilitating ERPD training, and providing individualtraining as requested. Additionally, support staff participated with the COMPASS management team tooffer district level training to faculty and principles on topics such as RtI.In order to streamline the support request process, the COMPASS management team created an on-linesupport request system, which was utilized beginning in Year Three. This system allowed school levelsupport staff and principals to place requests for support. These requests could be for instructionalsupport, content area support, or behavioral support. School level support staff could request thatmembers of the COMPASS management team come to their school and provide assistance with dataanalysis, model teaching, or other support as needed. Principals could schedule whole level schoolprofessional development for members of the COMPASS management team or individual faculty support.COMPASS Final Report, page 6
Each school utilized six types of data in order to determine if support was needed:1. End of Grade and End of Course testing. Test results are reviewed at the beginning of the schoolyear by teachers, PLCs, and principals. Scores below a three indicate that the student is eligiblefor RtI intervention, and that instructional support may be needed.2. Pre- and post- benchmark assessments. Results are reviewed at least twice per year by PLCs,thprincipals, and IFs. Scores below the 75 percentile on AIMSweb testing indicate a need for RtIintervention and show that instructional support may be needed.3. Formative assessments. These are reviewed at least every 4.5 weeks by PLCs and IFs.Adequate performance scores will be established by the PLCs. Failure to make adequateperformance indicates a need for support staff help.4. RtI assessments. These are reviewed at least every 15 days by the RtI team. Adequateperformance standards will vary by students and treatment plan. Failure to make adequateperformance indicates a need for additional RtI intervention services and that instructionalsupport may be needed.5. Classroom Walk-Throughs. These are conducted periodically by principals.6. Teacher observations of students while teaching.A need for content area or behavior support was indicated after a review by PLCs of items 1, 2, 3, 5, and16 above. If the data indicated that students were struggling, PLCs contacted their school’s InstructionalFacilitator (IF) to request additional support. If within the IF’s scope of expertise, the IF provided support.If not, the IF contacted the appropriate COMPASS support personnel and arranged for him/her to meetwith the PLC.At least twice per month, each school’s Leadership Team met to review items 1, 2, 5, and 6. They alsodetermined if a school or individual faculty member was in need of instructional support. After reviewingthe data, the Leadership Team completed a Strategic Curriculum and Instruction (CI) form, which outlineda request for support for teachers who were working with struggling students. In order to complete theform, the Leadership Team did the following: Identify which content area will be addressed and which support structures are needed tosupport the content area;Determine what type of support is needed (coaching, training, etc.) and will be provided;Determine the support delivery method (coaching, courses, training, etc.);Determine what, if any, additional professional develo
school level before and after implementation of the program in COMPASS schools, and at the same time points in comparison schools. Effects were a
Eric Clapton Journeyman Eric Clapton Me & Mr. Johnson Eric Clapton One More Car, One Mor Eric Clapton Pilgrim Eric Clapton Reptile Eric Clapton Sessions for Robert J [C Eric Clapton Unplugged Eric Clapton Riding with the King Eric Clapton & B.B. King At Last! Etta James Eurythmics : Greatest Hits Eurythmics American Tune Eva Cassidy Eva .
compass reading (a) Fe-rich rock bodies can only use sun compass C. Using clinometer 1. align vertical edge of compass with angle of plane 2. adjust bubble level of clinometer 3. read angle from vernier scale on compass D. Computing vertical elevation 1. measure eye height from ground surface (E.H.) 2. sight compass to top of object (e.g. top .
COMPASS GROUP COURSE CATALOG 3 Associate Brand Guidelines Compass Group North America Version 1.0 June 2011 Success: it’s embedded in our Compass Group . culture. Create your own sucess story at Compass Group. Introduction . COMPASS GROUP. is committed to enhancing the skills, knowledge, and
Introduction to the Brunton Compass Geo420k, Lab 1. M. Helper, Jackson School of Geosciences, UT Austin Mirror Lift Pin for Needle Compass Card Sighting Arm Parts of the Brunton Compass . Compass must be horizontal (bull’s eye bubble centered), with compass edge flush to the tilted plane Strike.
2236E 1--1 1 Description The Reflector Compass Equipment is a magnetic standard compass, class A. A floating magnetic compass (1--1.1) is gimbal--mounted in a compass binnacle made of glass fibre reinforced plastic (1--1.3). The helmsman is provided with a sector of the magnetic compass ca
3. Compass May Obtain My Personal Information – Compass has my consent to request personal information about me from sources that Compass considers relevant to the services I have asked Compass to provide. Depending on what I have asked Compass to help me with,
COMPASS SURVEYING 1. Familiarity with instruments used in compass surveying - prismatic compass. 2. Setting up the compass - observation of bearings. 3. Traversing with prismatic compass and chain calculation of included angles and check. 4. Traversing with prismatic compass and chain closed traverse covering the given area recording. 5.
Accessing Compass Compass is a web-based system that is accessible on any modern web browser (Internet Explorer, Firefox, Chrome, Safari) or by using the Compass iOS or Android apps. Search for 'Compass School Manager' in the store. Every family receives a separate login to Compass, which will be provided to you by Murrumba State Secondary
