A Pilot Study On Pearson’s Interactive Science 2011 Program
Final ReportA Pilot Study onPearson’s InteractiveScience 2011 ProgramPrepared by:Danielle DuBose, Research AssociateMiriam Resendez, Senior ResearcherDr. Mariam Azin, PresidentSubmitted on August 5, 2010For inquiries, please contact PRES Associates at:info@presassociates.com(307) 733-3255 2010 PRES Associates, Inc.Prepared by PRES Associates, Inc. – An Independent Evaluation Company1
Table of ContentsExecutive Summary . 6Project Background . 11Project Overview. 13Methodology . 14Sites and Sample Characteristics . 16Results . 19What preliminary relationships are observed between use of the Pearson Interactive Science2011 program and key student and teacher outcomes? . 24What do users of the Pearson Interactive Science 2011 program think about the program? Whataspects of the program do they find most useful? Least useful? What, if any, suggestions forprogram improvement do they have? . 38How do teachers use the Pearson Interactive Science 2011 program in their classroom? Doteachers of differing pedagogical approaches and philosophies implement the programdifferently across science classrooms? . 47How should the Pearson Interactive Science 2011 program best be used in order to maximizeits impact on student performance? . 53What type(s) of training and preparation is needed in order to promote effective implementationof the Pearson Interactive Science 2011 program? Are the built-in teacher resources useful toteachers in helping them prepare to effectively deliver science instruction in their classroom?. 56Which types of assessments and outcome measures will be most sensitive to picking up theeffects of the Pearson Interactive Science 2011 program? What is the reliability, validity andsensitivity of data collection instruments used during the pilot study? . 59Conclusion . 60References. 61Appendix A: Tables of Statistical Results. 64Appendix B: Implementation Guidelines . 68Appendix C: Description of the Regular (non-PIS) Science Curriculum and Resources 74Prepared by PRES Associates, Inc. – An Independent Evaluation Company2
Table of Figures and TablesTablesTable 1.Table 2.Table 3.Table 4.Table 5.Table 6.Table 7.Table 8.Table 9.Table 10.Table 11.Table A1.Table A2.Table A3.Table A4.Table A5.Table A6.Table A7.Table A8.Table A9.Study Site School Level Student Demographics . 17Number of Teachers and Study Classes that Used the Ecology and the EnvironmentModule . 18Number of Teachers and Study Classes that Used the Earth’s Structure Module. 18Number of Teachers and Study Classes that Used the Forces and Energy Module .18Regular Science Program used by Participating Pilot Teachers.18Usefulness Ratings of the Pearson Interactive Science Program by Pilot Teachers.39Frequency of Use of Pearson Interactive Science Components . 49Average Number of Labs and Time to Complete Labs and Chapters During the 2009and 2010 Pilot Study .52Training Schedule . 55Psychometric Properties of the Pearson Interactive Science Custom Assessments . 58Correlation Between Pearson Interactive Science Custom Assessments and theTerraNova3, Science Assessments . 59Pre and Post Pearson Interactive Science Custom Test Statistics by Item TypeTableA2. PRE and Post TerraNova3 Test Statistics . 63PRE and Post TerraNova3 Test Statistics. 64Pre and Post TerraNova3: Content Specific Results. 64Frequency of use of Pearson Interactive Science Components Required byImplementation Guidelines as reported by Teacher Activity Logs – Earth’sStructure. 64Frequency of Use of Pearson Interactive Science Components Optional Activities byImplementation Guidelines as Reported by Teacher Activity Logs-Earth’s Structure .65Frequency of use of Pearson Interactive Science Components Required byImplementation Guidelines as reported by Teacher Activity Logs – Ecology and theEnvironment. 65Frequency of Use of Pearson Interactive Science Components Optional Activities byImplementation Guidelines as Reported by Teacher Activity Logs—Ecology and theEnvironment. 65Frequency of use of Pearson Interactive Science Components required byImplementation Guidelines as Reported by Teacher Activity Logs – Forces andEnergy . 65Frequency of Use of Pearson Interactive Science Components Optional Activities byImplementation Guidelines as Reported by Teacher Activity Logs-Forces andEnergy . 66FiguresFigure 1. Student Performance on the Pearson Interactive Science Assessment: Earth’sStructure. 23Figure 2. Student Performance on the Pearson Interactive Science Assessment: Ecology andthe Environment. 23Figure 3. Student Performance on the Pearson Interactive Science Assessment: Forces andEnergy . 23Prepared by PRES Associates, Inc. – An Independent Evaluation Company3
Figure 4. Student Performance on the Pearson Interactive Science Earth’s Structure Test byType of Test Items . 24Figure 5. Student Performance on the Pearson Interactive Science Ecology and theEnvironment Test by Type of Test Items . 24Figure 6. Student Performance on the Pearson Interactive Science Forces and Energy Test byType of Test Items . 25Figure 7. Overall Student Performance on the TerraNova3 Science Assessments.25Figure 8. Student Performance on the TerraNova3 Earth Science Content Area.26Figure: 9. Student Performance on the TerraNova3 Life Science Content Area. 26Figure 10. Student Performance on the TerraNova3 Physical Science Content Area .27Figure 11. Student Performance on the TerraNova3 Science Assessment: Level 16 (Grade 6) . 27Figure 12. Student Performance on the TerraNova3 Science Assessment: Level 17 (Grade 7) . 27Figure 13. Student Performance on the TerraNova3 Science Assessment: Level 18 (Grade 8) . 28Figure 14. Student Performance on the TerraNova3, Science Assessment: Level 16 (Grade 6)by Ability Level . 28Figure 15. Student Performance on the Pearson Interactive Science Assessment: Ecology andthe Environment by Ability Level of 6th Graders . 28Figure 16. Student Performance on the TerraNova3, Science Assessment: Level 17 (Grade 7)by Ability Level . 29Figure 17. Student Performance on the Pearson Interactive Science Assessment: Earth’sStructure by Ability Level of 7th Graders . 29Figure 18. Student Performance on the Pearson Interactive Science Assessment: Forces andEnergy by Ability Level of 8th Graders . 29Figure 19. Percent of Students who Agreed the Pearson Interactive Science Program HelpedThem Learn Science . 30Figure 20. Teacher Perceptions of the Degree to Which the Pearson Interactive ScienceProgram Helped Students’ Problem-Solving and Cognitive Skills . 30Figure 21. Teacher and Student perceptions of the Degree to Which They Were Engaged inScience While Using Pearson Interactive Science Relative to their Regular ScienceProgram. 31Figure 22. Percent of Teachers and Students Who Agreed that the Pearson Interactive ScienceProgram Affected Student Interest in Science . 32Figure 23. Percent of Teachers and Students Who Agreed the Pearson Interactive ScienceProgram Helped Students with Science Connections and Applications.33Figure 24. Percent of Teachers and Students Who Agreed the Pearson Interactive ScienceProgram Helped Students on Tests and Future Science Classes . 33Figure 25. Percent of Teachers Who Agreed the Pearson Interactive Science Program ProvidedUseful Information to Effectively Teach Science. 34Figure 26. Teacher Perceptions of the Degree to Which the Pearson Interactive ScienceProgram Helped Them With Their Science Instruction Relative to Their RegularScience Program . 34Figure 27. Average Level of Teacher Preparedness for Instructional Activities . 35Figure 28. Average Level of Teacher Preparedness for Inquiry-Related Instructional Activities. 35Figure 29. Teacher Perceptions of the Degree to which the Pearson Interactive Science ProgramHelped with Individualizing Instruction Relative to Other Science Programs. 36Figure 30. Percent of Teachers and Students Who Liked the Pearson Interactive ScienceProgram. 37Prepared by PRES Associates, Inc. – An Independent Evaluation Company4
Figure 31. Teacher and Student Ratings of Preference for Pearson Interactive Science Programvs. Regular Science Program . 38Figure 32. Average Descriptive Ratings by Students on the Pearson Interactive ScienceProgram. 38Figure 33. Teacher and Student Perceptions of the Degree to Which Pearson InteractiveScience Program Components are Better Relative to their Regular Science Program.39Figure 34. Percent of Students and Teachers Who Liked the Pearson Interactive ScienceWorktext . 40Figure 35. Percent of Students and Teachers Who Agreed That They Liked the Organizationof the Pearson Interactive Science Worktext . 42Figure 36. Percent of Teachers and Students Who Liked the Labs Activities in the PearsonInteractive Science Program . 43Figure 37. Teacher Ratings of the Pearson Interactive Science Labs by Type . 44Figure 38. Teacher and Student Perceptions of the Degree to Which Students Engaged inActivities While Using the Pearson Interactive Science Program. 46Figure 39. Teacher Perceptions of the Degree to Which Their Classroom Activities andPractices Were Effected by the Pearson Interactive Science Program .47Figure 40. Student Perceptions on the Degree to Which Teachers Engaged in VariousClassroom Activities While Using Pearson Interactive Science . 48Figure 41. Percentage of Teachers Who Assigned Supplemental Activities While UsingPearson Interactive Science . 50Figure 42. Teacher and Student Perception of the Degree to Which Inquiry Related ActivitiesOccurred While Using Pearson Interactive Science Relative to their Regular ScienceProgram. 51Figure 43. Percent of Teachers Who Felt Training and the Teacher’s Edition Were Helpful . 56Figure 44. Student Performance on the Pearson Interactive Science Assessments by Type ofTest Items. 58Prepared by PRES Associates, Inc. – An Independent Evaluation Company5
Executive SummaryPlanning, Research, and Evaluation Services(PRES) Associates, Inc. conducted a pilot studyon the new Pearson Interactive Science 2011program in the Spring, 2010. This pilot studywas designed to expand on the 2009 pilot studyby gathering quantitative and qualitative dataduring a more extended period of time1 toprovide a more comprehensive picture ofprogram implementation as well as preliminaryoutcome data on a diverse set of student andteacher outcomes. In addition, informationobtained from this pilot study is being used toinform the design of the more rigorousexperimental study on the effectiveness of thisprogram to be undertaken during the 2010-2011school year. Specifically, additional potentialoutcome measures, including customassessments were piloted to ensure thatassessments used during the 2010-2011 RCT aresensitive to the Pearson Interactive Scienceprogram.The pilot study consisted of 381 students ingrades 6-8 and 8 science teachers spread across3 middle schools in the states of Missouri, Utah,and Washington. Three Pearson InteractiveScience modules were piloted and consisted ofEarth’s Structure, Ecology and theEnvironment, and Forces and Energy. Whatfollows is a summary of the key findings fromthe study arranged by research questions.What preliminary relationships are observedbetween use of the Pearson Interactive Science2011 program and key student and teacheroutcomes? 1Students showed significant growth frompre- to post-testing on the Earth Structure,Ecology and the Environment, and Forcesand Energy custom assessments. GainsIn particular, while the 2009 teachers piloted only one PearsonInteractive Science chapter, the 2010 teachers piloted onemodule which contains 4-8 chapters.ranged from 8% to 19%. Students alsoshowed significant growth on the Earth’sStructure and Ecology and the Environmenttests across all the different types of testitems: multiple choice, fill-in-the-blank andconstructed response. However, mixedfindings were obtained on the Forces andEnergy test. Students who took the Forcesand Energy test showed significantimprovement on the multiple choice and fillin-the-blank items, but showed nosignificant change on constructed responseitems. These findings are somewhatconsistent with the findings of the Spring2009 pilot study and may suggest that thecontent of this module is more challengingthan the other modules used in the pilotstudy and that students experienced greaterdifficulties communicating and applyingconcepts related to Physical Science. Performance on the TerraNova3, a nationalnorm-referenced exam, was also examined.It should be noted that substantial gainswere not expected since TerraNova3assessment contains items across all contentareas of science, many of which were notcovered within the modules used during thestudy. With this caveat in mind, resultsindicate that across all grades, studentsshowed a marginally significant learninggain from pre to post testing, p .10. Inaddition, while the percentile gain is small at.2%, it is noteworthy when one considersthat it is a general rule of thumb that if astudent makes a year’s growth for a year ofinstruction, then the percentile rank willremain the same. To more closely examinethe relationship between TerraNova3student science performance and the PearsonInteractive Science program, researchersalso analyzed separately TerraNova3questions that were more directly aligned tothe specific science areas taught during thestudy. Results showed significant gains forlife and earth science students. NoPrepared by PRES Associates, Inc. – An Independent Evaluation Company6
impacted their reading and writing skills as aresult of the numerous opportunities forreading and writing offered within theworktext.significant gains were observed for physicalscience students. Subgroup analysis by ability level showedthat 6th graders of all ability levelsdemonstrated significant improvement onboth the custom assessment andTerraNova3. Among 7th grade students,average and high ability students showedsignificant growth on the custom assessmentand average level students showedsignificant growth on the TerraNova3.While 8th grade students who were ofaverage and high ability showed significantimprovement on the custom assessment, nosignificant differences were observed on theTerraNova3. Self-reported learning outcomes were alsoexamined during the pilot study. Themajority of students reported that thePearson Interactive Science worktextprovided them with useful information tolearn and understand science. In general,teachers also felt that the program helpedstudents with their problem-solving andother higher order cognitive skills.With respect to student engagement andinterest in science, students noted that theinteractive aspects of their worktext alongwith the integrated lab activities helpedengage them in the learning process.Teachers also reported a higher level ofstudent engagement while using the PearsonInteractive Science program as compared towhen they used their regular scienceprogram. Students self-reported only aslightly greater level of engagement.All teachers and the vast majority ofstudents felt that the Pearson InteractiveScience program helped students makeconnections between science, real worldapplications, and other subject areas. Theyalso felt that the program positivelyOverwhelmingly, teachers and studentsagreed that the Pearson Interactive Sciencestudent worktext prepared students to dowell in state/national tests and future sciencecourses.Furthermore, teachers indicated that the PISprogram was more helpful than their regularscience program in providing good ideas forhands-on science activities, helping to teachscience vocabulary, providing them withresources, and minimizing the preparationand planning time needed to prepare forlessons. Teachers also reported gains in theirlevels of preparedness to: a) deliver effectivescience instruction; b) make connectionsbetween science and other disciplines; c)help students communicate skills related toscience; and d) develop student problemsolving skills. Additionally, teachers weresignificantly more prepared to teach sciencethrough a hands-on instructional approachand manage a class of students using handson or laboratory activities.What do users of the Pearson InteractiveScience 2011 program think about theprogram? What aspects of the program do theyfind most useful? Least useful? What, if any,suggestions for program improvement do theyhave? The majority of teachers and studentsenjoyed using the Pearson InteractiveScience program and would like to use theprogram during the following school year.Students and teachers felt the program waseasy to understand, engaging and wellorganized.Prepared by PRES Associates, Inc. – An Independent Evaluation Company7
When asked to directly compare the PearsonInteractive Science program with theirregular science program, teachers rated thePIS program more favorably. Students,however, rated both science programssimilarly overall. However, when askedabout specific components of curricula,students and teachers rated the overallpresentation of the student worktext andease of use as better than their regularprogram. Teachers also rated the PearsonScience Interactive program more favorablythan their regular science program withrespect to: 1) math activities, 2) format ofthe student book, 3) writing opportunities inthe worktext, 4) how science is explained, 5)science labs, and 6) types of exercises andquestions in the program.How do teachers use the Pearson InteractiveScience 2011 program in their classroom? In general, teachers commented that theythought that all pedagogical components ofthe Pearson Interactive Science programwere useful. Teachers especially liked theFigures/Activity Art/Animations, “Do theMath”, Big Questions, “Apply It!”, and“Explore the Big Question.”With respect to the worktext specifically, themajority of students loved that they couldwrite in their worktext, that they “owned”the book, and could keep all their notes inone place. They also liked the portability ofthe worktext. However, about 33% ofstudents felt indifferent towards theworktext.Teachers and students liked the labs andinvestigations they used from the PearsonInteractive Science program and felt the labzones kept students interested in scienceclass. When asked to rate each of the varioustypes of labs, teachers rated the InquiryWarm-Up Labs the most favorably followedby the Quick Labs and Lab Investigations. Participating teachers did well in followingthe implementation guidelines and Teacher’sEdition (TE). The only required activity thatwas not used as directed by theimplementation guidelines was the MyPlanet Diary feature. In addition, whilemany teachers reported using supplementalactivities, they were mostly digitalresources, computer programs and teacherdeveloped labs and worksheet.When asked to compare their engagement invarious instructional activities during theFall semester (when they used their regularscience program) versus the Spring semester(when they used the PIS program), bothstudents and teachers reported that studentsanswered textbook or worksheet questions,used mathematics as a tool with scienceproblems, and explained in writing theiranswers to questions in science class moreoften while using the Pearson InteractiveScience program as compared to when theyused their regular science program. Teachersreported that they introduced new sciencetopics by exploring Big Questions andassigning a hands-on lab activity more oftenwhile using the Pearson Interactive Scienceprogram as compared to when they usedtheir regular science program. In addition,teachers felt that they were better able toassess their students’ level of understandingduring and after the lesson while using thePIS program.For the most part, it was feasible forstudents to complete one Inquiry Warm Upand Quick Lab per lesson and one LabInvestigation per chapter. Teachers alsoreported that the lab time estimates werefairly accurate. However, similar to the 2009pilot, some teachers noted that it did notPrepared by PRES Associates, Inc. – An Independent Evaluation Company8
designed and organized – teachers noted thatit was easy to use, including the labs.include time for teacher preparation whichcould be considerable.How should the Pearson Interactive Science2011 program best be used in order tomaximize its impact on student performance? Based on the information gathered duringthe two pilot studies, it is recommended thatteachers use the program as outlined in the2010 pilot study implementation guidelines.As noted, it was feasible for students tocomplete one Inquiry Warm Up and QuickLab per lesson and one Lab Investigationper chapter. In addition, most teachers wereable to complete the required components ofthe program as outlined in the guidelineswhile maintaining a reasonable pace.Moreover, for the most part pilot teachersagreed that the key components noted in theguidelines were essential for studentunderstanding. The only exceptions wereuse of the My Planet Diary, Assess YourUnderstanding and the Lab Investigations.Nevertheless, these components areconsidered critical by the authors as theyengage, evaluate, and allow students toexplore – all aspects of the research-based5Es pedagogy.What type(s) of training and preparation isneeded in order to promote effectiveimplementation of the Pearson InteractiveScience 2011 program? Are the built-inteacher resources useful to teachers in helpingthem prepare to effectively deliver scienceinstruction in their classroom? Teachers commented that there weresufficient resources to effectively implementthe Pearson Interactive Science program.They also reported that they enjoyed thetraining and were adequately prepared to usethe Pearson Interactive Science program.Their preparedness to use the program wasalso facilitated by how the program was Recommendations for future trainingsinclude: (1) develop and employ a trainingmodel to promote consistency in trainings;(2) begin with the program’s research baseand philosophy as it would be helpful forteachers to fully understand the backgroundrelated to why/how Pearson created thisprogram to improve student science skills;(3) provide a general overview of all theteaching resources available as part of thePearson Interactive Science program, whereto find them and how to use them, includinga modeling of the lesson; (4) scheduletrainings approximately 1-2 weeks prior toimplementation so that informationconveyed is fresh in their minds; and (5)provide additional training on the digitalpath that will focus on detailed informationon what is available and how to incorporatethe technology into their activities.Which types of assessments and outcomemeasures will be most sensitive to picking upthe effects of the Pearson Interactive Science2011 program? What is the reliability, validityand sensitivity of data collection instrumentsused during the pilot study? On the custom assessments students didequally well on multiple choice andconstructed response test items.Furthermore, they showed the greatestimprovement on the fill-in-the-blank testitems which for the most part focused onvocabulary words. This finding is consistentwith the findings of the small-scale pilotconducted in Spring 2009 in which studentsshowed the greatest improvement on the fillin-the-blank test items. In addition, and asexpected, the custom assessments were alsomore sensitive to the measurement ofstudent growth than the TerraNova3. Thatsaid, the TerraNova3 is important to includePrepared by PRES Associates, Inc. – An Independent Evaluation Company9
in the RCT as it will provide normative data(e.g., percentile rankings) on studentperformance so that comparisons can bemade to a national sample. The custom tests used in the pilot studyshowed high levels of internal consistency(Chronbach’s alpha) and split halfreliability, indicating they arepsychometrically sound. Furthermore, thecustom assessments were correlated with theTerraNova3 assessments to obtaininformation on the concurrent validity of thecustom assessment. The obtainedcorrelations are adequate and suggestive ofconcurrent validity.In sum, researchers were able to obtainadditional information that suggests that thePearson Interactive Science program isassociated with positive student outcomes.However, these findings are only preliminaryand not conclusive. That said, the upcoming fullyear randomized control trial has been designedto produce rigorous quantitative evidence uponwhich strong conclus
Pearson’s Interactive Science 2011 Program Final Report Prepared by: Danielle DuBose, Research Associate Miriam Resendez, Senior Researcher . Student Performance on the TerraNova3 Science Assessment: Level 18 (Grade 8) .28 Figure 14. Student Performance on the TerraNova3, Science
Pearson Education LTD. Pearson Education Australia PTY, Limited. Pearson Education Singapore, Pte. Ltd. Pearson Education North Asia, Ltd. Pearson Education Canada, Ltd. Pearson Educación de Mexico, S.A. de C.V. Pearson Education—Japan Pearson Education Malaysia, Pte. Ltd. The Libra
Pearson Education LTD. Pearson Education Australia PTY, Limited. Pearson Education Singapore, Pte. Ltd. Pearson Education North Asia, Ltd. Pearson Education Canada, Ltd. Pearson Educatión de Mexico, S.A. de C.V. Pearson Education—Japan Pearson Education Malaysia, Pte. Ltd. Library of Co
Pearson (UK) 80 Strand, London WC2R 0RL, UK T 44 (0)20 7010 2000 F 44 (0)20 7010 6060 firstname.lastname@pearson.com www.pearson.com Pearson (US) 1330 Avenue of the Americas, New York City, NY 10019, USA T 1 212 641 2400 F 1 212 641 2500 firstname.lastname@pearson-inc.com www.pearson.com Pearson Education One Lake Street, Upper Saddle River,
Forecast Pilot Supply & Demand. 26 UND U.S. Airline Pilot Supply Forecast (2016) predicts cumulative pilot shortage of 14,000 by 2026. Boeing Pilot Outlook (2017) projects worldwide growth in pilot demand, with 117,000 pilots needed in North America by 2036. CAE Airline Pilot Demand Outlook (2017) indicates 85,000 new
Pearson BTEC Level 4 HNC The Pearson BTEC Level 4 HNC in Business is a qualification with a minimum of 120 credits of which 60 are mandatory core. The Pearson BTEC Level 4 HNC programme must contain a minimum of 65 credits at level 4. Pearson BTEC Level 5 HND The Pearson BTEC Lev
Data sheet Pilot-operated servo valve, type ICS Danfoss DCS MWA) 2016.01 DKRCI.PD.HS2.A9.22 520H8639 5 Function ICS 1 Pilot The ICS main valve is a pilot operated valve. The types of pilot valves used determine the function. The ICS main valve with pilot valve(s) controls refrigerant flow by modulation or on/off in
Data sheet Pilot valves for pilot operated servo valves Danfoss DCS (MA) 2016.04 DKRCI.PD.HN0.B7.02 520H0831 6 Differential-pressure pilot valve, type CVPP (LP) and CVPP (HP) CVPP is a differential-pressure pilot valve
ICS 1 Pilot The ICS main valve is a pilot operated valve. The types of pilot valves used determine the function. The ICS main valve with pilot valve(s) controls refrigerant flow by modulation or on/off in accordance with the pilot valve and main valve status. The manual spindle can be used to open the valve plate.
