A Pilot Study On Pearson’s Interactive Science Program

preferred the new Pearson InteractiveScience program. In particular, manystudents said that, in comparison to theirother science program, the PIS program wasmore entertaining, did not includeextraneous content (i.e., “gets to the point”),was easy to understand and follow, providedmore examples, contained more labopportunities, was more interactive, and wasportable (lightweight). Moreover, whenasked what they liked best about theprogram, the most oft-cited features werethe writing opportunities and the labs. Teachers also clearly indicated preferencefor PIS. They felt that, compared to theirother science program, the PIS program waseasier to use, facilitated a sense of studentownership because students had their ownworktexts they could write in, had what theyneeded to effectively teach science, andpromoted better science understanding.Other general themes that emerged fromrespondents with regard to the variousfeatures of the program included: Teachers and students loved the abilityto write in the worktext -- the majority ofstudents indicated that they felt moreinvested in learning and learned more asa result of being able to interact and usetheir worktext in this manner.Overall, teachers who used the PearsonInteractive Science program noted thatthe program was easy to use.Teachers generally felt that pacing wasappropriate or easily modified.Teachers and students loved the layoutof the program (i.e., design andorganization).Teachers and students also thought theillustrations in the text were visuallyappealing and helped promote studentinterest and learning.Teachers and students found the labs funand engaging. They also felt that the labsrelated to the content in the book andhelped enhance comprehension. Thatsaid, some users indicated that thedirections were vague or confusing forsome of the labs.Although most teachers commented thatthe content and readability of theprogram was at-grade level (i.e., gearedtowards on-level students), there weresome comments that the program wastoo easy.Some recommendations noted bystudents and teachers included: (1)providing more challenging activities formore advanced students; (2) allow forgreater spaces for writing; and (3) makedirections for labs more student-friendly. How do teachers use the PearsonInteractive Science program in theirclassrooms? Do teachers of differingpedagogical approaches and philosophiesimplement the program differently acrossscience classrooms?Participating teachers closely followed theTeacher Edition (TE) and attempted the vastmajority of activities (including labs) thatwere noted in their TEs. That said, teachersreported that they used the DifferentiatedInstruction, 21st Century Skills, andRemediate sections of their TEs with lessfrequency and more on an as needed basis.With regards to labs, in most cases the timeit took them to complete the labs wasgreater, and for some significantly so, thanwhat was estimated. Indeed, this concurswith the feedback that was obtained fromteachers with regard to lab pacing. Teachersthought the time estimates were oftentimesunrealistic and that, as a result, gettingthrough lessons and the chapter was at timesdifficult and subsequently slowed downtheir pacing.Prepared by PRES Associates, Inc. – An Independent Evaluation Company6

Furthermore, teachers of differentpedagogies (inquiry and mixed instructionalapproach) did not demonstrate anydiscernible differences in the way theyimplemented their PIS lessons. That is,regardless of their instructional philosophy,teachers used the PIS program comparably.The information detailed in this report onhow teachers used the program will behelpful in the development and piloting ofimplementation guidelines to be used for thelarger scale pilot study in Spring 2010. How should the Pearson InteractiveScience program best be used in order tomaximize its impact on studentperformance?Based on information gathered during thepilot study, it is suggested that in the Spring2010 pilot study, teachers engage in thelesson plan as directed in the TE (e.g., Beinglessons with Planet Diary, point out anddiscuss the Big Question, etc.). However, itis also suggested that narrower guidelines beprovided to teachers with respect to labs sothat teachers are able to cover as many topicareas as they can while still providingengaging, hands-on lab opportunities. Inaddition, if differentiated instructionactivities/worksheets and 21st Century skillsections are deemed essential to studentscience performance, then there will need tobe greater emphasis on these activitiesduring the training and within theimplementation guidelines since theseactivities were not completed with muchfrequency when teachers were left to theirown devices. During the upcoming pilotstudy, researchers can more thoroughlyexamine the feasibility of draftimplementation guidelines which will beused and, to the extent variation occurs, lookat whether differences in how teachers usethe program influences teacher and studentoutcomes. What type(s) of training and preparationis needed in order to promote effectiveimplementation of the Pearson InteractiveScience program? Are the built-in teacherresources useful to teachers in helpingthem prepare to effectively deliver scienceinstruction in their classrooms?Since one exploratory question embedded inthis smaller pilot study was to determinehow teachers would typically use theprogram on their own (i.e., without highlydetailed implementation guidelines),trainings on the program was focused onproviding a general overview of the keycomponents and the theoretical basis behindthe program. Trainings did not providehighly detailed information on how toimplement the program nor did they convey“must use” components. That said, the pilotstudy provided information on how theprogram should ideally be used in futureresearch studies.With this in mind, it will be important forfuture studies of the PIS program to includea more structured, formal training so as toensure that teachers use the program in away that will optimally benefit students.Such training may include, for example, athorough discussion of the program’sresearch base and philosophy, and a detailedwalk through of all the teaching resourcesavailable as part of the Pearson InteractiveScience program, where to find them, andhow to use them (e.g., lesson modeling). Forthe Spring 2010 pilot, we can also examinein more detail the training model and itssufficiency to prepare teachers to hit theground running.Furthermore, teachers felt that they weresupplied with sufficient resources toeffectively implement the PIS program.They also reported that they were quicklyable to learn how to use the program, andPrepared by PRES Associates, Inc. – An Independent Evaluation Company7

the lessons and labs were noted as beingeasy to implement. Due to the more limitedamount of training provided to the NorthCarolina teachers, these teachers felt that thetraining was lacking as they were unable togo over all the materials in much detail. Thatsaid, they also indicated that the TE wasextremely helpful and facilitated theirimplementation of the science programthrough self-study. What preliminary impact does thePearson Interactive Science programhave on student science outcomes?It should be noted that the pilot study doesnot include a comparison group and wasonly conducted over a limited period oftime; therefore, while preliminary data onstudent learning gains and other data onparticipant perceptions and feedback wereobtained, the pilot study does not allow forstrong causal inferences to be made inregards to the effectiveness of the program –the 2010-11 RCT will be designed to dothis.With this in mind, results showed thatstudents demonstrated significant growthfrom pre- to post-testing on the Earthquakesand Populations & Communities chaptertests. No significant difference was observedon the Work & Machines chapter test. Thislack of significant difference may be due toseveral factors (and combinations thereof)such as: (1) a small sample was tested; (2) aWork & Machines teacher commented thatsome of the concepts covered in this chapterwere too advanced and that many of herstudents did not have the prerequisiteknowledge; and (3) teacher instructionalfactors. To the extent that this chapter willbe included in the larger-scale pilot study,researchers will examine if similar resultsare obtained and also gather more detailedinformation that can be used to inform thecontent and development of this chapter.Students also showed significant growth onthe Earthquakes and Populations &Communities tests as measured by thedifferent types of test items: multiple-choice,fill-in-the-blank, and constructed response.That is, students showed improvement on allthree different types of items. However,mixed findings were obtained on the Workand Machines test. While students showedsignificant improvement on the multiplechoice items, there was also a significantdecrease on the constructed-response items,and no significant change on the fill-in-theblank items.Furthermore, pilot students showedsignificant growth on the science chaptertests regardless of whether they were maleor female, minority or not a minority, andlow to high performing.Interestingly, results also showed that on theconstructed-response items, students ofteachers who employed an inquiry-basedapproach to instruction showed greatergrowth than students of teachers who used amore blended approach. Given observeddifferences in student test scores by teacherpedagogy, it will be important to measureteachers’ instructional approach in futureresearch studies to ascertain the extent towhich usage of the program and studentperformance is affected by this. Which type(s) of assessments andoutcome measures will be most sensitiveto picking up the effects of the PearsonInteractive Science program? What arethe psychometric properties of theinstruments used during the pilot study?Examination of difference scores by thevarious types of items included in thePrepared by PRES Associates, Inc. – An Independent Evaluation Company8

chapter tests showed that, with the exceptionof the Work & Machines test, students didequally well on the multiple-choice andconstructed response tests. That is, studentsgenerally improved at a similar rate.Furthermore, they showed the greatestimprovement on fill-in-the-blank test itemswhich tended to focus on vocabulary. Thissuggests that the program may be quitehelpful in improving students’ knowledge ofscience terminology.As previously noted, the PIS program alsoseems to impact other areas such as studentengagement and application of science. Itwill be important to continue to measurethese constructs during future researchstudies in order to more fully examine theimpacts of this program.In summary, the pilot study proceededsmoothly despite initial hurdles associated withobtaining necessary permissions for thevideotaping and student focus groups. Teacherscommented that they appreciated theopportunity to pilot the PIS program and asnoted earlier, strongly recommended theprogram. Moreover, the majority of studentsindicated that they wished they could usePearson Interactive Science as their core scienceprogram.Prepared by PRES Associates, Inc. – An Independent Evaluation Company9

Project Background"A decade ago, the Trends in InternationalMathematics and Science Study (TIMSS)benchmarking study showed that our bestdistricts could compete with anyone in theworld, but our worst districts—which, ofcourse, were in low-income communities—were on par with third-world countries. Thiskind of extreme inequity in education is notunique to science, but it has enormousrepercussions in the workforce wherescience-based industries are desperate forskilled workers You need to make inquirybased science relevant to kids—stimulatetheir curiosity—connect it with their lives.Together we need to change the nationaldialog about science—to prepare our kids tobe honestly critical and technicallycompetent.” (U.S. Secretary of EducationArne Duncan, March 2009)The world is being transformed due to everincreasing advances in technology and science.As a result, in order to succeed in futureeducational and career endeavors, a strongscience foundation is no longer a bonus, but anecessity. It is essential that our youth be able tothink critically and be provided with the toolsthey need to analyze both simple and complexsituations.Unfortunately, according to the NationalAssessment of Education Progress report(NAEP, 2005), 8th grade students in the U.S.showed no significant gains in scienceperformance from 1996 to 2005. Similarly,results from the Trends in InternationalMathematics and Science Study (TIMSS, 2009)showed that among 8th graders, there was alower percentage of U.S. students performing ator above the advanced benchmark in science in2007 than in 1999 (10 v. 12 percent). This iscause for concern, especially given the backdropof the No Child Left Behind Act (NCLB) of2001, which enacted the requirement that by the2007-2008 school year, states must administerscience assessments at least three times during astudent’s academic career. In an effort to furthersupport education, the Obama administrationrecently enacted the American Recovery andReinvestment Act (Feb. 2009), which includesthe 5 billion Race to the Top fund, awardingdistricts who make advanced reforms, includingin science education.Such emphasis on attainment of statestandards and performance on state assessmentshas highlighted a profound need to learn about“what works” in science education.Accordingly, educators are requiring evidencethat educational curricula produced bypublishers are based upon research-proveninterventions and have documented evidencethat they will produce a demonstrable, positiveimpact on student science achievement.“The preponderance of evidenceprovided by meta-analyses andevaluations of individual curricula seemto confirm that inquiry-based sciencecurricula produce larger effects onstudent achievement than do the more‘traditional’ science curricula (Clewell etal., 2004, p. 9).The 2011 Pearson Interactive Scienceprogram (PIS) incorporates prior research oneffective science instruction and seeks toimprove upon how science is being taught inclassrooms by: 1) embedding inquiry-basedteaching strategies into

Using Pearson Interactive Science. 23 Figure 5. Teacher and Student Perceptions of the Degree to which the Pearson Interactive . represents a model of inquiry-learning that lends itself to higher-order thinking skills and u