Make An Impact: Using Data Dashboards To Tell Your PBIS Story

APBS 2020, Miami ‐ Session F103/13/2020Make an Impact: Using DataDashboards to Tell Your PBIS StoryKaren Elfner, M.A.Nichole Fintel, MSPH3/13/20This product was developed by the Florida PositiveBehavioral Interventions and Support Project, a projectfunded by the State of Florida, Department of Education, K‐12 Public Schools, Bureau of Exceptional Education andStudent Services, through federal assistance under theIndividuals with Disabilities Education Act (IDEA), Part B.1FLPBIS:MTSS Project - Supporting Florida’sSchool Districts with PBIS ImplementationWe CoachDistrict LeadersWho CoachSchool-Based LeadersWho CoachTeachersWhat We DoWho CoachStudents & FamiliesWho inform culturally responsivepractices to address local needs.2K. Elfner & N. Fintel1

APBS 2020, Miami ‐ Session F103/13/2020The Role of Data In PBISOUTCOMESPRACTICESSource: Center on PBSIS3The Role of Data In PBISOUTCOMESPRACTICESSource: Center on PBSIS4K. Elfner & N. Fintel2

APBS 2020, Miami ‐ Session F103/13/2020Data-Based Decision MakingData is used to makedecisions at:–Every tier of support–Every step of theProblem‐Solving ProcessDefineAnalyzeWhat’s the problem?Why is it occurring?EvaluateImplementIs it working?What are we going to do?5Data-Based Decision MakingData is used to makedecisions at:–Every tier of support–Every step of theProblem‐Solving ProcessDefineAnalyzeWhat’s the problem?Why is it occurring?EvaluateImplementIs it working?What are we going to do?6K. Elfner & N. Fintel3

APBS 2020, Miami ‐ Session F103/13/2020FLPBIS Project’s Evaluation Data SourcesPBIS Implementation Fidelity– PBIS Implementation Checklist (PIC)– Benchmarks of Quality (BoQ)– Tiered Fidelity Inventory (TFI)– PBIS Tier 1 WalkthroughOutcome Data Summary with Equity Report– Disaggregated student discipline outcomes– Average daily attendance– % with chronic absences7FLPBIS Project’s Evaluation TimelineSEPTOCTNOVDECJANFEBMARAPRMAYJUNJULAUG8K. Elfner & N. Fintel4

APBS 2020, Miami ‐ Session F103/13/2020FLPBIS Project’s Evaluation TimelineMid-Year 1SEPTOCTNOVDECJANFEBMARAPRMAYJUNJULAUGFall PIC9FLPBIS Project’s Evaluation TimelineMid-Year2Mid-Year 1SEPTOCTNOVFall PICDECJANFEBMARAPRMAYJUNJULAUGSpringPICPBIS Tier 1 Walkthrough10K. Elfner & N. Fintel5

APBS 2020, Miami ‐ Session F103/13/2020FLPBIS Project’s Evaluation TimelineMid-Year2Mid-Year pringPICFall PICTFI (T2 & T3)Demographic DataPBIS Tier 1 WalkthroughOutcomeData11FLPBIS Project’s Evaluation TimelineMid-Year2Mid-Year 1SEPTOCTNOVDECJANFEBModel SchoolsApplication WindowEnd-YearMARAPRMAYJUNJULAUGBoQFall PICSpringPICTFIDemographic DataPBIS Tier 1 WalkthroughOutcomeData12K. Elfner & N. Fintel6

APBS 2020, Miami ‐ Session F103/13/2020FLPBIS Project’s Evaluation TimelineMid-Year2Mid-Year 1SEPTOCTNOVDECJANFEBModel SchoolsApplication WindowEnd-YearMARAPRMAYJUNJULAUGBoQFall PICSpringPICTFIDemographic DataPBIS Tier 1 WalkthroughOutcomeData13Collecting data alone is not enough!14K. Elfner & N. Fintel7

APBS 2020, Miami ‐ Session F103/13/202015Creating a System to Make Data Available,Interconnected, and Usable to ImplementersRelevant data are AvailableLinked toActionPlanningEasy toUnderstandEffective Data System16K. Elfner & N. Fintel8

APBS 2020, Miami ‐ Session F10Available3/13/2020Linked toActionPlanningEasy toUnderstandCan PBIS implementation decision‐makers atyour school/district access timely data?17AvailableLinked toActionPlanningEasy toUnderstandDo these data link to actionable insights that inform PBISimplementation?– Is your PBIS system being implemented as intended (fidelity)? Which components are not working? Why?– Is your PBIS system working for all students? Which student outcomes are not being impacted?Which students are you missing in your PBIS system?18K. Elfner & N. Fintel9

APBS 2020, Miami ‐ Session F10Available3/13/2020Linked toActionPlanningEasy toUnderstandCan PBIS implementation decision‐makersunderstand the data being shared?– Are data transformed into visualizations that makeresults/trends easy to spot and digest?19Evolution of FLPBIS’ Data Reports:Excel Spreadsheets20K. Elfner & N. Fintel10

APBS 2020, Miami ‐ Session F103/13/2020Evolution of FLPBIS’ Data Reports:Web-based Programmed Static Graphs & Charts2122K. Elfner & N. Fintel11

APBS 2020, Miami ‐ Session F103/13/2020Evolution of FLPBIS’ Data Reports:AvailableEasy toUnderstandLinked toActionPlanningInteractive Data DashboardsBased on PBIS ImplementationEarly Warning Indicators23FLPBIS Data in24K. Elfner & N. Fintel12

APBS 2020, Miami ‐ Session F103/13/2020If you have data available or if you were creating a dashboard,what kinds of questions would you want the dashboard toanswer? (At a school‐ or district‐level)School‐Level Questions: Pink CardDistrict‐Level Questions: Green Card25District Storytelling ExampleSource: ‐facts‐2/#6208225a3a7c26K. Elfner & N. Fintel13

APBS 2020, Miami ‐ Session F103/13/2020Once Upon a Time There was a school district 73,000 students83 total schools43 Active PBIS schools10 years engaged in PBIS27That engaged in PBIS Trained school leadership team on PBIS Provided ongoing Professional Development Encouraged schools to submit and use datafor continuous improvement28K. Elfner & N. Fintel14

APBS 2020, Miami ‐ Session F103/13/202029District-Level Data DashboardDemo DistrictCinderella30K. Elfner & N. Fintel15

APBS 2020, Miami ‐ Session F103/13/2020Demo DistrictCinderellaCinderella31They found that most of their schools and studentsresponded positively.Percent of Schools with an Effective CoreIneffective 25.64Tier 1 is effective for 3 out PBISof every 4 PBIS schools– 33 schools with effective corehave more than 1020%schoolsof students receiving 2 Effective 74.36referrals32K. Elfner & N. Fintel16

APBS 2020, Miami ‐ Session F103/13/2020And when they didn’t, they could find out if theyneed support to implement more effectively.?33And target those identified schoolswith the support they need.4 Schools Reported anIneffective Core and PoorImplementation Fidelity.34K. Elfner & N. Fintel17

APBS 2020, Miami ‐ Session F103/13/2020But this is not the end of the story!For this was no ordinary district. They were not willing to stopwith broad positive outcomes.They wanted to make sure ALL students were having positiveresponses to their Tier 1 preventive system.35For those Schools with an Effective Core, did they haveEquitable Discipline Rates?36K. Elfner & N. Fintel18

APBS 2020, Miami ‐ Session F103/13/202017 PBIS Schools withEffective Core haverelatively EquitableDiscipline Outcomes! ATLANTIS ELEMENTARY SCHOOLCHALLENGER 7 ELEMENTARY SCHOOLCOCOA BEACH JUNIOR/SENIOR HIGH SCHOOLEAU GALLIE HIGH SCHOOLENDEAVOUR ELEMENTARY SCHOOLFAIRGLEN ELEMENTARY SCHOOLGOLFVIEW ELEMENTARY MAGNET SCHOOLHARBOR CITY ELEMENTARY SCHOOLHERBERT C. HOOVER MIDDLE SCHOOLHERITAGE HIGH SCHOOLJOHN F. TURNER, SENIOR ELEMENTARY SCHOOLLONGLEAF ELEMENTARY SCHOOLPALM BAY ELEMENTARY SCHOOLPINEWOOD ELEMENTARY SCHOOLRIVIERA ELEMENTARY SCHOOLSABAL ELEMENTARY SCHOOLSUNRISE ELEMENTARY SCHOOL 10 Schools ReportedEffective Core, butInequitable DisciplineOutcomes.ANDREW JACKSON MIDDLE SCHOOLAPOLLO ELEMENTARY SCHOOLCAMBRIDGE ELEMENTARY MAGNET SCHOOLCOQUINA ELEMENTARY SCHOOLCROTON ELEMENTARY SCHOOLIMPERIAL ESTATES ELEMENTARY SCHOOLJUPITER ELEMENTARY SCHOOLMILA ELEMENTARY SCHOOLSHERWOOD ELEMENTARY SCHOOLTHOMAS JEFFERSON MIDDLE SCHOOLNote: 6 schools reporting aneffective core did notprovide equity data.37They toiled away problem solving theInequitable Discipline. Shared the data with school staff Focus groups with students and families impacted by theinequitable discipline.– Shared results with school staff Used a Culturally Responsive problem solving structure tocategorize issues and select strategies to address the problemCCurriculum & Instructional PracticesAcademic, behavior & SELAAwarenessPersonal, cross‐cultural, best practicesRRelationshipsPositive, authentic; Students, family, communityEEnvironmentClimate, systems & policy variables; District, school & classroom levelsDDisciplineImplementation of policies/ procedures38K. Elfner & N. Fintel19

APBS 2020, Miami ‐ Session F103/13/2020And the District Lived HappilyEver After Identifying schools that needed extra support Making sure that PBIS works for all students39Far away in a scorching land, where now and againthe breezes blew and the waters came upon the land,there once was a school Middle School812 students54% FRL82% white, 10% black,8% Hispanic, 8% multi‐racial40K. Elfner & N. Fintel20

APBS 2020, Miami ‐ Session F103/13/2020They trained a leadership team in PBIS to Develop a school‐wide behaviorsupport system To reduce the rate of disciplineincidents across campus asevidenced by a reduced rate ofoffice discipline referrals41They implemented well! At least for a while.42K. Elfner & N. Fintel21

APBS 2020, Miami ‐ Session F103/13/2020And they watched their Referral Rate decline, until italso did not.43Perhaps they shouldhave paid closerattention when none ofthe 5 random studentsthey surveyed couldidentify their schoolwide expectationsduring the previousschool year?44K. Elfner & N. Fintel22

APBS 2020, Miami ‐ Session F103/13/2020So they looked closely at the critical elements thatneeded work.45Unable to find their magic wand, they decided toproblem solve Reduce Office Referral RateDefineAnalyzeWhat’s the problem?Why is it occurring?Not implementingPBIS with fidelityEvaluateImplementIs it working?What are we going to do?Staff buy‐in is and hasbeen low46K. Elfner & N. Fintel23

APBS 2020, Miami ‐ Session F103/13/2020They pledged to obtain staffbuy-in come hell or highwater. Well, let’s hope there’snot high water this year!47What school or district questions do you . Elfner & N. Fintel24

APBS 2020, Miami ‐ Session F103/13/2020If you have data available or if you were creating a dashboard,what kinds of questions would you want the dashboard toanswer? (At a school‐ or district‐level)School‐Level Questions: Pink CardDistrict‐Level Questions: Green Card49FLPBIS Project’s Next Steps Develop additionaltraining materials Create new visualizations Design a statewideproject‐level dashboard50K. Elfner & N. Fintel25

APBS 2020, Miami ‐ Session F103/13/2020Contact UsKaren Elfnerkchilds2@usf.eduNichole Fintelnsnyder@usf.eduwww.flpbis.org51K. Elfner & N. Fintel26

evolution of flpbis’ data reports: excel spreadsheets 19 20. . harbor city elementary school herbert c. hoover middle school heritage high school john f. turner, senior elementary school longleaf elementary school palm bay elementary school pinewood elementary school .