Behavior Assessment In RtI - Direct Behavior Ratings

Behavior assessment in RtI:Considerations whenselecting assessment toolsSandra M. Chafouleas, Ph.D.Neag School of Education, University of ConnecticutT. Chris Riley-Tillman, Ph.D.Department of Psychology, East Carolina UniversityAmy M. Briesch, Ph.D.Department of Counseling and Applied Psychology,Northeastern University

Review strengths and limitations ofdifferent school-based behaviormethods within a multi-tiered model ofassessment

1 in 3 pendingaknows someone whohasdisproportionateamountleftdueof thetimeprofessionrespondingtotoissuesrelated todisciplinesignificantbehaviorand behavior(Publicbychallengespresented2004)ofa Agenda,smallCONSIDER numberstudents (U.S. Dept. ofEd., 2000)1 in 5 children has mentalhealth needs yet theSchool discipline is a topmajority will not receiveconcern by the Americanneeded servicespublic (Rose & Gallup,(Hoagwood & Erwin,2005)1997; U.S. SurgeonGeneral, 1999)

Response to Intervention (RTI) An assessment and intervention process forsystematically monitoring student progressand making data-based decisions about theneed for instructional modifications orincreasingly intensified services (seewww.rti4success.org).

Tertiary (FEW)Reduce complications, intensity,severity of current casesSecondary (SOME)Reduce current casesof problem behaviorPrimary (ALL)Reduce new casesof problem behavior

IEP or other student-specific Increasethe numberbehavior goalrelatedoftoappropriatebehaviorsfor thisacquisition ofappropriategroupof studentssocial skillsIndividualTargeted Decreasenumberof goalIncreases inthepro-socialbehaviorStudent-specificbehaviorof student bodybehaviors forinappropriaterelated to decreasein problem thisDecreasesininappropriategroupbehaviorbehavior of student body Decreases in number of studentsreferred for an evaluation forbehavior related disordersUniversal

. but then, how will you know whatyou are doing is working? To make effective decisions about whichpractices are needed and then toevaluate the outcomes of thosepractices, you need DATA!

Developing evidence-based assessment (EBA)begins through a priori delineation ofa) the purposes of assessment, and thenb) identification of the special requirements for each purpose(and associated criteria for stating when requirement ismet)Commentary by Kazdin (2005)

A Few Caveats to Establishing EBA (Kazdin, 2005):Absence of a gold standardcriterion One measure can’t do it all Multiple measures areneeded to evaluatedifferent facets Co-morbidity of “problems” What are the mostrelevant problem features? Multiple perspectives arevaluable yet agreement may(will) be low! What is THEmeasure Ishould use?

Screening Who needs help? Diagnosis Why is the problem occurring? Progress Monitoring Is intervention working? Evaluation How well are we doing overall?Emphasized bythe NationalCenter onResponse toIntervention

ALL BELOW plus functional assessment dataALL BELOW plus Individual Norm-referenced comparison data More detailed profiles of students’strengths/weaknessesTargeted Formative data sources sensitive toincremental changeEFFICIENT, EXTANT SOURCES such as Total number of office discipline referrals Number of students who are suspended orexpelled Number students referred or found eligible forspecial education, particularly in the area ofemotional disturbanceUniversal

ScreeningDiagnosisEvaluationProgressMonitoring

Extant data Standardized behaviorrating scales Systematic directobservation Direct Behavior RatingCurrentlydominate inclinic andresearch

Definition: Data sources that already exist within the setting(“permanent products”)Advantages: Already available Highly contextually relevant Natural occurrence can reduce/limit reactivity(Adapted from Chafouleas, Riley-Tillman, & Sugai, 2007)

Examples: Office discipline referrals (ODRs) Attendance and tardy records Suspension/expulsion data Special education data (e.g. referrals foremotional disturbance) Data from existing behavior managementplans (e.g. token economy)

“an event in which (a) a studentengaged in a behavior thatviolated a rule/social norm in theschool, (b) a problem behaviorwas observed by a member ofthe school staff, and (c) the eventresulted in a consequencedelivered by administrative staffwho produced a permanent(written) produce defining thewhole event” (Sugai, Horner, &Walker, 2000, p. 96)

MAJOR Abusive Language/InappropriateLanguage/ ProfanityArsonBomb Threat/ ComplianceDisruptionDress Code ViolationFighting/ PhysicalAggressionForgery/ TheftGang Affiliation DisplayHarassment/BullyingInappropriate Display ofAffectionInappropriate Location/Out of Bounds Lying/CheatingOther Behavior PropertyDamage/Vandalism Skip class Truancy Tardy Technology Violation Unknown Use/Possession of Alcohol Use/Possession ofCombustibles Use/Possession of Drugs Use/Possession of Tobacco Use/Possession of WeaponsMINOR Defiance/Disrespect/Non-complianceDisruptionDress Code ViolationInappropriate LanguageOtherPhysical Contact/ PhysicalAggressionProperty MisuseTardyTechnology ViolationUnknownSource: 2009-10 Referral Definitionswww.swis.org

ScreeningDiagnosisEvaluationProgressMonitoring

Discrepancy ratioBut howmuch studentisbehavior/too much?peerbehavior2x significantdiscrepancyWillie’s ODR 10/Mean ODR 3.5DR 2.86

Staff at West HighSchool note concernabout the number offights occurring amongstudents.Staff are re-assigned toincrease levels of activesupervision in thoseareas at key times.ODRs over thepast 2 months arereviewed“Johnny and Sam” arebrought to BehaviorSupport Team foradditional supportplanning.Review revealed a) most fightingincidents are occurring outsidecafeteria and in bus loading areaAND b) Johnny and Sam are themost likely culprits.

FRMS Total Office Discipline ReferralsSustained ImpactPre3000PostTotal ODRs2500200015001000500094-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 05-06Academic Years

Example for diagnosis withODR?

Complements othersources in providingcontextually relevantinformation Source of progressmonitoringinformation Less resourceintensive (datareadily available!) Limited applicationwithin prevention Tough to establishand maintainconsistent/ accurateuse. Unknownpsychometricadequacy Challenging to createa system for efficientorganization andinterpretation

Screening – yes, but may be limited inprevention/early intervention rolesProgress monitoring – yes, but creating usablesystem for interpretation/presentation can bechallengingDiagnosis – maybe, with regard to addingcontextual relevanceEvaluation – yes, relevance within the specificsetting but limited with regard to normcomparisons

Definition:Tools that require an individual to rate the behavior ofanother based on past observation of that person’sbehaviors (Kratochwill, Sheridan, Carlson, & Lasecki, 1999). Examples: Behavior Assessment System for Children – 2 (BASC-2) Achenbach System of Emprically-Based Assessment(e.g. CBCL) Conner’s Rating Scales – 3 Social Skills Intervention System (SSIS)

Comprehensive scales: large number of items (often100 ) that cluster together to assess a wide range ofbehaviors “General purpose” (Merrell, 2008) Often include broadband and narrow-band syndromes(Ramsey, Reynolds & Kamphaus, 2002). Narrow band scales: focused on one or twobehavioral constructs Attention (Brown ADD Scales; Brown, 2001) Adaptive behavior (Vineland-II; Sparrow, Balla, & Cicchetti, 1984)

Problem entionInternalizingAutism SpectrumSocial lityEmpathyEngagementSelf-Control

Ages 7-1727 items (2 week reflection)NegativemoodI am sad once in a whileall the timeInterpersonalProblemsIneffectivenessI like being withpeopleI do mostthings I do not wantto be withpeople at allO.K.wrongAnhedoniaNegative SelfEsteemI am tired Once in awhileI look O.K.all the timeI look ugly

ScaleItem ExamplePhysicalSymptomsI have pains in my chestMy hands feel sweaty or coldHarmAvoidanceI check to make sure things are safeI worry about doing something stupid or embarrassingSocial AnxietyI have trouble asking other kids to play with meI worry about other people laughing at meSeparation/PanicI keep the light on at nightI avoid going places without my familyAges 8-19

But whataboutscreening andprogressmonitoring?

T score 61-70 elevated riskT score 71 highly elevated risk

Short form (approx. 40 items) Fewer items per scale Recommended for progress monitoring ADHD Index Inattention Hyperactivity/Impulsivity Learning Problems Aggression Executive Functioning Peer Relations Family Relations 10 items that best differentiate children with ADHD fromthose without a clinical diagnosis Recommended for screening and progress monitoring Global Index 10 best items from original Conners’ Rating Scales Progress monitoringTemper outbursts, Excitable/impulsive, Restless, Cries often, Inattentive, Fidgeting, Disturbsother children, Easily frustrated, Fails to finish things, Mood changes quickly

47 items designed to assess scales of AttentionProblems, Hyperactivity, Internalizing Problems,Adaptive Skills

May be most helpful in diagnostic assessment.Provide a commonunderstanding of thespecific behaviors thatare indicative of a givencluster term.May also be suited foruse in screening andevaluative assessmentpractices.May not be sensitive toincremental change. May be feasible only foroccasional use giventime/cost. Many clinically-focused(i.e., focus on problemrather than pro-socialbehavior). Do not directly assessbehavior –rater bias maybe present.

Screening – yes, but scope and size ofmeasures varies widely Progress monitoring – not likely Diagnosis – yes, most common usewithin clinical settings Evaluation – maybe, if the period of timeis sufficient and constructs measuredare relevant

Definition:Data collected by an observer watching anenvironment/person for some period of time Examples: Percentage of intervals observed to be actively engaged Frequency of positive peer initiations throughout theday Recording how long it takes to transition in the hallway(duration)

Studies suggest moderate to high levels ofreported use 67% of school psychologist report using directobservation in 4 of their last 10 case Shapiro &Heick (2004), 63% to 73% of School Psychologistreport moderate to frequent use (Riley-Tillmanet. al, 2008).

Frequency - number of events in a period oftime (e.g., 4 hits in a 6 hour day)Rate - number of events per unit of time (e.g.,4 social initiations per hour)Percentage of opportunities – use ifbehaviors follow specific cues (e.g., followeddirections given on 40% of occasions)

Data recorded during pre-specified intervals of time,then summarized into percentage of intervals ofbehavioral occurrences Time-based techniques result in approximations ofbehavioral events because behavior is sampled inone of three basic ways: Whole interval recording Partial interval recording Momentary time sampling

Duration Total time (e.g., actively engage in reading for 12minutes) Percent of time (e.g., out of seat for 35% of the readingperiod) Average time per event (e.g., each temper tantrumlasted an average of 7.5 minutes)Latency – time for behavior to begin after prompt orantecedent cue provided (e.g., on average 2 minutesto begin task after teacher direction given)

BASC-2 Student Observation System (Reynolds & Kamphaus, 2004) 15-minute observation w/ 30-second intervals Response to teacher, Peer Interaction, Works on School Subjects, TransitionMovement, Inappropriate Movement, Inattention, Inappropriate Vocalization,Somatization, Repetitive Motor Movements, Aggression, Self-Injurious Behavior,Inappropriate Sexual Behavior, Bowel/bladder problems Academic Engaged Time Code of the SSBD (Walker & Severson, 1990) Time spent engaged in academic material Let stopwatch run Divide AET by Total Time Behavioral Observation of Students in Schools (Shapiro, 2004) 15-minute observation w/ 15-second intervals Active/Passive Engaged, Off-task motor/verbal/passive, Teacher-Directed Instruction