Evaluating Child Welfare Practice Model Fidelity And .

Evaluating Child Welfare Practice ModelFidelity and Supporting Practice and SystemChange through AssessmentPENNSYLVANIA’S WAIVER EXPERIENCE WITH THECANS, FAST, AND SPANSL a u r a Pa c ka r d Tu c ke r, A s s o c i a te Re s e a r c h e r, C h a p i n H a l l a t t h e U n i v e r s i t y o f C h i c a g oM a r y E l i za b e t h Ra u k t i s , Re s e a r c h A s s i sta n t P r o fe s s o r, U n i v e r s i t y o f P i t t s b u r g h S c h o o lo f S o c i a l Wo r k31st Annual Research & Policy Conference Child, Adolescent and Young Adult Behavior Health.March 4-7, 2018 Tampa FL3/20/2018

Overview Pennsylvania Title IV-E Waiver Overview CANS & FAST Evaluating Assessment Fidelity Measuring Fidelity with Targeting Trajectories Measuring Fidelity with the SPANS3/20/2018

Pennsylvania’s Title IV-E Waiver Overview The State’s Child Welfare Demonstration Project (CWDP) began on July 1, 2013and is currently in its fourth year. Six Pennsylvania counties are participating The counties agreed to respond to a change in federal funding policy by (1) developing a new case practice model using family engagement andstructured assessment, and (2) the introduction or expanded use of evidence-based programs (EBPs). The State’s CWDP seeks to improve outcomes for children around permanency,safety, and wellbeing.3/20/2018

Participatingcountiesin PA’s ChildParticipating countiesin Pennsylvania’sChild WelfareDemonstration DelawarePhiladelphia

IF families are engaged as part of a team, andPennsylvania’sTitle IV-E WaiverOverviewTHEORY OF CHANGEIF children and families receive comprehensive,structured screening and assessment to identifyunderlying causes and needs, and that assessmentinformation is used to develop a service plan, andIF that plan identifies roles for extended family membersand various supports and connects them to evidencebased services to address their specific needs,THEN, children, youth, and families are more likely toremain engaged in and benefit from treatment, so thatthey can remain safely in their homes, experience fewerplacement changes, experience less trauma, andexperience improved functioning3/20/2018

CANS & FAST Assessments Child and Adolescent Needs and Strengths (CANS) Family Advocacy and Support Tool (FAST) Conversation with parent, child, youth Assessment gathers information about Strengths and Needs Creates a Common Language across disciplines and betweenprofessionals and families Provides a Mapping and Framework Decision support for service planning Data collection and analysis3/20/2018

FAST Rating ScaleScoreLevel of NeedAppropriate Action0No evidence of need, can alsoindicate a clear strength1Significant history or possibleneed which is not interferingwith functioningNeed interferes withfunctioningNeed is dangerous or disablingNo action neededOR opportunities forstrength-based planningWatchful waiting/preventionOR opportunities forstrength buildingAction/ Intervention233/20/2018Immediate/ Intensive Action

CANS Rating Scale - NeedsScoreLevel of NeedAppropriate Action0No evidence of needNo action needed1Significant history or possibleneed which is not interferingwith functioningWatchful waiting/Prevention/ Additionalassessment2Need interferes withfunctioningAction/ Intervention3Need is dangerous or disablingImmediate/ IntensiveAction3/20/2018

CANS Rating Scale - StrengthsScoreLevel of StrengthAppropriate Action0Centerpiece StrengthCentral to Planning1Strength PresentUseful in Planning2Identified StrengthMust be Built/ Developed3No Strength IdentifiedStrength Creation orIdentification may be Indicated3/20/2018

Evaluating CANS & FAST FidelityAre theassessmentstargeting theanticipatedpopulation? Embed assessment in CWtrajectory Understand how timingrelates to stated targetpopulation and system entryAre the Utilize the SPANS measureassessments Review how the identifiedleading to quality needs and strengths arepresent in the family servicefamily serviceplans and servicesplans?3/20/2018CANS& FASTFidelity

Measuring Fidelity withTargeting Trajectories3/20/2018

CANS & FAST Targeting Target Populations - Each of the 6 CWDP counties has their own stated targetpopulation for each assessment (although there is overlap) Child Welfare Trajectory - To evaluate targeting fidelity, determined where thechild was in their child welfare event trajectory when they received their firstassessment Flash Reports - Reviewed targeting analysis with the CWDP counties to gainqualitative data on discrepancies between intended and actual assessmenttargeting3/20/2018

CANS & FAST Targeting – TrajectoryPopulations1. Population 1 - Children with no prior substantiations or placements prior to thisassessment (not risen to level of an “official” child welfare action: substantiation orplacement)2. Population 2 - Children with at least one substantiation or placement, but not in anout-of-home placement at the time of this assessment (has been subject to anofficial action, but not in care at the time our agency conducted this CANS/FAST)3. Population 3 - Children in an out-of-home placement at the time of this assessment(deeply involved at the time of this CANS/FAST)Population 1SubstantiatedInvestigation3/20/2018Population 3Population 2Out-of-HomePlacementExit fromPlacement

How many children received their first FAST assessment in SFY14, SFY15, & SFY16?SFY 2014FAST Targeting1.Population 1 - Children withno prior substantiations orplacements prior to thisassessment2.Population 2 - Children with atleast one substantiation orplacement, but not in an outof-home placement at thetime of this assessment3.Population 3 - Children in anout-of-home placement at thetime of this angoSFY 20161857271,0794,36592For children who received their first FAST assessment in fiscal year 2016, wherewere they in their child welfare opulation 13/20/201807073,5134,610208SFY 20153953891,8616,11243LackawannaPopulation 2PhiladelphiaPopulation 3Venango

How many children received their first CANS assessment in SFY14, SFY15, & SFY16?SFY 2014CANS Targeting1.2.3.Population 1 - Children withno prior substantiations orplacements prior to thisassessmentPopulation 2 - Children with atleast one substantiation orplacement, but not in an outof-home placement at thetime of this assessmentPopulation 3 - Children in anout-of-home placement at thetime of this elphiaVenangoSFY 2016866392715976126For children who received their first CANS assessment in fiscal year 2016, wherewere they in their child welfare lation 13/20/20181,92903043576610SFY 20151,123881922291,14951DauphinLackawanna PhiladelphiaPopulation 2Population 3Venango

Proximal Eventfollowing a firstSubstantiatedInvestigationFor children who received their firstSUB in fiscal year 2016, what was theirnext event?CrawfordNO SECOND EVENT191CANS0FAST3SUB34PLACE70STOP15TOTAL 1ST SUBs IN SFY2016313Dauphin Lackawanna Philadelphia %0%CrawfordDauphinLackawanna PhiladelphiaNO SECOND EVENTCANSFASTSUBPLACEVenangoSTOP

Measuring FamilyService Plan Fidelitywith the SPANS3/20/2018

What is the SPANS?Service Process Adherence to Needs andStrengths (SPANS): A fidelity tool thatdetermines the degree to which child and familyneeds and strengths identified in the FAST/CANSare being addressed or used in planning andservice provision.SPANS-CANSSPANS-FAST3/20/2018Are we doing what we said we would do?

How is the SPANS Used to answer thequestion “Are we doing what we saidthat we would do?”QualityAssuranceIndividual Level(Supervision)Group/Team Level(Supervision)Agency/Program LevelSystem Level3/20/2018

How is the SPANS Implemented in theWaiverRandom Sample of cases selected from each county withinvarious categories e.g. new, continuing The size of the sample is based upon the size of the countyCWS with Philadelphia and Allegheny the largest samples,Venango and Crawford are the smallestCase files (focusing on plan and notes but all of the file isread) read by two reviewers and independently scored andthen consensus scores are given for each case3/20/2018

Scoring only “Actionable” itemsCANS Any need rated “2” or “3” is the focus of the scoring any strength rated “O” or “1” is the focus of the scoringFAST Any need rated “2” or “3”Quality Assurance Questions:When items are rated actionable in the CANS how likely are they to appear in the FamilyService Plan? Is this supported in documentation? If not, how is the absence supported.When items are rated actionable in the FAST how likely are they to appear in the treatmentplan?How are needs prioritized?3/20/2018

CANS-SPANSActionable Items in CANS less likely to appear in the PlanWithin Life Functioning: Intellectual Developmental; Physical/Medical;Sleep; Social Functioning; School Behavior; School Achievement; SchoolAttendanceWithin Caregiver Strengths and Needs: Social Resources; Organization;Residential StabilityWithin Youth Behavioral/ Emotional Needs: Anger Control; SubstanceAbuse; Depression; Adjustment to Trauma; AttachmentWithin Youth Risk Behaviors/Factors: Exploited, Runaway; SuicideRisk; Sanction Seeking Behavior; Sexual AggressionWithin Transition Age: Job functioning; Independent living; Peer SocialExperienceWithin Trauma Experiences: Physical Abuse; Sexual Abuse;Witness/Victim Criminal Acts; Neglect; Emotional Abuse; Disruption inCaregiving; Witness to Family ViolenceWithin Youth Strengths: Spiritual Religious; Vocational; CommunityConnection; Interpersonal Skills; Interests; Relationship Permanence;Family3/20/2018

FAST-SPANSActionable Items in FAST less likely toappear in the plansWithin Family Together: Extended FamilyRelations; Financial Resources;Parental/Caregiver collaboration; FamilyConflictWithin Caregiver A Status; CaregiversPosttraumatic Reactions; Caregiver’sPartner RelationsWithin Child A Status: Relationship withbio Mother; Relationship with Bio Father;Social Functioning3/20/2018

SummaryDiscussions from the FLASH Talks suggested that as a waiverprogram early in the implementation, that while safety andpermanence related items (family and parents needs andhousing and safety) immediate needs were beingaddressed, well-being items in terms of child behavioralhealth and trauma were identified, prioritized but notaddressed as completely in the plans.Why? Differed by county but some reasons werecaseworker training, limited services, family engagement3/20/2018

Using the CANS and FAST with the SPANSPoorly done FAST or CANS means that the SPANS is more difficult toscore accuratelyInadequate paper filing results in missing information in paper files—less of a problem with electronic recordsTime-intensive As a research project we required 2 independent raters Implementing as a quality improvement activity, the SPANS wouldbe done by a supervisor and not require a second independentrater3/20/2018

Summary: SPANS Although there was a waiver with an emphasis on trauma-informed care andwell-being, plans and documentation about services was more focused onsafety. Plans were often “cookie cutter” with little individualization despite the use ofindividualized Assessments. Strengths were difficult for workers to operationalize in plans. Need to do a better job in training on assessments to move beyond how to dothe assessment and Focus on how to use the assessment.3/20/2018

Summary: Targeting Targeting analysis highlighted differences in counties’ practices aroundassessment implementation – both in intended and actual target populations. Understanding differences in targeting population is necessary to understandhow assessments may impact county outcomes. Bringing these findings to the counties in the Flash Report forum helped theevaluation team gain insight into the context and challenges around assessmentimplementation at the county level.3/20/2018

Questions? Contact Info: Laura Packard Tucker, Associate Researcher, Chapin Hall, ltucker@chapinhall.org Mary Elizabeth Rauktis, Research Assistant Professor, University of Pittsburgh Schoolof Social Work, mar104@pitt.edu3/20/2018

Clearfield Cambria Potter Tioga Bradford Susquehanna Cameron Clinton Blair Bedford Huntington Fulton Franklin Mifflin Juniata Perry Cumberland Adams York Dauphin Snyder Union Lycoming . Differed by county but some reasons were caseworker