Definition Of Academic Competence - Sacramento State

Definition of AcademicCompetenceThe Academic CompetenceEvaluation Scales (ACES) Academic competence is a multidimensional construct composed ofthe skills, attitudes, and behaviors of alearner that contribute to academicsuccess in the classroom.Rating scale technology foridentifying students with academicdifficulties and planning interventionsto improve their functioning.Copyright 2001 by The Psychological CorporationCopyright 2001 by The Psychological Corporation1Purpose and Uses ofthe ACES ACES OverviewAssess academic skills and enablingbehaviors for students in grades k-12 andcollege Three record forms: Teacher, Student, andCollege StudentXX XFacilitate planning and evaluation ofclassroom-based interventions for studentsexperiencing academic difficultyCopyright 2001 by The Psychological Corporation 3ACES Teacher:Sample ItemsY2Teacher Form appropriate for Grades k-12Student Form appropriate for Grades 6-12College Student Form appropriate for students at 2and 4-year institutionsTechnical ManualScoring Assistant (Optional)Copyright 2001 by The Psychological Corporation4Companion InterventionResources: AIMSPlease see your copy of the ACES recordform to review sample items.Copyright 2001 by The Psychological Corporation5 The Academic Intervention Monitoring System(AIMS) is a companion guidebook thatdescribes effective intervention strategies forskills measured by the ACES. The AIMS Forms are questionnaires designedto facilitate identification of specific instructionalor learning tactics used by teachers, parents,and students.Copyright 2001 by The Psychological Corporation61

Companion InterventionMonitoring Method: GASKey Steps in a ProblemSolving Process Goal Attainment Scaling (GAS) provides a quickand reliable method for assessing change instudent skills during intervention. GASs aredeveloped by users after completing the ACES. Together, the ACES, AIMS, & a GAS can beused as part of a 5-step problem-solvingprocess for developing and evaluatinginterventions for academic difficulties.1. Identify academic concerns2. Analyze academic concerns within theinstructional environment3. Plan for intervention4. Implement intervention & monitor progress5. Evaluate intervention7Copyright 2001 by The Psychological CorporationCopyright 2001 by The Psychological Corporation8Model ofAcademic Competence Research using the ACES has indicated that theskills, attitudes, and behaviors contributing toacademic competence fall into one of twodomains:AcademicCompetenceStudy SkillsMotivationAcademic Skills or Academic Enablers.InterpersonalSkills(DiPerna & Elliott, 1999)AcademicSkillsEngagement Reading Mathematics Critical Thinking9Copyright 2001 by The Psychological CorporationAcademic Skills andEnablers Subscales Academic SkillsSubscalesXXXReading/Language ArtsMathematicsCritical Thinking Copyright 2001 by The Psychological CorporationAcademic Skills andEnablers SubscalesYMotivation reflects a student’s approach,persistence, and level of interest regardingacademic subjectsYEngagement reflects attention and activeparticipation in classroom activitiesAcademic EnablersSubscalesXXXX10MotivationEngagementStudy SkillsInterpersonal SkillsCopyright 2001 by The Psychological Corporation11Copyright 2001 by The Psychological Corporation122

Teacher Judgements ofAcademic PerformanceAcademic Enablers SubscalesYStudy skills are behaviors that facilitate theprocessing of new material and taking testsYInterpersonal skills include cooperative learningbehaviors necessary to interact with others Teachers are accurate in predicting students’achievement on standardized tests anddistinguishing between students with low or highacademic performance.Key Studies on “teachers as tests”XXXCopyright 2001 by The Psychological Corporation13ACES-Teacher Form:Fast FactsYYYYY2 scales: Academic Skills (33 items) & AcademicEnablers (40 items)Requires less than 20 minutes to complete2 ratings/item: Proficiency & ImportanceCompetence ranges: Developing, Competent,& AdvancedWorksheet for linking assessment results to interventions& creating a GASYY141000 studentsXXXXFour grade clusters: k-2, 3-5, 6-8, 9-12(n 250 each)Even distribution across sex64% Caucasian, 15% African American,15% Hispanic, 6% Other Race9% students with disabilities, 19% students at-risk,72% students with no identified concernsCopyright 2001 by The Psychological Corporation16ACES-Teacher: Evidencefor ReliabilityPrincipal Components Analysis (PCA) including all 73items yielded a 2-factor solution: Academic Skills &Academic EnablersSeparate PCAs of the 33 Academic Skills items and 40Academic Enablers items yielded 3- and 4-factorsolutions respectively100% of items on the Academic Skill subscales hadprimary loadings .40100% of items on the Academic Enablers subscales hadprimary loadings .40Copyright 2001 by The Psychological Corporation 15ACES-Teacher: Evidencefor Internal StructureYCopyright 2001 by The Psychological CorporationACES-Teacher:Standardization SampleCopyright 2001 by The Psychological CorporationYHoge & Coladarci (1989)Gresham, Mac Millan, & Bocian (1997)Demaray & Elliott (1998)17YYYYInternal consistency coefficients (Cronbach’salphas) from .94-.99SEMs from 2.46-3.07 for Academic Skills and3.63-4.73 for Academic EnablersTest-retest stability coefficients from .88-.97Inter-rater correlations from .31-.65Copyright 2001 by The Psychological Corporation183

ACES-Teacher: Evidencefor ValidityYYYIdentifying Target Skillsfor InterventionAcademic Skills - Iowa Test Basic Skills Compositecorrelations from .66-.76Academic Skills - Social Skills Rating System AcademicCompetence correlations from .75-.80Discriminant function analysis with known groups(students with no identified concerns vs. students at-riskor with disabilities) indicated the ACES correctly identifiedaverage of 86% of students across multiple samplesCopyright 2001 by The Psychological CorporationXYUse ACES Behavior Classification Scheme toidentify strengths, performance problems, andacquisition problems in academic skills and/oracademic enablersCopyright 2001 by The Psychological CorporationYAnalyze target skills within the context of theinstructional environment (using AIMSIntervention Forms, observations, and/orinterview)YPlan intervention based on the results of thisanalysis. (Consult AIMS Guidebook and otherresources for intervention strategies.)Proficiency 3, 4, or 5Importance 2 or 3Performance ProblemsXXYYProficiency 2Importance 2 or 3Acquisition ProblemsXXProficiency 1Importance 2 or 3Copyright 2001 by The Psychological Corporation21Monitor Intervention ProgressYDevelop Goal Attainment Scale (GAS) tomonitor progress via teacher ratingsYImplement intervention and collect GAS data tomonitor effectivenessCopyright 2001 by The Psychological Corporation20Analyze Environment &Plan InterventionsStrengthsXTeacher form includes worksheet to facilitatethe use of assessment results to select anddesign interventions19ACES BehaviorClassification SchemeYYCopyright 2001 by The Psychological Corporation22GAS: A General DescriptionGoal Attainment Scaling (GAS) is a method ofevaluating interventions that is:1. Individualized,2. Criterion-referenced,3. Atheoretical,4. Time efficient, and5. Based on rating scale technology.23Copyright 2001 by The Psychological Corporation244

GAS: Specific FeaturesGoal Attainment Scale WorksheetThere are three common features of GASs.They all have .YYY 2A target behavior(s),Descriptions of treatment outcomes in objectiveterms, andThree to five descriptions of probable treatmentoutcomes that range from “least favorable” to“most favorable.” 10-1- 2Copyright 2001 by The Psychological Corporation25Evaluate InterventionOutcomesYYYVisual analysis of GAS resultsYYPretest & Posttest ACES ratings analyzed usinga Reliability Change Index(RCI Posttest Mean-Pretest Mean/SEM)Copyright 2001 by The Psychological CorporationYThree forms: Teacher, Student, & CollegeAssesses academic skills and enabling behaviors forstudents in Grades k-12 or collegeDesigned to facilitate a pre-referral problem-solvingprocessResearch indicates strong evidence for internal structure,reliability, and validityAIMS companion Guidebook and Intervention Forms toassist with intervention planning27Summary ThemesCopyright 2001 by The Psychological Corporation28References Respects Teachers’ KnowledgeDemaray, M. K., & Elliott, S. N. (1998). Teachers’ judgments of students’academic functioning: A comparison of actual and predictedperformances. School Psychology Quarterly, 13, 8-24. Solution FocusedDiPerna, J. C., & Elliott, S. N. (1999). Development and validation of theAcademic Competence Evaluation Scales. Journal ofPsychoeducational Assessment, 17, 207-225. 26ACES: Key Pointsto RememberACES provides two methods for assessing theeffects of an intervention:YCopyright 2001 by The Psychological CorporationData-Drive & Theoretically BalancedGresham, F. M., MacMillan, D. L., & Bocian, K. M. (1997). Teachers as“tests”: Differential validity of teacher judgments in identifying studentsat-risk for learning difficulties. School Psychology Review, 26, 47-60.Hoge, R. D., & Coladarci, T. (1989). Teacher-based judgments ofacademic achievement: A review of literature. Review of EducationalResearch, 3, 297-313.Copyright 2001 by The Psychological Corporation29Copyright 2001 by The Psychological Corporation305

YPrincipal Components Analysis (PCA) including all 73 items yielded a 2-factor solution: Academic Skills & Academic Enablers YSeparate PCAs of the 33 Academic Skills items and 40 Academic Enablers items yielded 3- and 4-factor solutions respectively Y100% of items on the Academic Skill subscales had primary loadings .40