MDE Assessment Intregrity Guide - Michigan
AssessmentIntegrity GuideMICHIGAN DEPARTMENT OF EDUCATIONOFFICE OF EDUCATIONAL ASSESSMENT ANDACCOUNTABILITYMichigan Standards for Professional and EthicalConduct in Test Administration and Reporting2020-21
MICHIGAN DEPARTMENT OF EDUCATIONASSESSMENT INTEGRITY GUIDEOffice of Educational Assessment and AccountabilityThe Michigan State Board of Education approved this guide on April 12, 2016 with updates on9/14/20. This document pertains to the expected professional and ethical conduct of all assessmentprograms administered by the Michigan Department of Education (MDE), Office of EducationalAssessment and Accountability (OEAA). This includes all Michigan summative, interim, and formativeassessments.If you have questions or wish to report suspected test administration irregularities on any statewideeducational assessments or accountability issues in Michigan, contact:Michigan Department of EducationOffice Educational Assessment and Accountability608 West Allegan StreetPO Box 30008Lansing, MI 48915toll-free phone: 877-560-8378email: MDE-OEAA@michigan.govfax: 517-335-1186Copies of this guide may be obtained on theStudent Assessment web page (www.michigan.gov/OEAA).STATE BOARD OF EDUCATIONMichael F. Rice, Ph.D., State SuperintendentMichelle Fecteau Tom McMillinJudith Pritchett Pamela PughLupe Ramos-Montigny Nikki SnyderTiffany D. Tilley Dr. Casandra E. Ulbrich2
3Table of ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Case Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Importance of Assessment Security . . . . . . . . . . . . . . . . . 5Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Assessment Security Goals for the State of MichiganAssessment System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Common Assessment Irregularities. . . . . . . . . . . . . . . . . . . . 6OEAA Assessment Monitoring. . . . . . . . . . . . . . . . . . . . 28Communication Protocol. . . . . . . . . . . . . . . . . . . . . . . . . . 8Section 1 – Prevention of Testing Irregularities.9Testing Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Eligible Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Testing Personnel Roles and Responsibilities. . . . . . . . . 10Identification of Testing Personnel . . . . . . . . . . . . . . . . . 10Assessment Security Training . . . . . . . . . . . . . . . . . . . . . 11District/Building Assessment Coordinator TrainingRequirements: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Test Administrators/Room Supervisors/Proctors/ Training Requirements:. . . . . . . . . . . . . . . . . . . . 12Technology Coordinators and Other Staff (anyone whohandles or has access to secure materials) TrainingRequirements: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Test Administration Training . . . . . . . . . . . . . . . . . . . . . . 12OEAA Assessment Security Compliance Form . . . . . . . 12Assessment Supports for Students. . . . . . . . . . . . . . . . . 13District and School Internal Assessment Monitoring. . . . . . 27OEAA Targeted and Random Assessment Monitoring . 28Targeted Assessment Monitoring . . . . . . . . . . . . . . . . . . . 28Random Assessment Monitoring . . . . . . . . . . . . . . . . . . . . 28Test Administration Observation MonitoringProcedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28OEAA Internet and Media Monitoring . . . . . . . . . . . . . 29OEAA Supports and Accommodations Monitoring.29Statistical Analysis of Results and DataForensics Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Types of Data Forensics Analyses Used for the StateAssessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Unusual Score Gains and Losses. . . . . . . . . . . . . . . . . . . . .Corrective Change Analysis . . . . . . . . . . . . . . . . . . . . . . . .Occurrence of Perfect Scores. . . . . . . . . . . . . . . . . . . . . . .Response Time Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .Person-Fit Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3030303030Data Reporting Practice . . . . . . . . . . . . . . . . . . . . . . . . . 31Appropriate Data Reporting . . . . . . . . . . . . . . . . . . . . . . . . 31Student Test Preparation. . . . . . . . . . . . . . . . . . . . . . . . . 13Section 3 - Follow-Up Investigations . . . . . . . . . . 31Professional Student Test Preparation Practices. . . . . . . 14Internal Investigation. . . . . . . . . . . . . . . . . . . . . . . . . . . 31Prohibited Student Test Preparation Practices . . . . . . . . 14Independent Investigation . . . . . . . . . . . . . . . . . . . . . . . 32Test Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Section 4 – Remediation . . . . . . . . . . . . . . . . . . . . 33Testing Schedule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Off-Site Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Seating Charts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Testing Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Video Surveillance Cameras. . . . . . . . . . . . . . . . . . . . . . . .School Decision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351516161719Retention of Test Administration Documentation . . . . . 19Assessment Security & Test Administration Practices.Professional Assessment Security Practices . . . . . . . . . . . .Prohibited Assessment Security Practices. . . . . . . . . . . . . .Professional Test Administration Practices. . . . . . . . . . . . . .Prohibited Test Administration Practices . . . . . . . . . . . . . .2020212223Student Prohibited Behavior . . . . . . . . . . . . . . . . . . . . . 24Handling Prohibited Behavior . . . . . . . . . . . . . . . . . . . . . . 24Section 2 – Detection of Irregularities. . . . . . . . . 26Reporting Irregularities. . . . . . . . . . . . . . . . . . . . . . . . . . 26Allegations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Independent Review Panel. . . . . . . . . . . . . . . . . . . . . . . 35Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Appendix A – Assessment Integrity ProcessFlowchart. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Appendix B – Sample Testing Schedules. . . . . . . . 37Appendix C – Sample Test AdministrationObservation Checklist . . . . . . . . . . . . . . . . . . . . . 38Appendix D – Professional Standards andGuidelines for Best Testing Practices andAssessment Security. . . . . . . . . . . . . . . . . . . . . . . 40Appendix E – Keeping AssessmentMaterials Secure . . . . . . . . . . . . . . . . . . . . . . . . . . 41Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
MICHIGAN DEPARTMENT OF EDUCATIONASSESSMENT INTEGRITY GUIDEOffice of Educational Assessment and Accountability4IntroductionA primary function of the Michigan Department of Education (MDE) Office of Educational Assessmentand Accountability (OEAA) is to establish, develop, and implement a state assessment system thatfairly, accurately, and with validity measures Michigan’s content standards.State assessments are required under both state and federal law, to ensure all children are learning andreceiving a high-quality education.Michigan assessments include summative and benchmark assessments. The state’s summativeassessments are required while the benchmark assessment offered by the state is optional. The tablebelow details Michigan’s state assessments.AssessmentDescriptionGradesACT WorkKeysGiven each spring as a component of the MME exam to measure work skills.11Early Literacyand MathematicsBenchmarkAssessmentsBenchmark assessment given each fall, winter, and spring tomeasure growth of early literacy and mathematics skills.K-2MI-AccessState summative alternate assessment given each spring tostudents who have, or function as if they have, significantimpairments, and whose IEP (Individual Education Program)Team has determined that general assessments, even withaccommodations, are not appropriate for the student.3-8, 11M-STEPState summative assessment given each spring to assess studentprogress on Michigan’s content standards.3-8, 11PSAT 8/9Given each spring to measure student knowledge of state standards in ELA and mathematics.8SAT with EssayGiven each spring as a component of the MME exam tomeasure student knowledge of state standards in ELA andmathematics.11WIDA ACCESS forELLsGiven each winter to English learners to measure English language proficiency.K-12WIDA AlternateACCESS for ELLsGiven to English learners who have, or function as if they have,a significant cognitive disability to measure English languageproficiency.1-12W-APT and WIDAScreenerAssessment screening tools used by educators to measureEnglish language proficiency of students who have recentlyarrived in the U.S. or in a particular district.K-12
5The OEAA develops assessments and establishesprofessional conduct standards based upon laws andprofessional guidelines and best practices (refer toAppendix D for this list of resources).This Assessment Integrity Guide focuses on four mainintegrity themes: Prevention – standards and best practicesfor the test integrity and security aspects ofthe design, development, operation, andadministration of state assessments, bothpaper/pencil and online test administrations, toprevent irregularities from occurring Detection – guidelines for assessmentmonitoring, reporting, and working with theOEAA when irregularities are found Follow-Up Investigations – guidelines for thestate and local educational agency workingtogether to investigate irregularities Remediation – guidelines for working with theOEAA to resolve irregularities and to ensurevalid results for all studentsAll test administration practices are subject to thisultimate question: Will the test administration leadto student results that accurately reflect a validand reliable measure of what each student knowsand is able to do compared to Michigan’s Academicstandards? Following these guidelines will ensurethat all students have equal opportunities to showtheir knowledge, skills, and abilities, and are activelyinvolved in demonstrating those opportunities throughtheir engagement with the test.Educators, students, parents, school boards,legislators, researchers, and the public must haveconfidence that psychometrically-sound testing,scoring, and reporting will be handled ethically and inaccordance with the best administrative practices andprocedures.1Dr. Greg Cizek emphasizes that valid testing requiresthe results to be useful, interpretable, accurate, and1comparable. The technical merits of scores froman assessment must meet industry standards withrespect to fairness, reliability, and validity. Of thesestandards, the most important is validity, and cheatingundermines the integrity and validity of the resultsfrom an assessment.OEAA staff fully support the advice of the NCME andother professional organizations on maintaining testintegrity and the validity for the state assessment.Importance of Assessment SecurityThe primary goal of assessment security is to protectthe integrity of the assessment and to ensure thatresults are accurate and meaningful. To ensure thattrends in achievement results can be calculatedacross years in order to provide longitudinal data, acertain number of test questions must be repeatedfrom year to year. If any of these questions are madepublic, the validity of the test may be compromised,because students may know the questions andanswers in advance of taking the assessment. If thereliability or validity of a test is compromised, the testscores of individual students or entire classes may beinvalidated, and disciplinary actions may be taken.Appropriate testing practices are not always universallyunderstood, leading to test irregularities. Goodtesting practices are sometimes violated becausethe individual involved is not informed aboutwhat is appropriate for a standardized assessmentadministration. To help school staff securely administerstate assessments and have a common understandingof what practices are appropriate, the OEAA hasprepared these guidelines.This Michigan State Board of Education (SBE)approved Assessment Integrity Guide includes theexpected professional conduct guidelines of educatorswho administer state assessments and ensure propertest administration and academic integrity. It isintended to be used by districts and schools in theBased on the National Council on Measurement in Education Test and Data Integrity Document, Oct. 2012
MICHIGAN DEPARTMENT OF EDUCATIONASSESSMENT INTEGRITY GUIDEOffice of Educational Assessment and Accountabilityfair and appropriate administration of stateassessments. State assessments are an importantand required tool used to monitor the state,district, school, and student achievement results.For assessments to yield fair, accurate, and validresults, they must be administered under thesame standardized conditions to all students.Assessment Security Goals for the Stateof Michigan Assessment System To adhere to high professional testadministration and security standards To maintain consistency across all testingoccasions and sites (i.e., students andschools) To protect the investments of resources,time, and energyCommon Assessment IrregularitiesIn 2013, the federal Office of Inspector General(OIG) released an audit report entitled TheU.S. Department of Education’s and Five StateEducational Agencies’ Systems of InternalControl over Statewide Test Results. The reportdetailed specifics around assessment securitypolicies and practices for states. The OIG auditfocused on allegations of cheating on statewidetests that have been reported in multiple statesand the District of Columbia. The analysis ofmedia reports on cheating that occurred duringthe previous 10 years indicated that the fivemost prevalent methods of cheating includedthe following: 6To provide secure assessments that resultin valid and reliable scoresusing actual test questions to preparestudents for the tests erasing students’ wrong answers andfilling in the correct answers indicating the correct answers tostudents during testing allowing students to change answersafter giving them the correct answers allowing students to discuss answers witheach otherIn addition, the analysis listed several otheralleged methods of cheating, which included: completing incomplete test booklets altering attendance records failing to cover testing materials duringthe assessments arranging the classroom to facilitatecheating reading questions aloud to students whowere not eligible for that accommodation not testing all eligible students obtaining testing materials when notauthorized to do soThe following table outlines the specific typesof security breaches that can occur for paper/pencil (P/P) test administrations, computerbased testing (CBT), and computer-adaptivetesting (CAT). Asterisks show the potentialrisk to state test administrations or results.Please note that the overall level of securityrisk depends on numerous factors, such as testdesign, item types, item exposure, quality ofproctoring, the testing environment, conflicts ofinterest, methods for transmission and storage,encryption levels, quality of training, and more.
7BEFORE, DURING, AND AFTERLost or stolen bookletsObtaining unauthorized access to secure assessment materialsEducators logging into tests to view questions or change responsesP/P**Hacking into udents giving or receiving unauthorized assistance from other studentsduring a test administration***Teachers providing answers or providing assistance to students duringtesting***Students accessing non-allowable resources (notes, textbooks, theinternet)***Taking photos of test items and sharing them on the internet or socialmedia***Use of actual test questions or answers during the test***Accommodations being used inappropriately to cheat*****P/PCBTCATAltering test scores***Erasing wrong answers***Educators or students engaging others to take a test on a student’sbehalfDURINGKeystroke loggingAFTERChanging responses on the computerReconstructing assessment materials through memorization***Memorized test items or answers being posted online*******Printing, emailing, or storing test information in a computer outside thetest delivery systemAccessing or altering assessment materials or scores during the transferof data*
MICHIGAN DEPARTMENT OF EDUCATIONASSESSMENT INTEGRITY GUIDEOffice of Educational Assessment and AccountabilityAppropriate test preparation activities should promote quality long-term learning. Good test-takingskills and appropriate content learning help to ensure the validity of student test scores.The best way to promote appropriate test administration practices is to ensure that teachers and testadministrators understand and recognize acceptable and unacceptable practices. This document isintended to provide more information to assist schools in developing professional knowledge andclarification on test administration standards for Michigan educational staff and students.Communication ProtocolIn an effort to keep individuals well informed regarding assessment security policies and procedures,the OEAA recommends the following communication protocol to keep staff informed regardingassessment security policies and procedures. The two-way communication at each entity is importantto maintain assessment security.The protocol for communication for assessment security is as follows:DISTRICTDistrict AdministratorMichigan Departmentof EducationDistrictAssessmentCoordinatorOffice of EducationalAssessment rdinatorTestAdministrator/Proctor
9Section 1 – Prevention of TestingIrregularitiesThis section addresses standards for assessmentintegrity and security aspects of the design,development, operation, and administration of paper/pencil and online state assessments, to preventirregularities from occurring. The OEAA requiresdistrict and building coordinators involved in testadministration to complete the following activities: Adhere to the testing administration windowsand testing schedules Provide integrity and security training to allstaff Assign explicit responsibility for assessmentsecurity and monitor the effectiveness of eachschool’s efforts Work with the OEAA when necessary toprevent irregularities Adhere to all test administration rules andpoliciesTesting WindowEach component of the state assessment program (i.e.,M-STEP, Michigan Merit Exams, MI-Access, PSAT 8/9,WIDA Screener, WIDA ACCESS for ELLs, and EarlyLiteracy and Mathematics Benchmark Assessments)has its own Test Administrator Manual (TAM) and itsown testing window. The testing windows provideample opportunity to complete testing while keepingtest items secure. For additional information onthese windows refer to the Testing Schedule forSummative Assessments. (https://www.michigan.gov/documents/mde/Testing Schedule for SummativeAssessments 635008 7.pdf) and the TAM for eachassessment. M-STEP and MI-Access assessments aretypically scheduled as follows: Paper/pencil test administrations occur onthe scheduled day and as early in the dayas possible. Each TAM provides estimatedtest administration times and the testingschedule guidelines that schools must follow inadministering specific tests. Any alteration tothe schedule must be approved in advance bythe OEAA. Online test administrations allow forscheduling flexibility for schools. Online testadministrations can be scheduled at any timeduring the instructional day throughout thedesignated testing window. Students do nothave to be
assessments. If you have questions or wish to report suspected test administration irregularities on any statewide educational assessments or accountability issues in Michigan, contact: Michigan Department of Education. Office Educational Assessment and Accountability 608 West Allegan Street PO Box 30008 Lansing, MI 48915 toll-free phone: 877 .
MDE Performance Measures Historical Annual Summary FY 2005 – 2011 9 MDE Performance Measures Historical Annual Summary FY 2012 – 2018 10 . Per the Budget Reconciliation and Financing Act of 2018 (SB 187/HB 161), MDE reallocated 22 million of this penalty to the General Fund. This year, penalties collected from environmental violators .
Scope of Education Identity & Access Management The current Identity & Access Management system used by MDE is called MIDMS. MIDMS manages user access to state data systems with a user role on behalf of an organization (e.g. school, district, or state agency) in MDE-ORG. MIDMS manages user access to state data systems for MDE and .
100% attendance (required by the Michigan Department of Education (MDE) for SCECH programs), verified by signing in AND out each day. Note: Make-ups are not available. Please complete the MDE online evaluation at michigan.gov/moecs within 30 days of the email notification. Steps in the process: 1.
MICHIGAN STRATEGIC FUND MEMORANDUM DATE: March 15, 2022 TO: The Honorable Gretchen Whitmer, Governor of Michigan Members of the Michigan Legislature FROM: Quentin L. Messer Jr., President, Michigan Strategic Fund SUBJECT: FY 2021 MSF/MEDC Annual Report The Michigan Strategic Fund (MSF) is required to submit an annual report to the governor and the Michigan
In 2012, the Governor and Michigan Legislature passed legislation requiring Michigan Virtual TM, formally Michigan Virtual University , to establish a center for online learning research and innovation. Known as Michigan Virtual Learning Research Institute (MVLRI ), this center is a natural extension of the work of Michigan Virtual .
children and youth engage in self-care. The Michigan Department of Education (MDE) continues to strive forward to become a Top 10 education state in 10 years. To do so, Goal 2 of the MDE’s Top 10 in 10 plan is to implement, with strong district and building leader
- Prepare presentation materials for lay audience. Overarching application theme: assurance for onboard automotive systems 7 Objectives of Course MDE RE SE . Maintenance costs much more than development CSE870: Advanced Software Engineering (Cheng): Intro to Software Engineering 44 Summary MDE RE SE U.S. software is a major part .
Attila has been an Authorized AutoCAD Architecture Instructor since 2008 and teaching AutoCAD Architecture software to future architects at the Department of Architectural Representation of Budapest University of Technology and Economics in Hungary. He also took part in creating various tutorial materials for architecture students. Currently he .
