Discovery Education Assessment K-HS Benchmark Assessments .
Discovery Education AssessmentK-HS Benchmark Assessments in ELA and MathematicsWhat validation data or reports are available? Are technical specifications available for theassessments?Please see the following Technical Manual for validation studies and psychometric information pertaining toDiscovery Education’s interim assessments. Additional reports and technical evidence is available uponrequest.
COMMON CORE INTERIM BENCHMARKTECHNICAL MANUALAPRIL 2013
Discovery Education AssessmentCommon Core Interim Benchmark Technical ManualTable of ContentsI.Introduction: Discovery Education Assessment. 2II.Assessment Standards and Content Validity . 3A.B.C. 3. 4. 7Common Core Standards.Test Development and Review .Web Alignment.III.Test Administration Online. 9IV.Test and Item Scores and Test Reports. 10Test and Item Scores .Interim Assessment Reports.Interim Assessment Reports Examples . 10. 11. 12Reliability, Proficiency Levels, Validity and Growth . 21A.B.C.D.A.B.C.V.Test Reliability .Proficiency Levels.Validity.Vertical Scale Averages and Growth. 21. 22. 23. 26. 29A. Test and Question Statistics, Reliability and Scale Scores.B. Web Alignment Study . 29. 42Appendices.Page 1
I.Introduction: Discovery Education AssessmentDiscovery Education has focused on the use of formative assessments to improve K-12 student learningand performance. Bridging the gap between university research and classroom practice, DiscoveryEducation Assessment offers effective and user-friendly assessment products that provide classroomteachers and students with the feedback needed to strategically adapt their teaching and learning activitiesthroughout the school year.Discovery Education Assessment has pioneered a unique approach to formative assessments using ascientifically research-based continuous improvement model that maps diagnostic assessments to eachstate’s high stakes summative tests. Discovery Education Assessment’s Predictive Test-Specific InterimAssessments are aligned to the content assessed by each summative assessment allowing teachers to trackstudent progress toward the standards and objectives used for accountability purposes.Furthermore, Discovery Education Assessment subscribes to the Standards for Educational andPsychological Testing articulated by the consortium of the American Educational Research Association,the American Psychological Association, and the National Council on Measurement in Education.This technical manual presents information about the Common Core Interim Assessments used during2011-2012 and 2012-2013 school year.Discovery Education Assessment across the United StatesIn 2012-2013, K-2 students in the United States took over 3.3 million Common Core interim benchmarkassessments created by Discovery Education Assessment. These students were found in twenty onedifferent states across the country. As more states transition to the Common Core State Standards, morestudents are being assessed with Discovery Education Assessment’s Common Core Interim BenchmarkAssessments.Page 2
II.Assessment Standards and Content ValidityContent validity evidence shows that test content is appropriate for the particular constructs that arebeing measured. Content validity is measured by agreement among subject matter experts about testmaterial and alignment to state standards, by highly reliable training procedures for item writers, bythorough reviews of test material for accuracy and lack of bias, and by examination of depth ofknowledge of test questions.To ensure content validity of all tests, Discovery Education Assessment carefully aligns the content of itsassessments to a given state’s content standards and the content sampled by the respective high stakestest. Discovery Education Assessment hereby employs one of the leading alignment researchmethodologies, the Webb Alignment Tool (WAT), which has continually supported the alignment ofour tests to state specific content standards both in breadth (i.e., amount of standards and objectivessampled) and depth (i.e., cognitive complexity of standards and objectives). All Discovery EducationAssessment tests are thus state specific and feature matching reporting categories of a given state’slarge-scale assessment used for accountability purposes.Common Core StandardsThe Common Core State Standards Initiative is a state-led effort to establish a shared set of educationalstandards in English language arts and mathematics for grades K-12. The standards are adoptedvoluntarily by state, and are designed to prepare students to be ready to enter college or join theworkforce upon graduation. These standards will not be tested until the 2014-2015 school year.Since the implementation of the Common Core standards, Discovery Education Assessment is workingwith the adopted states to help transition to these new standards from their old state standards. Below arethe Common Core State Standards that can be found on the interim benchmark assessments in Readingand Mathematics, grades 3-8, and Algebra 1 & 2 in High School.Common Core English Language Arts Reporting CategoriesReading: LiteratureEnglish Language Arts Standards : WritingReading: Informational TextEnglish Language Arts Standards: LanguageReading: Foundational SkillsEnglish Language Arts Standards: Listening &SpeakingCommon Core Algebra 1 & 2 Reporting CategoriesNumber & QuantityFunctionsAlgebraStatistics & ProbabilityPage 3
Common Core Mathematics Reporting CategoriesOperations & Algebraic ThinkingRatios & Proportional RelationshipsNumber & Operations in Base TenThe Number SystemNumber & Operations—FractionsExpressions & EquationsMeasurement & DataGeometryStatistics & ProbabilityTest Development and ReviewAlignmentFor the last nine years, Discovery Education has led in assuring educators that its items are alignedspecifically to each state’s curriculum or district pacing guide. Every change in a state’s curriculum wascarefully identified and items changed to meet the current year’s statewide assessment.Alignment is done by trained test developers who have subject matter expertise, teaching experience inthe grade/subject, and the assessment expertise to produce appropriate items. Each individual hascertification in the grade/subject and has at least three years teaching experience in the area. Thealignment is managed within our software and mapped one item at a time. This is not a softwarealignment but rather a teacher who compares each item to the state standard to determine if it isaligned.We begin by matching our tests to the Common Core State Standards that are assessed. We agree that forteachers to have confidence in the results, they must be assured that an assessment is aligned to theirstandards. The test has to exhibit content validity, which is demonstrated when test items represent thesubject area, such as math or reading. In other words, a math assessment must have items that match oralign to the Common Core Standards and benchmarks defined by a state’s curriculum and high stakes’test. Difficulty levels are based on actual prior student performance and provide teachers with a crucialcomparison of how a current class or individual student compares to what is generally expected of studentperformance on these items.Discovery Education Assessment pioneered a unique approach to benchmark assessments using ascientifically research-based continuous improvement model that maps diagnostic assessments to eachstate’s high stakes test. Discovery Education’s Predictive Test-Specific Benchmark tests are aligned tothe content assessed by each state test allowing teachers to track student progress toward the standardsand objectives used for accountability purposes. This same predictive approach is being applied to statetests as they transition to Common Core assessments.Page 4
ItemsDiscovery Education employs only certified, experienced teachers with content majors and master’sdegrees to align, create, and develop items; the content review, copy editing, and quality controldepartments are also staffed by competent, qualified teachers with graduate degrees. Discovery Educationintentionally employs teachers with familiarity in varied subjects, age groups, and ability levels, whichgives the item development teams an impressive range of expertise. They work with psychometric staff toreview and systematically match items to Common Core standards. All items have appropriatepsychometric properties from field testing that permit accurate, valid, and reliable predictive tests. Ourcommitment to rely on competent, seasoned educators throughout the entire development process assuresthat items and tests are accurate, appropriate, and accessible.Discovery Education content team begins the test development process with the state standards, testblueprints and test specifications. We begin by matching our items to the Common Core standards thatare assessed.No software tool can sufficiently match items to state standards. Curriculum experts must do this job oneitem at a time. Discovery Education Assessment’s software facilitates the curriculum expert’s job ofaligning each item to a Common Core standard. This task is re-aligned every year, using prior yearstudent performance statistics to assure continuous alignment, reliability, and validity. All Common Corestandards are loaded into the Discovery Education Assessment tool, which allows our curriculum expertsto build state-specific tests. Item notes and field test data are available at the time of item selection foreach grade and subject test.Bias StatisticsAll Discovery Education assessments incorporate a systematic, official statistical bias analysis, usingRasch analysis on gender, ethnicity, and differing abilities. Discovery Education Assessment iscommitted to assuring students, teachers, and administrators that we are sensitive to and cognizant of theneed for assessments to be bias-free. While it is important to know that a test measures what it ispurported to measure, it is just as important to know what a test does not measure. Discovery Educationassessments are designed and reviewed to guard against culture or gender bias and to address issues ofdisability.Types of ItemsDiscovery Education assessments feature multiple choice questions that measure the maximum range ofcognitive skills in the content areas. Using multiple choice questions reduces the cost, test taking time,and provides immediate results with diagnostic and predictive capabilities. However, we generallyinclude constructive response open response and performance tasks with the assessments that could bemanually scored and used by teachers in the classrooms for formative purposes. In addition, DiscoveryEducation services provide access to performance tasks and tools for locally created items.Page 5
Refreshing Item PoolDiscovery Education Assessment routinely works with schools to field test new benchmark items toreplenish our pool of available items. This approach assures that we continuously have field tested itemsavailable. This improvement process also assists our test coordinators to receive specific feedback onevery item, thereby further increasing the reliability of each item included.Quality ControlQuality control is a crucial aspect of Discovery Education Assessment’s approach to item construction.The quality control process ensures that every component of a Discovery product is consistent andaccurate within and across printed or online versions of tests. The quality control departmentdetermines that the online and printed versions of tests match exactly, that reports are accessible andcomplete, and that scores on Discovery Education tests are correlated to the proficiency or masteryspecifications provided by the state. A key role of the quality control department involves testing thefunctionality of the online interface and examining reports for accuracy.Copy Editing & ProofreadingA vital part of the development team is the finalization team, responsible for ensuring the typographicalaccuracy of all assessments. The copy editing personnel proofread the assessments after they are releasedfrom the content review team. The editors correct any typographical or mechanical errors that appear inthe test, and they also look for errors in the layout and placement of graphics, instructions, page numbers,or margins. Copy editors also perform a final examination of formatting to ensure that each test isformatted to match the state’s high stakes tests’ formatting. When the assessment is proofread andnecessary changes are made, the test is converted into PDF format, then proofread again. If any errors stillexist, they are corrected before a copy of the final test is posted for printing. The proof from the printer isthen edited one final time. These multiple instances of thorough proofreading enable Discovery Educationto produce tests that are not only exemplary in content but also accurate in grammar and mechanics.Overview of Benchmark Item Review StepsItems are reviewed multiple times by experienced teachers, a psychometrician, a grammarexpert, and Director of Testing for spelling errors, errors in usage, and awkward phrasing.All items are reviewed by at least 2 reviewers to have a single correct answer and appropriatedistracters.All items are reviewed for grade appropriateness in content and readability, using FleschKincaid scale.Items have field test and actual administration data to support reliability of grade level in termsof difficulty and content validity.Item Depth of Knowledge (DOK) is reviewed and displayed for educators on reports.Page 6
Where items are expected to include vocabulary “above grade level,” items are reviewed toassure that context clues are on grade level.In math items, required computations are reviewed to be appropriate for grade level andappropriate to the time-constraints of the formative environment.Items that require critical thinking skills are measured in terms of steps required and thedifficulty levels to provide an appropriate mix across the skill area.Many items require graphics to support the thinking skill measurement. These items arereviewed to assure that the graphics print and appear on the web with clarity, appropriate levelof detail, and appropriate grade level to measure the skill.Graphic items are kept simple enough to assure web display within 2-3 seconds but complexenough to measure the skill or accomplishment being tested.Item answers are varied to distribute them randomly across answer options A, B, C, and D sothat no discernable pattern is possible for correct answers.Web AlignmentDiscovery Education contracted with an independent research team, Test Prep, led by Dr. Michael K.Smith in April 2012 to conduct a Web alignment study of the Discovery Education interim benchmarkassessments with the Common Core State Standards. This process is repeated after any revisions aremade to the interim benchmark assessments.The WAT (Web Alignment Tool) version 2 was used to measure categorical concurrence, depth-ofknowledge consistency, range-of-knowledge correspondence and balance of representation. A summaryof the results are below. Results of this study are being used to make appropriate revisions to the 20122013 interim benchmarks. A plan of action has already been set in place to increase depth-of-knowledgeconsistency in the reading assessments and the categorical concurrence in the mathematics assessments.For more details on this alignment study, please see Appendix B for the document Web Alignment Studyof Discovery Education Assessment Benchmarks with Common Core Standards.Page 7
Mathematics Tests Alignment ceDepth-of-KnowledgeConsistencyRange of KnowledgeBalance ofRepresentationReading Tests Alignment nceDepth-of-KnowledgeConsistencyRange of KnowledgeBalance ofRepresentationPage 8
III.Test Administration OnlineTo administer interim benchmark tests online an administrator must first import students to the DiscoveryEducation website (www.discoveryeducation.com). Once students are imported and populated into gradepool and teacher classes, testing can begin at any time after the recommended window opens. For stepson importing students, populating classes, and administering interim benchmarks online, please refer tothe Assessment User Guide. This document is located at: http://assessment.discoveryeducation.com/startand on the help site at discoveryeducation.com.Page 9
IV.Test and Item Scores and Test ReportsTest and Item ScoresDiscovery Education Assessment reports the following item and test scores on its Interim Assessments.Student Level Scores:Test Percent Correct:The percent correct on an interim benchmarkTest Number Correct:The number correct on an interim benchmarkReporting Category Percent Correct:The percent correct on a particular reporting categoryReporting Category Number Correct:The number correct on a particular reporting categoryVertical Scale Score:A scale score on a 1000 to 2000 scaleState Percentile:The percent of students that score lower than a particular scalescore. The state percentile is based on all students in a particularstate that completed an interim assessmentNational Percentile:The percent of students that score lower than a particular scalescore. The national percentile is based on a stratified randomnational sampling of students who completed an interimassessmentProficiency Prediction:A prediction of the level of student performanceClass, Grade, School, and District Level Scores:Item Percent Correct:The percent of students in a class, grade, school or district thatanswer an item correctlyItem Percent Incorrect:The percent of students in a class, grade, school, or district thatanswer an item incorrectlyTest Mean Number Correct:The arithmetic mean (average) of the number correct on aninterim assessment for a class, grade, school, or districtTest Mean Percent Correct:The arithmetic mean (average) of the percent correct on aninterim assessment for a class, grade, school, or districtVertical Scale Mean:The arithmetic mean (average) of the vertical scale score on aninterim assessment for a class, grade, school, or districtPage 10
Proficiency Level Number:The number of students in each proficiency level on an interimassessment for a class, grade, school, or districtProficiency Percent:The percent of students in each proficiency level on an interimassessment for a class, grade, school, or districtMedian State Percentile:The middle value of all state percentiles for a school or districtInterim Assessment ReportsDiscovery Education produces multiple standard report formats to coincide with each interim assessment,as well as dynamic district reports. Interactive reports are linked to recommended digital remediationselected from Discovery Education’s award-winning streaming. Discovery Education recognizes thetremendous value in prompt, easy to read reports that allow all stakeholders, including students, parents,teachers, administrators and district staff, to instantly determine how learning is progressing. While thesoftware and standard reports are not customizable, all reports allow easy export of data to CSV orEXCEL file formats for flexible reporting or import into other reporting tools.Discovery Education Assessment reports the following:For Teachers:Proficiency predictions by subject point to which students are at risk.Predictions of proficiency levels within each Common Core standard define specificallywhat to focus on in remediation.Performance Indicator results viewed with the test items to define detailed gaps in studentthinking processes.Growth of student performance across time and comparable across grade, schoo
the Common Core State Standards that can be found on the interim benchmark assessments in Reading and Mathematics, grades 3-8, and Algebra 1 & 2 in High School. Common Core English Language Arts Reporting Categories Reading: Literature English Language Arts Standards : Writing Reading: Informational Text English Language Arts Standards: Language
PRICE LIST 09_2011 rev3. English. 2 Price list 09_2011. 3. MMA. Discovery 150TP - Multipower 184. 4 Discovery 200S - Discovery 250: 5 Discovery 400 - Discovery 500: 6: TIG DC: Discovery 161T - Discovery 171T MAX 7: Multipower 204T - Discovery 220T 8: Discovery 203T MAX - Discovery 300T 9: Pioneer 321 T 10:
I. Introduction: Discovery Education Assessment Discovery Education has focused on the use of formative assessments to improve K-12 student learning and performance. Bridging the gap between university research and classroom practice, Discovery Education Assessment offers effective and user-friendly assessment products that provide classroom
About Discovery Education As part of Discovery Communications (Discovery Channel, Animal Planet and many more), Discovery Education provides engaging digital resources
BGP support 21 Discovery overview 21 IP Availability Manager discovery 23 MPLS Topology Server discovery 24 Imports topology from IP Availability Manager 25 . Relationships between the L2VPN, MPLS, and transport models 94 6 Discovery of L3VPN Objects 96 L3VPN discovery ove
Discovery HD Theater are delivered in 1080i format. Discovery HD Theater offers some ofthe best Discovery programs,such as Tomb ofthe LostPharaohs. Discovery HD Theater also showcases, for the first time in HD, popular Discovery Networks programming such as the Discovery Channel's docume
ProSAFE etwork Management System Data Sheet NMS300 Discovery and Registration Automated Device Discovery Includes top-level, subcomponents and interfaces/ports as applicable Automated Link Discovery Ethernet link discovery with LLDP protocol Discovery Scheduling Ability to schedule discovery tasks to be executed at specified time/date(s)
4 Electronic Discovery. 2/2/2011. Introduction to Electronic Discovery Electronic Discovery, or "e -discovery", is simply the extension of discovery to include data in . electronic format. referred to as Electronically Stored Information "ESI". Recent University of California at Berkeley Analysis : 93% of data created in 2003 is digital;
3 www.understandquran.com ‡m wQwb‡q †bq, †K‡o †bq (ف ط خ) rُ sَ _ْ یَ hLbB َ 9 آُ Zviv P‡j, nv‡U (ي ش م) اْ \َ َ hLb .:اذَإِ AÜKvi nq (م ل ظ) َ9َmْ أَ Zviv uvovj اْ ُ Kَ hw ْ َ Pvb (ء ي ش) ءَ Cﺵَ mewKQy ءٍ ْdﺵَ bِّ آُ kw³kvjx, ¶gZvevb ٌ یْ"ِKَ i“Kz- 3
