Retrieval Practice With Short-answer, Multiple-choice, And .

2y ago
56 Views
2 Downloads
327.80 KB
20 Pages
Last View : 1d ago
Last Download : 2m ago
Upload by : Olive Grimm
Transcription

MemoryPublication details, including instructions for authors and subscription etrieval practice with short-answer, multiplechoice, and hybrid testsaMegan A. Smith & Jeffrey D. KarpickeaaDepartment of Psychological Sciences, Purdue University, West Lafayette, IN, USAPublished online: 23 Sep 2013.To cite this article: Megan A. Smith & Jeffrey D. Karpicke (2014) Retrieval practice with short-answer, multiplechoice, and hybrid tests, Memory, 22:7, 784-802, DOI: 10.1080/09658211.2013.831454To link to this article: SE SCROLL DOWN FOR ARTICLETaylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy, completeness, or suitabilityfor any purpose of the Content. Any opinions and views expressed in this publication are the opinionsand views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy ofthe Content should not be relied upon and should be independently verified with primary sources ofinformation. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands,costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of the Content.This article may be used for research, teaching, and private study purposes. Any substantial orsystematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distributionin any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found ions

Memory, 2014Vol. 22, No. 7, 784 802, ieval practice with short-answer, multiple-choice,and hybrid testsMegan A. Smith and Jeffrey D. KarpickeDepartment of Psychological Sciences, Purdue University, West Lafayette, IN, USA(Received 29 May 2013; accepted 29 July 2013)Retrieval practice improves meaningful learning, and the most frequent way of implementing retrievalpractice in classrooms is to have students answer questions. In four experiments (N 372) we investigated the effects of different question formats on learning. Students read educational texts and practisedretrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions studentsfirst attempted to recall answers in short-answer format, then identified answers in multiple-choiceformat. We measured learning 1 week later using a final assessment with two types of questions: thosethat could be answered by recalling information verbatim from the texts and those that requiredinferences. Practising retrieval in all format conditions enhanced retention, relative to a study-onlycontrol condition, on both verbatim and inference questions. However, there were little or no advantagesof answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, therewas an advantage of answering short-answer or hybrid questions over multiple-choice questions. Theresults challenge the simple conclusion that short-answer questions always produce the best learning, dueto increased retrieval effort or difficulty, and demonstrate the importance of retrieval success forretrieval-based learning activities.Keywords: Retrieval practice; Testing effect; Learning; Question format; Short-answer; Multiple-choice.Practising retrieval is an effective strategy toenhance meaningful learning (e.g., Karpicke &Blunt, 2011). Retrieval practice can be implemented in the classroom through many activities, but most research to date has focused onfrequent testing and quizzing in the classroom(e.g., McDaniel, Agarwal, Huelser, McDermott,& Roediger, 2011). If retrieval practice is to beimplemented in the classroom, then it is important to know which retrieval practice formats aremost effective for promoting meaningful learning.The purpose of this paper is to examine theeffectiveness of various retrieval practice formatson long-term meaningful learning.Past research has focused primarily on retrievalpractice via short-answer and multiple-choice questions because these formats are frequently employed in the classroom. Multiple-choice questionsrequire students to recognise and select a correctresponse among alternatives, while short-answerAddress correspondence to: Megan A. Smith, Department of Psychological Sciences, Purdue University, 703 Third Street,West Lafayette, IN 47907, USA. E-mail: smith598@purdue.eduThe writing of this paper was supported in part by grants from the National Science Foundation (DUE-0941170 and DRL1149363) and the Institute of Education Sciences in the US Department of Education (R305A110903). The opinions expressed arethose of the authors and do not represent views of the Institute or the US Department of Education. A portion of this project was anundergraduate honours thesis completed by Megan Smith under the instruction of Jeffrey Karpicke. We wish to thank AltheaBauernschmidt for numerous constructive discussions, Emily Boyne, Cathrine Brattain, Kait Cross, Kelli Olifirowicz, SamanthaOstler, Victor Panfil, and Nikita Saoji for help with data collection and scoring, and Philip Grimaldi for technical assistance.# 2013 Taylor & Francis

RETRIEVAL PRACTICEquestions require students to recall and produceresponses. Multiple-choice and short-answerquestions most readily test over verbatim orfactual information, but can also be used to testhigher-level concepts from Bloom’s (1956) Taxonomy (Marsh, Roediger, Bjork, & Bjork, 2007).For example, they can be used for inferencequestions that require students to put informationthat they have learned together, and for application questions where students are required to takewhat they have learned and apply it to a newcontext. From an instructor’s perspective, multiple-choice questions have advantages. Relative toshort-answer questions, multiple-choice questionsare easier to administer and grade. This is especiallytrue with the availability of clickers and onlinetesting systems, which can be used to administerand score multiple-choice questions in large classrooms (e.g., Mayer et al., 2009; McDaniel,Anderson, Derbish, & Morrisette, 2007). Conversely short-answer questions are more difficult toadminister, and take much more time to grade.However, some research has shown thatretrieval practice using short-answer questionsbenefits learning more than multiple-choice questions. Short-answer questions require students toengage in more effortful and complete retrievalpractice than multiple-choice questions, and moreeffortful retrieval has been theorised to explainwhy retrieval practice is effective (Pyc & Rawson,2009). For example, Kang, McDermott, andRoediger (2007) had subjects study journal articles and then answer initial short-answer questions or answer initial multiple-choice questions.Subjects received feedback by viewing the correctanswer to each question after they provided theiranswer. Three days later, subjects returned andtook a final retention assessment in both shortanswer and multiple-choice formats. On the finalmultiple-choice assessment, answering initialshort-answer questions produced greater performance than answering initial multiple-choicequestions. On a final short-answer assessment,answering initial short-answer questions producednumerically greater performance relative to answering initial multiple-choice questions, but thiseffect did not reach statistical significance. Importantly, the authors concluded that feedbackwas crucial in order to find these differences. Inan experiment where feedback was not provided,practising retrieval by answering short-answerquestions did not result in the best learningoutcomes. The lack of a short-answer benefit islikely because initial retrieval success is fre-785quently lower on short-answer questions than onmultiple-choice questions, disadvantaging theshort-answer group. Kang and his colleaguesargued that feedback made up for the initialsuccess differences between the two formats byensuring that all subjects saw the correct answers.Other experiments have reported a retentionbenefit for practising retrieval with shortanswer questions relative to multiple-choice questions (e.g., Butler & Roediger, 2007; McDaniel,Anderson et al., 2007; Clariana, 2003; Duchastel,1981; Gay, 1980; for similar findings in the adjunctquestions literature see also Anderson & Biddle,1975; Hamaker, 1986; Williams, 1965). In addition,answering multiple-choice questions exposes students to false information by presenting luresalong with the correct answer, and there is someevidence that this can lead to retention of thisfalse information (e.g., Roediger & Marsh, 2005).Results such as these have led to the conclusionthat practising retrieval with short-answer questions is superior to multiple-choice questions forenhancing student learning, even if these tests aremore difficult to administer.While some have found a retention advantagefor practising retrieval with short-answer questions over multiple-choice questions, the effectseems to occur only under specific circumstances.As noted above, Kang et al. (2007) reported thatthe difference between their short-answer andmultiple-choice conditions did not reach the levelof significance when the final assessment was inthe short-answer format. Conversely, Gay (1980)reported the opposite result. Gay had subjectsrepeatedly practice retrieval with short-answeror multiple-choice questions over material ina college course and found that short-answerquestions led to superior learning only when thefinal assessment was in short-answer format.When the final assessment was in multiple-choiceformat, no differences between the initial retrieval practice formats were found. It is possible toexplain these results using the transfer-appropriate processing framework (Morris, Bransford, &Franks, 1977). The transfer-appropriate processing framework posits that memory will be bestwhen the processing necessary at the time ofretrieval matches the processing that was necessary at the time of encoding. The transfer-appropriate processing framework would predict thatretrieval practice via short-answer tests show thegreatest advantage over other formats whenthe final test is in short-answer format. Whilethis explanation seems plausible for Gay’s study,

786SMITH AND KARPICKEthe opposite was found to be true in Kang andcolleagues’ (2007) studies, leading to the conclusion that transfer appropriate processing cannotexplain format differences.Furthermore, there have also been studiesthat have failed to find a retention advantage ofretrieval practice with short-answer over multiple-choice questions at all (e.g., Clariana & Lee,2001; Duchastel & Nungester, 1982; Frase, 1968;Haynie, 1994; Williams, 1963). It is possible toexplain these results; for example, Clariana andLee (2001) and Williams (1963) found trendsfavouring short-answer over multiple-choicetests but their results were not statistically significant. In addition, Haynie (1994), Duchasteland Nungester (1982), and Frase (1968) did notprovide feedback to their subjects. Recall thatKang and colleagues (2007) reported that feedback is necessary for retrieval practice with shortanswer questions to produce more learning thanmultiple-choice questions. Taken together, theseresults cast doubt on the retrieval practice advantage of short-answer tests, or suggest that theeffect only occurs under specific circumstances.The benefit from retrieval practice might notdepend on the format of the retrieval activity, butmight instead depend on both retrieval difficultyand retrieval success. Some have argued thatretrieval difficulty is the reason short-answerquestions can produce greater retention benefitsover multiple-choice questions (e.g., McDaniel,Roediger, & McDermott, 2007). The more effortful the retrieval practice, the greater the benefit.Still, success during retrieval practice is alsoimportant for final retention (Butler, Marsh,Goode, & Roediger, 2006; Marsh, Agarwal, &Roediger, 2009). Thus if initial retrieval practice istoo difficult and retrieval success is low, laterretention will likely suffer. Practising retrievalwith multiple-choice questions often leads togreater success relative to short-answer questions.It is possible that both retrieval difficulty andretrieval success determine whether a retrievalactivity will be better in a given situation. Theeffects of retrieval practice will be best whenretrieval is both difficult and successful (Pyc &Rawson, 2009).One solution to these problems is to combineshort-answer and multiple-choice formats, whichwe refer to as hybrid formats. Combining shortanswer and multiple-choice formats should leadto a benefit due to the more effortful retrievalduring short-answer responding, but greater success during multiple-choice responding. In educa-tional settings these hybrid formats should haveyet another benefit: the multiple-choice questionsmake them easier to administer and score. Park(2005) investigated the effectiveness of a hybridtesting format with sixth-grade students. Hecreated a computerised retrieval practice systemto combine the benefits of both short-answer andmultiple-choice formats (see also Park & Choi,2008). The computer first presented the questionto students without alternatives so they couldrespond as if they were answering a short-answerquestion. Then, when the students were ready,multiple-choice alternatives appeared so theycould find and select the answer they had alreadyretrieved. Using this format for retrieval practiceallows for quick objective scoring while stillretaining the retention benefits of short-answerretrieval practice. Park found that sixth-gradestudents who practised retrieval via this newhybrid format performed better on a final assessment 4 days later relative to students whopractised retrieval in the standard multiple-choiceformat. Because hybrid formats combine shortanswer and multiple-choice questions they maybe especially effective formats for improvingmeaningful learning; however, aside from Park’spapers there has been very little research on thistopic (for one exception see Butler, Huelser,Caruso, & Roediger, 2008).The purpose of these experiments was toexamine retrieval practice with hybrid formats,which could hold great potential for computerbased retrieval practice systems to improve meaningful learning. One of our goals was to replicatethe finding that retrieval practice with a hybridformat produces more meaningful learning than amultiple-choice format, and to examine whether ahybrid format might produce more learning andretention than a short-answer format. Anothergoal was to see whether we could enhance theeffects of hybrid formats by introducing spacingbetween the two retrieval attempts. In the originalhybrid format (Park, 2005; Park & Choi, 2008)the questions were answered in the short-answerand then multiple-choice format immediatelyafter one another. It is possible that spacing therepetition of questions within the hybrid formatwould result in greater benefits than the wayhybrid formats were implemented in prior research (for a review and discussion of the spacingeffect in learning and memory research seeCepeda, Pashler, Vul, Wixted, & Rohrer, 2006).Spacing retrieval practice has been shown toimprove memory (e.g., Karpicke & Bauernsch-

RETRIEVAL PRACTICEmidt, 2011), and spacing the same question mightallow students to practise retrieval twice insteadof just remembering the answer (see Jacoby, 1978).When students engage in repeated retrieval, theirperformance increases (e.g., Karpicke & Roediger,2008). To meet this goal we examined two hybridformats: a hybrid-massed format that was similarto that of Park (2005; see also Park & Choi, 2008)and a hybrid-spaced format that introduced spacedrepetitions. Finally, a third goal was to measurethe percentage of lures from the multiple-choiceformat that are produced on the final assessmentin Park’s method (see Roediger & Marsh, 2005).The four experiments reported here examinedthe relative benefits of different retrieval practiceformats on learning of meaningful educationalmaterials. Importantly, during retrieval practicestudents were required to answer questions tapping conceptual knowledge that was directlystated in the text (verbatim questions) and makeinferences connecting more than one concept inthe text (inference questions, Experiments 1, 2,and 3; see Karpicke & Blunt, 2011). We asked ifretrieval practice with short-answer questionswould produce greater meaningful learning relative to multiple-choice questions, and whether acombination of the two formats would produceeven greater retention. Subjects read educationally relevant texts, and then practised retrievalwith one of four initial retrieval formats: astandard short-answer format, a standard multiple-choice format, or a hybrid format. Two hybridformats were examined. The first was the hybridmassed format where the short-answer and multiple-choice presentations occurred one right afterthe other (Park, 2005). The second was a hybridspaced format where the short-answer and multiple-choice presentations were spaced apart fromone another. All subjects received feedback afterpractising retrieval (Kang et al., 2007). In additionwe assessed the relative difficulty of our retrievalformats by recording response times to answerquestions (see Benjamin, Bjork, & Schwartz,1998; Gardiner, Craik, & Bleasdale, 1973; Karpicke & Bauernschmidt, 2011; Karpicke & Roediger, 2007a; Pyc & Rawson, 2009). One of thereasons retrieval practice is thought to be morebeneficial with short-answer questions than withmultiple-choice questions is that short-answerquestions are thought to induce more difficultand complete forms of retrieval (see McDaniel,Roediger, et al., 2007). We recorded responsetimes to assess whether our short-answer format787was more difficult than our multiple-choice format.All subjects answered final short-answer questions (Experiments 1, 2, 3, and 4) or final multiple-choice questions (Experiment 4) to assessmeaningful learning of the materials. The finalassessment was primarily in short-answer formatto require subjects to produce what they learnedand not just recognise correct answers on the finalassessment. However, in Experiment 4 we examined performance on both a final short-answerand multiple-choice assessment. During the finalassessment students again answered verbatimquestions directly tapping conceptual knowledgeand inference questions requiring students tocombine and integrate information. By includingboth direct conceptual questions and questionsrequiring inferences on the final assessment, wewere able to ask whether different retrievalpractice formats interacted with the type ofknowledge being assessed, which would haveimportant implications for educators seeking toimprove meaningful learning. Experiment 4 onlyincluded verbatim questions on the final assessment, and the reasons for this will be discussedlater.EXPERIMENT 1MethodSubjects. A total of 80 Purdue Universityundergraduates participated in Experiment 1.All subjects were native speakers of English.Materials. Text materials and questions weretaken from Butler, Flanagan, Roediger, andMcDaniel (2007). The materials consisted offour texts, each organised into four paragraphs. An example text is provided in the Appendix. Each text covered a single topic: Venice(540 words), Galileo (534 words), First Crusade(590 words), and KGB (568 words). The order inwhich the texts were presented was held constantfor all subjects (Venice, Galileo, First Crusade, and

Retrieval practice with short-answer, multiple-choice, and hybrid tests Megan A. Smith and Jeffrey D. Karpicke Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA (Received 29 May 2013; accepted 29 July 2013) Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval

Related Documents:

Manipulations of Initial Retrieval Practice Conditions 7 Retrieval Practice Compared to Restudy and Elaborative Study 7 Comparisons of Recall, Recognition, and Initial Retrieval Cueing Conditions 8 Retrieval Practice With Initial Short-Answer and Multiple-Choice Tests 9 Positive and Negative Effects of Initial Multiple-Choice Questions 11

The 7 Basic Principles of Retrieval Practice Following are the seven basic principles of retrieval practice. 1. Keep It Short and Simple Retrieval practice should only take a few of minutes of class time and should be easy to explain, set up, and conclude. A perfect example is Agarwal and Bain’s (2019) retrieval

Retrieval Practice – Why Karpicke & Roediger, 2008 Learning pairs of words (Swahili – English) 15 Retrieval Practice – Why Retrieval Practice strengthens memory and interrupts forgetting. Retrieval Practice makes that knowledge easier to retrieve in the future. Neural pathways that make up a body of learning get stronger. 16

Retrieval practice in its many forms (clickers, mini-quizzes, practice problems, and so on) is excellent for improving learning. As discussed in our guide, How to Use Retrieval Practice to Improve Learning, . another, such as from short answer to multiple-choice. This includes all major question formats.

[B]. RETRIEVAL PHASE The retrieval phase is the reverse process of the storage phase. In this phase another automatic monorail will arrive at the retrieval reference point without any load (package) on it. The proximity sensor will sense it, the sensor will change to on state which sends the signal to PLC alerting it about the request of retrieval.

onstrating that retrieval practice is actually a powerful mnemonic enhancer, often producing large gains in long-term retention relative to repeated studying. Re-trieval practice is often effective even without feedback (i.e. giving the correct answer), but feedback enhances the benefits of testing. In addition, retrieval practice

MYSTERY EXPRESSION GAME!!!!! i. Twice x Answer I: ii. 9 less than the Answer I Answer II: iii. The sum of Answer II and the product of 5 and t Answer III: iv. Answer III decreased by two-fifths the cube of another number Answer IV: v. Half of Answer IV Answer V: ANSWER V is E

Tank 6 API-653 In-Service, Internal Inspection Report less severe corrosion than the west perimeter. The average thickness of the sketch plates away from the west perimeter was 0.281”. Other than the perimeter corrosion noted, the remainder of the tank bottom showed no signs of significant metal loss and the thickness readings appeared consistent with the readings from the 2004 robotic .