An Introduct An Introduction To Educational Design Research
An Introduction to Educational Design ResearchSLO is the Netherlands institute for curriculum development.We are bridging the contexts of policy, research, and practice.Our expertise focuses on the development of curricular goalsand content for various educational levels, from nationalpolicy to classroom practices. We closely collaborate withmany different stakeholders from policy circles, schools(boards, principals, teachers), research, civic organizations,and the society at large. This allows us to design and validaterelevant curriculum frameworks, to elaborate exemplarymaterials and to evaluate these in school practices. Ourproducts and services support both policy makers andschools and teachers in making substantive curriculardecisions and in elaborating these into relevant, inspiringand effective education.An Introductionto EducationalDesign ResearchISBN: 978 90 329 2329 7PO box 20417500 CA EnschedeThe NetherlandsT 31(0)53 484 08 40F 31(0)53 430 76 92E info@slo.nlwww.slo.nlSLO Netherlands institute for curriculum developmentEditors: Tjeerd Plomp & Nienke NieveenSLOEditors:Tjeerd Plomp & Nienke Nieveen
An Introductionto EducationalDesign ResearchProceedings of the seminar conducted atthe East China Normal University, Shanghai (PR China),November 23-26, 2007SLO Netherlands institute for curriculum developmentTjeerd Plomp & Nienke Nieveen (editors)
ColophonAuthors:Jan van den AkkerBrenda BannanAnthony E. KellyNienke NieveenTjeerd PlompEditors:Tjeerd PlompDesign:Axis Media-ontwerpers, EnschedeProduction:Netzodruk, EnschedeAN:7.5115.183ISBN:978 90 329 2329 7Order address:SLONienke NieveenP.O. Box 20417500 Enschedethe blicationsEnschede, 3rd print March 20102an introduction to educational design research
ContentsPreface51.Educational Design Research: an Introduction92.Curriculum Design Research3.The Integrative Learning Design Framework: An Illustrated Example fromTjeerd Plomp37Jan van den Akkerthe Domain of Instructional Technology534When is Design Research Appropriate?735Formative Evaluation in Educational Design Research896References and Sources on Educational Design ResearchBrenda BannanAnthony E. KellyNienke NieveenTjeerd Plomp and Nienke NieveenAuthor biographies103125an introduction to educational design research3
PrefaceThis book is the result of a seminar on ‘educational design research’ organized from November 23-26, 2007, by Prof Zhu Zhiting (Department of Educational Technology) of the Collegeof Educational Sciences at the East China Normal University in Shanghai (PR China).The primary goal of the seminar was to introduce a group postgraduate students and lectu-ring staff in China to educational design research as a research approach. The second goal ofthe seminar was to prepare, based on the contributions of a number international experts,proceedings of the seminar written in such a way that they can be used in postgraduateseminars on educational design research across China.About 75 people with backgrounds mainly in instructional technology, curriculum andinstructional design participated in the seminar. Most of them were working in teachereducation, in schools as instructional technologist and/or in distance education. Althoughparticipants had (through their studies) already knowledge and some experience in in-structional or course design and in research methods, they were eager to be introduced todesign research as a relatively new research approach for addressing complex problems ineducational practice.The seminar staff consisted of Profs Brenda Bannan and Eamonn Kelly (both George MasonUniversity, Fairfax, VA, USA) and Prof Jan van den Akker (University of Twente and NationalInstitute for Curriculum Development [SLO], Enschede, The Netherlands), and the two editors of this book Dr Nienke Nieveen (National Institute for Curriculum Development [SLO],Enschede) and Prof Tjeerd Plomp (University of Twente, Enschede, The Netherlands). As canbe seen from the table of content of this book, they are reflecting the background of the participants, as they represented experience in conducting design research in the domains ofcurriculum development, instructional technology and mathematics and science education.Experts were consciously invited from both Europe (The Netherlands) as well as the USA, soas to ascertain that variation in background and perspective on design research was represented in conducting the seminar.The chapters in this book are based on the presentations and the small group discussionsduring this seminar. Although the book does not provide a ‘how to do guide’ for designingand conducting design research, the chapters have been written in such a way that theyreflect both the conceptual underpinning and practical aspects of the ‘what’ and ‘how’ ofdoing design research (chapters by Plomp, Kelly and Nieveen), as well as provide the readeran insight in the specifics of doing design research in the domain of curriculum (chapter byVan den Akker) and instructional technology (chapter by Bannan).To assist the readers in finding their way in the abundance of literature on design research,we have added a chapter with references and sources on educational design research. Thisan introduction to educational design research5
bibliography is far from complete and reflects very much the background and the biasesof the editors of this book. Yet we trust that this chapter will assist the interested reader ingetting introduced to this exciting and promising research approach.We want to thank Prof Zhu Zhiting from the East China Normal University for taking theinitiative for this seminar. Similarly we want to thank our colleagues for contributing tothis book.But above all, we like to express our hope that this book will stimulate and support many(future) researchers to engage themselves in educational design research.Jan van den AkkerDirector General SLOTjeerd Plomp and Nienke NieveenEditors6an introduction to educational design research
an introduction to educational design research7
1. Educational Design Research:an IntroductionTjeerd PlompIntroductionThe purpose of this chapter is to provide an introduction to educational design research as aresearch approach suitable to address complex problems in educational practice for whichno clear guidelines for solutions are available. Educational design research is perceived asthe systematic study of designing, developing and evaluating educational interventions, such as programs, teaching-learning strategies and materials, products and systems - assolutions to such problems, which also aims at advancing our knowledge about thecharacteristics of these interventions and the processes to design and develop them.The need for a research approach that addresses complex problems in educational practicehas been argued by researchers in various ‘corners’ of the domain of education from thelack of relevance of much educational research for educational practice. For example, theDesign-Based Research Collective (2003:5) argues that educational research is oftendivorced from the problems and issues of everyday practice – a split that resulted in acredibility gap and creates a need for new research approaches that speak directly toproblems of practice and that lead to the development of ‘usable knowledge’.From his background in research in the domain of curriculum development andimplementation, Van den Akker (1999: 2) argues that many ‘traditional’ research approachessuch as experiments, surveys, correlational analyses, with their emphasis on descriptionhardly provide prescriptions that are useful for design and development problems ineducation. He claims that an important reason for design research1 stems from the complexnature of the educational reforms worldwide. Ambitious reforms cannot be developed atthe drawing tables in government offices, but call for systematic research supporting thedevelopment and implementation processes in a variety of contexts.In his review of the state of educational research and more specifically educationaltechnology research, Reeves (2006: 57) concludes that there is “a legacy of ill-conceived andpoorly conducted research that results in no significant differences or, at best, in modesteffect sizes”. He also argues for the domain of educational technology that educationaltechnologists, in stead of doing more (media) comparison studies, should undertake typesof design research. In other words, Reeves argues that in stead of doing more studiescomparing whether in a certain context method A is better than method B, it is better to1)which he calls ‘development research’ in his 1999 publicationan introduction to educational design research9
undertake design research aimed at developing an optimal solution for a problem incontext.In the field of learning sciences, the belief that context matters lead to the conclusion thatresearch paradigms that simply examines learning processes as isolated variables withinlaboratory settings will necessarily lead to an incomplete understanding of their relevancein more naturalistic settings (Barab & Squire, 2004; with reference to Brown, 1992). In thisfield, design-based research was introduced with the expectation that researchers wouldsystematically adjust various aspects of the designed context so that each adjustmentserved as a type of experimentation that allowed the researchers to test and generatetheory in naturalistic contexts (Barab & Squire, 2004: 3).These sources illustrate the need for design research as an alternative research approach.Before elaborating on design research this paper will first discuss more generally possiblefunctions of research and how research functions are related to research approaches. Thendesign research will be defined and characterized from various perspectives, such as thetype of knowledge the design researchers aim for, the type of research questions that canbe addressed, and the outputs of design research. This will be followed by a section inwhich different approaches to design research are introduced and sections discussing howdesign research can or should be conducted, with a more in-depth discussion of formativeevaluation as the most prominent research activity in design research. Conducting designresearch puts researchers in a situation in which they have to face a number of dilemmas.These will be discussed before ending the chapter with a few concluding remarks.A final note on terminology, following Van den Akker et al. (2006:4) we use design researchas a common label for a ‘family’ of related research approaches who may vary somewhat ingoals and characteristics – examples are design studies, design experiments, design-basedresearch, developmental research, formative research, engineering research.Research functions – research approachesBefore elaborating on the meaning of design research, it is important to position designresearch as a research approach next to other research approaches, which is the purpose ofthis section.The key focus in all scientific research is the search for ‘understanding’ or for ‘knowing’ withthe aim of contributing to the body of knowledge or a theory in the domain of research.Other broad aims of doing educational research are to provide insights and contributionsfor improving practice, and to inform decision making and policy development in thedomain of education.10an introduction to educational design research
Research functionsIn general, we can distinguish various research functions, each reflecting certain types ofresearch questions. Examples of research functions (with exemplary research questionsfitting the function) are:1. to describe: e.g. what is the achievement of Chinese grade 8 pupils in mathematics;what barriers do students experience in the learning of mathematical modelling2. to compare: e.g. what are the differences and similarities between the Chinese and theNetherlands curriculum for primary education; what is the achievement inmathematics of Chinese grade 8 pupils as compared to that in certain other countries3. to evaluate: e.g. how well does a program function in terms of competences ofgraduates; what are the strengths and weaknesses of a certain approach; etc4. to explain or to predict: e.g. what are the causes of poor performance in mathematics (i.e.in search of a ‘theory’ predicting a phenomenon when certain conditions orcharacteristics are met)5. to design and develop: e.g. what are the characteristics of an effective teaching andlearning strategy aimed at acquiring certain learning outcomes; how can we improvethe motivation of learners.In many research projects the research questions are such that in fact various researchfunctions do apply. For example, if the research question pertains to comparing themathematics achievement of Chinese grade 8 pupils as compared to that in certain othercountries, then as part of comparing the researchers will evaluate the achievement of grade8 pupils in each of the countries involved. Or, as another example, if one wants to designand develop a teaching-learning strategy for acquiring the competency of mathematicalmodelling (in grade 11 & 12), then researchers may first want to understand and carefullydescribe what barriers students experience with mathematical modelling, whilst also theevaluation function is important in determining whether the teaching-learning strategythat has been developed is effective. Both examples illustrate that usually a research projecthas a primary research function, but that other research functions are being applied to‘serve’ the primary research function.At the level of a research project, starting from a research problem or question, we aresupposed to have the following sequence:Research question (primary) research function choice of research approach.In this chapter we focus on research which has design and develop as the primary researchfunction.Research approachesMost text books on research methodology present and discuss a number of researchapproaches or strategies (see e.g. Denscombe, 2007). Usually each research approach can bean introduction to educational design research11
used for realizing more than one research function. Without going into detail here,examples of research approaches and their possible research functions are: survey: to describe, to compare, to evaluate experiments: to explain, to compare case studies: to describe, to compare, to explainaction research: to design/develop a solution to a practical problemethnography: to describe, to explaincorrelational research: to describe, to compareevaluation research: to determine the effectiveness of a programTextbooks on research methodology usually do not present and discuss design research: design research: to design/develop an intervention (such as programs, teaching-learningstrategies and materials, products and systems) with the aim to solve a complexeducational problem and to advance our knowledge about the characteristics of theseinterventions and the processes to design and develop them.In line with the remark that more than one research function may have to be applied toaddress a research question, it should be noticed that in a research project more than oneresearch approach may have to be applied. For example, if there is a need to compare howwell Chinese grade 8 pupils perform in mathematics as compared to a number of othercountries, the primary research function is to compare, leading in this case to a survey asthe best research approach. However, as part of the development of a valid and reliablemathematics test, the researchers may do correlational research to determine whether thetest being developed is valid, i.e. correlates with other measures of mathematicsachievement.As a final remark, it is important that design researchers, like all researchers, keep in mindthat also for their research the guiding principles for scientific research (Shavelson &Towne, 2002) apply, viz: Pose significant questions that can be investigated Use methods that permit direct investigation of the question 12Link research to relevant theoryProvide a coherent and explicit chain of reasoningReplicate and generalize across studiesDisclose research to encourage professional scrutiny and critiquean introduction to educational design research
What is design research?As stated educational design research is the systematic study of designing, developing andevaluating educational interventions (such as programs, teaching-learning strategies andmaterials, products and systems) as solutions for complex problems in educational practice,which also aims at advancing our knowledge about the characteristics of these interventionsand the processes of designing and developing them.The twofold yield of design research, viz. research based interventions as well as knowledgeabout them, can also found in definitions of design research by other authors. For example,the broad definition of Barab and Squire (2004) also encompasses most variations ofeducational design research: “a series of approaches, with the intent of producing newtheories, artefacts, and practices that account for and potentially impact learning andteaching in naturalistic setting.By its nature, design research is relevant for educational practice (and therefore also foreducational policy) as it aims to develop research-based solutions for complex problems ineducational practice. Starting point for design research are educational problems for whichno or only a few validated principles (‘how to do’ guidelines or heuristics) are available tostructure and support the design and development activities2. Informed by prior researchand review of relevant literature, researchers in collaboration with practitioners design anddevelop workable and effective interventions by carefully studying successive versions (orprototypes) of interventions in their target contexts, and in doing so they reflect on theirresearch process with the purpose to produce design principles.Many examples of the need for innovative interventions can be given at system level andinstitutional level. At system level, for example, one may want to develop a system for e-learning to serve a specific target group of students in higher education, and at the level ofschool or classroom one may want, for example, to address the question of what areeffective methods for collaborative learning. See also Gustafson & Branch (2002) whodeveloped a taxonomy of instructional development models based on a selectedcharacteristics; they distinguish between models with a classroom orientation, productorientation and system orientation.The research process in design research encompasses educational design processes. It is –like all systematic educational and instructional design processes - therefore cyclical incharacter: analysis, design, evaluation and revision activities are iterated until a satisfyingbalance between ideals (‘the intended’) and realization has been achieved.2)see also the chapter of Kelly in this book where he discusses when design research is appropriate.an introduction to educational design research13
This process can be illustrated in various ways. Just a few examples are presented here toshow how different authors have visualized the research process.Reeves (2006) depicts the design research approach as follows:Identify and analyseproblems byresearchers &practitioners incollaborationDevelopment ofprototype solutions:informed by stateof-art theory, existingdesign principals &technology innova tionsReflection to produce‘design principles’ &enhance solutionimplementation inpracticeIterative cycles oftesting & refinementof solutions inpracticeFigure 1: Refinement of Problems, Solutions, Methods, and Design Principles (Reeves, 2000,2006)McKenney (2001) illustrates in her study this cyclical process as follows:needs & contextanalysisdesign, development & form
In his review of the state of educational research and more specifically educational technology research, Reeves (2006: 57) concludes that there is “a legacy of ill-conceived and . evaluation as the most prominent research activity in design research. Conducting design
work/products (Beading, Candles, Carving, Food Products, Soap, Weaving, etc.) ⃝I understand that if my work contains Indigenous visual representation that it is a reflection of the Indigenous culture of my native region. ⃝To the best of my knowledge, my work/products fall within Craft Council standards and expectations with respect to
ART APPRECIATION* SYLLABUS: 1. Course Overview & Introduction *Warning Note: some of the art works shown and discussed in this class have nude female and male figures, violent a send xual content as well as other content that some may deem offensive. _ Description: An introduct
5 2002 The ohn Maxwell Company InTRODuCTIOn Preparation Step 1: Read the book, Becoming a Person of Influence.* Step 2: Watch the videos. Step 3: Study all elements of this Leader Guide and the Student Workbook. Step 4: Teach yourself the class. Work through each element just as your students will. Step 5: Make sure
1 Introduct Ion Introduction This booklet explains ACT Aspire Grade 4 Science test questions by presenting, with their answer keys, sample questions aligned to each reporting category on the test. A key includes the question’s depth-of-knowledge (DOK) level,
Execut ve summary 1 Chapter 1. Introduct on 5 1.1 Methodology 5 Chapter 2. Evolv ng d saster contexts 7 . the German government’s revision of its food assistance policy in emergency and transitional contexts. It involved interviews . ECHO talks about a gradual and important shift over the last 15 years .
water and environmental sanitation strategic plan table of content: 1. foreword 1.1. sdpwes elaboration team 1.2. strategic planning 1.3. sdpwes designing methodology 1.4. public involvement and information-sharing 1.5. wes sp adoption and implemeentation 2. general part 2.1. introduct
adiabatic flame temperature, chemistry of combustion, elementary reactions, chain reactions, multistep reaction mechanism, quantification of emissions, emission control methods. 6. CORROSION AND ITS CONTROL: . Fundamental of Organic spectroscopy by Y. R. Sharma 5. Introduct
his greatest prestige and popularity with his novel Ariadne, in . identifies with Dorinda’s midlife awakening because she has been through that experience herself: after spending her life trying to live up to the standards of supportive wife, loving mother and perfect hostess that her husband’s elitist circle expected of her, “being my own person only became possible as an idea or a .
