Systematic Quantitative Literature Reviews

2y ago
29 Views
2 Downloads
2.00 MB
34 Pages
Last View : 15d ago
Last Download : 2m ago
Upload by : Aliana Wahl
Transcription

Systematic quantitative literaturereviewsWhat are they and why use them?Catherine Pickering,School of Environment, ourcesMurray, R. (2011). How to Write a Thesis. McGraw Hill Open University Press.Maidenhead, England (Chapter on writing a literature review)Boote, and Beile (2005). Scholars before researchers: On the centrality of thedissertation literature review in research preparation. Educational Researcher. 34: 3‐15.Petticrew, M. and Roberts, H. (2006). Systematic Reviews in the Social Sciences: APractical Guide. Blackwell Publishing, Oxford, England.Randolph J.J. (2009). A guide to writing the dissertation literature review. PracticalAssessment, Research and Evaluation. 14: 1‐13.Crisp, B.R. (2015) Systematic reviews: A social work perspective. Australian Social Work,68:3, 284‐295.Pickering, C., Grignon, J., Steven, R., Guitart, D. and Byrne. J. (2014 ‐ on line). Publishingnot perishing: How research students transition from novice to knowledgeable usingsystematic quantitative literature reviews. Studies in Higher 914907Pickering, C.M. and Byrne, J. (2014). The benefits of publishing systematic quantitativeliterature reviews for PhD candidates and other early career researchers. HigherEducation Research and Development. 33: 841651.Lots resources on our method at ‐ iew

Literature review Process – gain understanding of the existingliterature and how your research will contributeto it. Product – demonstrate this in the documentDifferent audiences for literature reviews include –industry/company, academic, consultancy,government Relationship between thinking, knowledgeproduction and writing Literature reviews We all produce them 1. But what is my topic?2. How do I do them?3. What method are available?4. How do the methods differ?5. Why should I consider doing a Systematic Quantitative Literature Review?

Common things in reviews Define terms Justify selection of literature – it cannot beeverything So also justify omissions Have a clear structure and let the reader knowabout it early in the text (could be historical,conceptual or methodological) Link your work with the literature Critique the literature Define the gapCriteria for evaluating literature reviews(Boote and Beile 2005)Coverage Is there well justified criteria for inclusion and exclusion ofliterature?Synthesis Does it distinguish what done from what needs to be done? Does it place topic in broader scholarly literature? Does it place topic in historical context of field? Has the writer acquired and enhanced subject vocabulary? Articulated the important variables and phenomena? Synthesized and gained a new perspective on literature?

Criteria for evaluating literature reviews(Boote and Beile 2005)Methodology Identified main methods and techniques(advantages/disadvantages) Related ideas and theories to theseSignificance‐ Practical significance of the topic‐ Scholarly significance of the researchRhetoric Writing coherent, with a clear structure and style?Lets start by working out what you aregoing to review Focus on your topicWhat broad discipline areas?How do they fit together?What literature do I need to read?What is it an important topic?How do I structure/justify the topic?

Evaluate/synthesis/analysistopics related to your research questionYourresearch1. What's my question and what disciplinesare involved?Yourresearch

2. What's the literature I need to review?Yourresearch3. How do I structuring my literature review?Turning circles into a triangleThe literature to reviewThe text of theliterature reviewYourresearchAims

3. How do I structuring my literature review?Turning circles into a triangleThe literature to reviewThe text of the literature reviewStepped out argumentLeading to the aims11Yourresearch3223AimsWhat methods are available?1. Traditional narrative2. Meta‐analysis3. Systematic quantitative literaturereview

What about the traditional nonsystematic narrative review?It involves. Reading as much literature as possible Assessing its importance Constructing carefully argued narrative of you analysis ofthe current status of researchA method for qualitative/narrativereviews Create an audit trailDefine the focus of reviewSearch for relevant literatureClassify documentsCreate summary databaseIdentify constructs and linkagesSearch for differing opinionsCorroborate by checking with others

Evaluating papersThink about and make notes/database on What were the aims/objectives of the research? What were the outcomes? What approaches/methods/strategies were used? What was the context of the research How does it contribute to the field Is it connected to my research question, and how ?What about systematic approaches? ReproducibleRigorousComprehensiveClear rules for inclusion/exclusion of literature

PRISMAScreeningIncludedTheStatement. PLoS Med6(6): yMoher D, Liberati A,Tetzlaff J, Altman DG,The PRISMA Group(2009). PreferredReporting Items forSystematic Reviewsand Meta‐Analyses:IdentificationWhat about systematic approaches?Additional recordsidentified throughother sourcesRecords identifiedthrough databasesearchingRecords after l‐text articlesassessed foreligibilityFull‐text articlesexcluded, withreasonsStudies includedin quantitativesynthesisMaybe try a Meta‐analysis? Statistical method for combining results from separate studies to assess effectsize often using weighted average.Often need studies with similar methodology, similar subjects and similarresponse variablesCommon in health sciences and many other areas when enough suitabledatasets.Can need team of experts and lots of time !Deals with issues of low sample sizes and voodoo correlations in some singlestudiesSee interesting new meta‐analysis of the literature assessing relationshipsbetween student evaluations of teaching (SET) and student learning includingcritiquing previous meta‐analysis and conducting a much more rigorous one.And basically it says student satisfaction is NOT linked to leaning e/pii/S0191491X16300323

Examples of systematic reviews usingmeta‐analysisCochrane Databases ofsystematic reviews (mostlyhealth care but also social)Campbell Collaboration –public policy interventions(crime, education, socialwelfare etc)Petticrew, M. and Roberts, H. (2006) Systematic Reviews in the SocialSciences: A practical guide. Oxford: Blackwell Publishing. – Particularlyimportant if going to weight studies.Cochrane and Campbell reviews Clear rules regarding methods Need to have proposed methods registeredand evaluated before commencing Often costly/time consuming ( 50,000) Require team of specialists, includingdiscipline area, but also informationspecialists, statisticians, and researchers withexpertise in these reviews

So what about using a systematicquantitative literature review?Mapping the discipline.1. Systematic methods to survey literature and select papersto include are explicit and reproducible2. Quantitative measure of the amount (number of papers) ofresearch within different sections of topic3. Comprehensive assesses different combinations oflocations, subjects, variables and responses4. Structured working out what is important about theliterature (categories/subcategories) ‐ collecting, analysingliterature, and writing follows clear stepsEasier step by step process for collecting, analysing thedata and the writing the reviewStep 1Define topicStep 2Formulate research questionsStep 3Identify keywordsStep 4Identify & search databasesStep 5Read & assess publicationsStep 6Step 7Step 9Structure databaseEnter first 10% papersEnter bulk of papersStep 8Step 12Draft methodsStep 11Evaluate key results &draft results sectionStep 10Produce & reviewsummary tablesTest & revise categoriesSteps 13Draft introductionStep 14Draft discussion & abstractStep 15Revise document till readyfor submission

Summary of the different methodsWho commonly does the Meta‐AnalysisExperts & newPhD studentsExpertsTeams of expertsHow papers selectedRarely systematicPhD students &othersPhD students &othersSystematicCompiling data on papersRarely systematicSystematicSystematicComparing papersExpert evaluationQuantitative orexpert evaluationIf want toQuantitativeStandardEasyExpert evaluationHow can usually publish themStatistical analysisGap analysisStructure of the documentEasy for updatingNoDescriptiveNarrativeLimitedTeams of expertsSystematicYesDescriptiveStandardRe do statisticsMethod with benefits.1. Straight forward structure/process for undertakingand writing review2. Maps the literature by – finding geographic, scalar,theoretical and methodological gaps3. Identifies unknown unknowns4. Can be rapidly turned into academic paper5. Database can be easily updated6. Database useful for intro/discussion of otherpapers/later research7. Easier latter as do not have to re‐read the wholeliterature again !

Systematic quantitative literature viewsworks for studentsAveraging 33 citationsper year for eachpaperSo how do you do it iewIncludes –1. Youtube videos on each stage,2. Papers outlining the approach,3. Papers published using the method,4. Youtube videos of students talking about the method5. Example excel databases6. Youtube video on why publish during your PhDAlso article in The Conversation ‐But I will summarise it for you now!

Being systematicStep 1Define topicStep 2Formulate research questionsStep 3Identify keywordsStep 4Identify & search databasesStep 5Read & assess publicationsStep 6Step 7Step 9Structure databaseEnter first 10% papersEnter bulk of papersStep 8Step 12Draft methodsStep 11Evaluate key results &draft results sectionStep 10Produce & reviewsummary tablesTest & revise categoriesSteps 13Draft introductionStep 14Draft discussion & abstractStep 15Revise paper till ready forsubmissionBeing systematic when fishingAim: need to catch all the specified fish, but notspend forever, and minimise bycatch.Questions Why fish? Aims and research questions What fish? Papers vs books, thesis, reports andother grey literature, other languages etc What nets to use? Are there keywords that work?Title Keywords Abstract vs whole paper? Where to fish? Which Databases and how do theydiffer? How long to fish? When have you found all thespecified fish?

Step 1. Define topicWorks well for Emerging areas, Topics where methods so diverse cannot do meta‐analysis Trans‐disciplinary fieldsExamples on website and published in journals such as:Ambio, Analyse and Kritik, Asian and the Pacific Policy Studies, Australian SocialWork, Crop Protection, Current Developmental Disorders Reports, Current Issues inTourism, Ecological Economics, Environmental Modelling and Software, GlobalEnvironmental Change, Higher Education Research and Development, InternationalJournal of Mental Health Nursing, International Journal of Parasitology, Issues inEducational Research, Issues in Educational Research, Journal of Hospitality andTourism Research, Journal of Manufacturing Science, Nurse Education in Practice,Reliability Engineering and Systems, Restoration Ecology, Small Group Research,Sustainability, Teaching and Teacher Education, Tourism Management, Urban Forestryand Urban Greening, Wildlife ResearchStep 2. Formulate research questions.e.g.1. Who did the research and when?2. Where was the research done? – geographicalspread3. What are were the main themes?4. What methods were used?5. What subjects examined?6. What variables measured?7. What patterns found in results?8. What are the gaps and future trends?

Step 3. Key words Need to identify relevant literature, but not lotsand lots of irrelevant literature Trial and errorNB: check out bottom of this May need synonymswebpage to find out who is yourexpert librarianTalk to university arch‐publishingExample. (also use wildcards)‘bird’ in combination with; ‘trail’, ‘track’, ‘walking’,‘hiking’, ‘impact’, ‘disturbance’, ‘mountain bike’,‘effect’, ‘dog walking’, ‘horse riding’, ‘ecotourism’,‘tourism’ ‘recreation’.Step 4. Search databases relevant toyour field1.2.3.4.5.6.7.8.9.10.11.Google ScholarScience DirectScopus,ProQuestWeb of KnowledgeSageBio MedHein OnlineWestlawOVIDEBSCOHost

Records identified throughdatabase searchingAdditional records identifiedthrough other sources(n 745)(n 8)Records after duplicates removedEligibilityScreening(n 467)IncludedMoher D, Liberati A,Tetzlaff J, Altman DG,The PRISMA Group(2009). PreferredReporting Items forSystematic Reviewsand Meta‐Analyses:The PRISMAStatement. PLoS Med6(6): tionRecord info for PRISMA statementRecords screenedRecords excluded(n 467)(n 382)Full‐text articles assessedfor eligibilityFull‐text articles excluded,with reasons(n 85)(n 38)Studies included inquantitative synthesis(n 47)Step 5. Read and assess papersFor each publication: Is it relevant? Abstract for some, whole paper for othersNeed criteria for inclusion – reproducibility Original research papers only? (may want to limit to certaintypes of research)Use reference lists and citations of the paper to cross‐checkyou have all (most!) papers – that its systematic.How many relevant papers did you find? If 15 papers – narrative might be better, or broaden topic If 300 may need to narrow topic

Creating your own databaseStep 1Define topicStep 2Formulate research questionsStep 3Identify keywordsStep 4Identify & search databasesStep 5Read & assess publicationsStep 6Step 7Step 9Structure databaseEnter first 10% papersEnter bulk of papersStep 8Step 12Draft methodsStep 11Evaluate key results &draft results sectionStep 10Produce & reviewsummary tablesTest & revise categoriesSteps 13Draft introductionStep 14Step 15Draft discussion & abstractRevise paper till ready forsubmissionStep 6: Structure databaseWork out categories and subcategories.This provides structure for the reviewInclude data on.Who does research, where, using what methods, whatresponse variables, what subjects, what types ofanalysis was used, what found?Excel works well but can use other programs Each paper is a row Categories/subcategories are columns

May want to use word clouds to helpwork out categories, terms and themesCategories about the paperFull reference details: Authors namesYear, Journal title, Journal discipline, Articleresearch disciplineCategories about geographic locationof researchCity, State, Country, Continent, Climatic zone,General habitat types, others

Categories for subjects of researchFor Birds Number and name of bird species assessed. Conservation status of the birds? Type of foraging guild?Categories for response variablesFor birds Individual response? (physiological or behavioural), Population level response ? (density/abundance), Reproductive response ? (number of nests, number eggslaid, number of chicks that hatched or fledged)?Categories about the methods usedWhat you include depends on the discipline. Some examples. Observational vs experimental? Was it a BACI design or what. What statistics were used.? Natural science, social science or mixed? Which qualitative approache(s)? (interviews, content and textanalysis, case studies, observations, focus groups, archivalresearch), Which quantitative approache(s)? (questionnaire surveys, field‐surveys and samples, field experiments, GIS, remote sensing andsatellite imagery) Which mixed approach? (including existing data base and recordssearches, or other literature analysis).

Weighting methods/studies.Weight studies by types of evidence?1. Randomized control trails (number replicates, effect size etc)2. Before, After, Control, Impact (BACI) experiments3. Experiments with controls4. Observational studies with ‘controls’, Quasi‐experimentaldesigns,5. Observational studies without ‘controls’,6. Cohort studies7. Case studiesCan also use checklists to compare studies using similar methods –high, moderate and low quality.Problem if interdisciplinary study in how to assess different types ofevidence.Categories for results Studied and discussed, or actuallydemonstrated? Outcomes positive, negative, neutral, mixed orother? More detailed results – Statistically significant,size effect/number of replicates, power ofanalysis? Others ?

Step 7. Enter around 10% of papersStep 1Define topicStep 2Formulate research questionsStep 3Identify keywordsStep 4Identify & search databasesStep 5Read & assess publicationsStep 6Step 7Step 9Structure databaseEnter first 10% papersEnter bulk of papersStep 8Step 12Draft methodsStep 11Evaluate key results &draft results sectionStep 10Produce & reviewsummary tablesTest & revise categoriesSteps 13Draft introductionStep 14Draft discussion & abstractStep 15Revise paper till ready forsubmissionStep 8. How well do the categories work? Are they to narrow or broad?Do you need additional values, new subcatagories?Do the criteria apply to categories work in reality ?Reflection now saves lots of time later !Step 9: Enter rest of papers Again cross check your categories and criteria Check your database is comprehensive(reference lists)

Step 10: Produce and review summarytables so you can.1. Check your database is accurate (entryerrors)2. Start to work out the most important resultsA few examples of tables from papers CountryUSAAustraliaCanadaUKSouth wedenIsraelBrazilOther 1226517818231312122111111112189195Where studies?# papers on community gardens bycountries and # countries authors from(based on author affiliations).Figure 1. Location in USA of gardens in theliterature.

Can get fancynow withGoogle mapsand GISAbstracts fromconferencesDefinitions used inpapersCategoryTotal USA OthersCharacteristics of gardensDefinitionYes301812No553124Typology321Food producedYes754629Food only472225Food andflowers25232Food &revegetation422NoNot specified1284Methods used in papersCategoryMethodsScienceSocial scienceNatural scienceMixedMethodsInterviewCase studyObservationSurveyText analysisFocus groupsNatural scienceOtherType of dataQualitativeQuantitativeBothTotal USA 494551531284192311236

Number of papers by discipline and resultsNegativeJournal er Total US4Enviro. & l1693717167542671351151110432511OtherTotalUS Other TotalUS Other2232223132112861412575112140120Trends over time1008060402002014201220102008200620042002Social Science abstracts 485 (64%) (86 mixed)

Cluster analysis of relatedtheory/theoristLeximanceranalysis of themesThomas, S. (2014). Blue carbon:Knowledge gaps, critical issues andnovel approaches. EcologicalEconomics, 107: 22‐38Map concepts by extracting andranking a list of key words and phrasesfrom source texts. Then uses intelligentalgorithm to iteratively build athesaurus of concepts from more thanone or two keywords. Concepts areindexed and weighted.Identify related concepts, but alsotopics missing

3. Writing the reviewStep 1Define topicStep 2Formulate research questionsStep 3Identify keywordsStep 4Identify & search databasesStep 5Read & assess publicationsStep 6Step 7Step 9Structure databaseEnter first 10% papersEnter bulk of papersStep 8Step 12Draft

Maidenhead, England (Chapter on writing a literature review) Boote, and Beile (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Re

Related Documents:

group "Systematic Reviews" with 2,600 members. Jos Kleijnen, MD, PhD Kleijnen Systematic Reviews Ltd 6 Escrick Business Park Escrick, York, YO19 6FD United Kingdom Phone: 44-1904-727981 Email: jos@systematic-reviews.com Web: www.systematic-reviews.com

based, whereas Paul and Criado (2020) added more refined cate-gories such as structured theme-based reviews, framework-based reviews, bibliometric reviews, hybrid reviews, conceptual reviews, and meta-analytical reviews to that list, in addition to recommend-ing the criteria for article and journal selection and highlighting the

systematic reviews (which include quantitative and qualitative studies), but reporting guidelines addressing the presentation and synthesis of qualitative data should also be consulted (35, 36). PRISMA 2020 can be used for original systematic reviews, updated systematic reviews, or continually . PRISMA 2020 statement. .

How to write a systematic literature review: a guide for medical students Why write a systematic review? When faced with any question, being able to conduct a robust systematic review of the literature is an important skill for any researcher to develop; allowing identification of the current literature, its limitations, quality and potential. In addition to potentially answering the question .

Librarian as Collaborator - Search Search hedges/filters are pre-tested strategies that assist in limiting search results to a specific sub-set of the database. Example -PubMed filter to find systematic reviews - (systematic review [ti] OR meta-analysis [pt] OR meta-analysis [ti] OR systematic literature review [ti] OR

Quantitative Aptitude – Clocks and Calendars – Formulas E-book Monthly Current Affairs Capsules Quantitative Aptitude – Clocks and Calendars – Formulas Introduction to Quantitative Aptitude: Quantitative Aptitude is an important section in the employment-related competitive exams in India. Quantitative Aptitude Section is one of the key sections in recruitment exams in India including .

Morningstar Quantitative Ratings for Stocks Morningstar Quantitative Ratings for stocks, or "quantitative star ratings," are assigned based on the combination of the Quantitative Valuation of the company dictated by our model, the current market price, the margin of safety determined by the Quantitative Uncertainty Score, the market capital, and

2. The Sources of Evangelical Systematic Theology 3. The Structure of Evangelical Systematic Theology 4. The Setting of Evangelical Systematic Theology 5. The Satisfaction of Evangelical Systematic Theology Study 1: The Nature of Systematic Theology & the Doctrine of Revelation "God is most glorified in us as we are most satisfied in him." John .