Quantitative Research Methods - Nova Southeastern University

Quantitative Research MethodsFSEHS-ARC

Overview Research Process Quantitative Methods Designs Validity and Research Designs

A Definition of ResearchResearch is a process of steps used tocollect and analyze information in order toincrease our understanding of a topic orissue.

Problems With Research Today Contradictory or vague findings Questionable data Unclear statements about the intent ofthe study Lack of full disclosure of the datacollection procedure Inarticulate rendering of the researchproblem

The Process of ResearchIdentify theResearch ProblemReportandEvaluate ResearchReview theLiteratureAnalyze andInterpretDataCollect DataSpecify aResearchPurpose

The Process of Research:Identify the Research Problem Specify a problem Justify a problem Suggest a need to study the problem foraudiences

The Process of Research:Review the Literature Locate resources– Books– Journals– Electronic resources Select resources– Determine the relevant resources for thetopic– Organize the resources by developing a“Literature Map” Summarize the resources in a literaturereview

The Process of Research:Specify a Research Purpose Identify the purpose statement– The major intent of the study– The participants in the study– The site of the study Narrow the purpose statement– Quantitative: Write researchquestions and/or hypothesis– Qualitative: Identify a centralphenomenon and writesubquestions

The Research Process:Collect Data Determine the data-collection method Select the individuals to study Design or select data-collectioninstruments and outline data-collectionprocedures Obtain permissions Gather data

The Research Process:Analyze and Interpret Data Take the data apart to look at individualresponses Represent the data in tables, figures, andpictures Explain conclusions from the data thataddress the research questions

Ethical Considerations in Research Respect the rights of the participants Honor the requests and restrictions of theresearch site Report the research fully and honestly

Definitions of Quantitative andQualitative ResearchQuantitative ResearchA type of educational research in which the researcherdecides what to study; asks specific, narrow questions;collects quantifiable data from participants; analyzes thesenumbers using statistics; and conducts the inquiry in anunbiased, objective manner.

Characteristics of QuantitativeResearch An emphasis on collecting and analyzinginformation in the form of numbers An emphasis on collecting scores thatmeasure distinct attributes of individualsand organizations An emphasis on the procedures ofcomparing groups or relating factors aboutindividuals or groups in experiments,correlational studies, and surveys

Characteristics of Quantitative Methodsin the Process of ResearchQuantitativeCharacteristics Descriptive/ExplanatoryRole Justify ProblemSteps in the ResearchProcessIdentify a Problem Majorand Narrow Measurable/ObservableReview the Literature SpecificSpecify a Purpose PredeterminedInstruments Numeric Data Large Numbers Statistical Descriptionof Trends Comparisons/Predictions Standardand Fixed Objective and UnbiasedCollect DataAnalyze and Interpret DataReport and Evaluate

Quantitative Explaining whether anintervention influencesan outcome for onegroup as opposed toanother groupExperimental Research:Between-SubjectsApproachAssociating orrelating variablesin a predictablepattern for onegroup ofindividualsDescribing trends forthe population ofpeopleNonexperimentalResearch:Survey ApproachNonexperimentalResearch:Correlational Approach

Method, Research, Approach and Design(Edmonds & Kennedy, 2012)LevelMETHOD1ExplanationThe method is the theoretical, philosophical, and data analytic perspective. The methodcan be quantitative, qualitative, or mixed (e.g., a quantitative method1). RESEARCH2Research refers to the systematic process of group assignment, selection, and datacollection techniques. Research can be experimental, quasi-experimental, or nonexperimental (e.g., a quantitative method1 and experimental research2). APPROACH3The approach is the first step to creating structure to the design, and it details (a) atheoretical model of how the data will be collected, and (b) if one case, one group, ormultiple groups will be associated with the process (e.g., a quantitative method1,experimental research2 with a between-subjects approach3). DESIGN4The design is the actual structure or framework that indicates (a) the time frame(s) inwhich data will be collected or how and when the data will be analyzed using qualitativemethods, (b) when the treatment will be implemented (or not), and (c) the exact numberof groups that will be involved (e.g., a quantitative method1, experimental research2 with abetween-subjects approach3 and a pre- and posttest control group design4).Edmonds, W. A., & Kennedy, T. D. (2012). An applied reference guide to research designs: Quantitative, qualitative, and mixedmethods. Thousand Oaks, CA: Sage.

Experimental research utilizing a betweensubjects approach with a pre- and posttestcontrol group designResearch Question: Does active parent engagement in selecting andusing routine-based activities have a positive effect on children's languageand appropriate behavior tR1 (n 22)TELD-3, ECBIFamily-CenteredInterventionTELD-3, ECBIR2 (n 19)TELD-3, ECBI-TELD-3, ECBITime Chao, P., Bryan, T., Burstein, K., & Ergul, C. (2006). Family-centered intervention for youngchildren at-risk for language and behavior problems. Early Childhood Education Journal,34(2), 147-153.

Nonexperimental research utilizing acorrelational approach with an explanatory designResearch Question: What classroom motivationvariables are related to students’ sense ofbelonging?VariablesStudents (n 249)MotivationSelf, efficacy, instrumentality,goalsSense of belongingSense of belonging surveyWalker, C. O., & Greene, B. A. (2009). The relations between students’ motivational beliefsand cognitive engagement in high school. Journal of Educational Research, 102(6) 463-471.

Nonexperimental research utilizing a surveyapproach with a cross-sectional designResearch Question: What evidence exists todemonstrate the prevalence and consequences ofrecurrent low-back pain in children?Variable (N 500)Time point 1Low-back painLow-back painsurveyJones, M. A., Stratten, G., Reilly, T., & Unnithan, V. B. (2004). A school-based survey ofrecurrent non-specific low-back pain prevalence and consequences in children. HealthEducation Research, 19(3), 284-289.

Threats to Internal Validity ationExplanationAny event that occurs during the time of the treatmentand the posttest that could affect the outcome (e.g.,natural life events such as a death in the family, change injob, or moving).The natural process of changing, growing, and learningover time.The effects of practice familiarity in taking the same testmore than once (e.g., the participant who takes the samemath achievement test twice in the pre- and posttestmeasures may improve performance simply because of thefamiliarity with the test)The change in a measuring instrument over time (i.e.,some instruments undergo revisions).Edmonds, W. A., & Kennedy, T. D. (2010). A reference guide to basic research design for education and the social and behavioralsciences. New York, NY: Pearson.

Threats to External Validity e extent to which the sample (i.e., unit) represents thepopulation from which it is drawn (i.e., for a sample to represent apopulation, the researcher must employ the appropriate samplingprocedures and perform random selection).StimulusCharacteristicsand SettingsThe unique factors involved in providing the treatment orintervention, such as the setting and researchers (i.e., it is difficultto replicate contrived laboratory conditions to real-life scenarios).TreatmentVariationsVariations in the same treatment or the combination of multiple orpartial treatments account for different results.OutcomeVariationsObserving the effect of one type of outcome differs when alternateoutcomes are observed.ContextDependentMediationMediating variables related to outcomes differ between contexts orsettings.Edmonds, W. A., & Kennedy, T. D. (2010). A reference guide to basic research design for education and thesocial and behavioral sciences. New York, NY: Pearson.

Threats to Construct Validity (examples)ThreatExplanationSimilar to special treatment, the level of attention, ordifferentiated attention, between the groups from theAttention andexperimenter (e.g., the researcher spends more time with Group 1Contact withthan Group 2, and the differences observed in the outcome can beParticipantsexplained by the increased amount of attention and not due tothe intervention).SingleThe impact the researcher has on the development andOperations andimplementation of the treatment (i.e., researchers deliverNarrowtreatments differently based on experiences and expertise;Stimulustherefore, it is difficult to measure the impact the researcher hasSamplingon the treatment itself).The researchers’ expectancies, beliefs, and biases about theresults (e.g., if a researcher strongly believes anxiety reduces testExperimenterperformance, then the interaction between the researcher andExpectanciesthe participant may influence the outcome because the deliveryof instructions and adherence to protocols may change).Edmonds, W. A., & Kennedy, T. D. (2010). A reference guide to basic research design for education and the socialand behavioral sciences. New York, NY: Pearson.

Threats to Statistical Conclusion Validity(examples)ThreatExplanationPower is the extent to which the results of an analysisLow Statisticalaccurately reveal a statistically significant differencePowerbetween groups (or cases) when a statistical differencetruly exists.AssumptionViolating the assumptions (depending on the extent of theViolation ofviolation) of statistical tests can lead to over- orStatisticalunderestimation of practical and statistical significance ofTestsan outcome.Statistical significance can be artificially inflated whenError Rateperforming multiple pairwise tests, also referred to asProblemfamilywise error rate. (i.e., the probability of making a TypeI error when performing multiple pairwise analyses).Restriction ofLack of variability between variables weakens theRangerelationship and lowers statistical power.Edmonds, W. A., & Kennedy, T. D. (2010). A reference guide to basic research design for education and the social andbehavioral sciences. New York, NY: Pearson.

An applied reference guide to research designs: Quantitative, qualitative, and mixed methods. Thousand Oaks, CA: Sage. Level Explanation METHOD 1 The method is the theoretical, philosophical, and data analytic perspective. The method can be quantitative, qualitative, or mixed (e.g., a quantitative method 1). RESEARCH 2