Pros Cons Of Qualitative And Quantitative In User Experience Research

PROS & CONS OFQUALITATIVE ANDQUANTITATIVEIN USEREXPERIENCERESEARCH

PROS & CONS OFQUALITATIVE AND QUANTITATIVEIN USER EXPERIENCERESEARCHIN USER EXPERIENCE RESEARCH, all methodologies fall onKEY FOR CONTEXT OF PRODUCT USEDURING DATA COLLECTION:a scale from Qualitative to Quantitative. As you move across thisgradient, the usefulness, potential questions answered, and results garneredchange categorically. It’s important to understand the difference betweenthe types of data, and to know when which type of method is appropriate.In early stages of product development, you might lean on qualitative todo more exploratory work, while once you have an established product,quantitative methods may be more helpful in understanding how mostusers are interacting with your product. In this white paper, we’ll clarifythe strengths and weaknesses of each type of research, to help UXresearchers make the right decision for their product every time.Natural use of productScripted (often lab-based) use of productDe-contextualized / not using productCombination / hybridA LANDSCAPE OF USER RESEARCH METHODSChart created by C Christian Rohrer 1EyetrackingBEHAVIORALUsability Benchmarking (inlab)Usability Lab StudiesEthnographic Field StudiesATTITUDINALClickstream AnalysisA/B TestingParticipatory DesignFocus GroupsInterviewsQUALITATIVE (DIRECT)Moderated Remote Usability StudiesUnmoderated Remote Paned StudiesConcept TestingDiary/Camera StudiesCustomer FeedbackDesirability StudiesCard SortingUnmoderated UX StudiesTrue Intent StudiesIntercept SurveysEmail SurveysQUANTITATIVE (INDIRECT)

WHAT IS QUALITATIVERESEARCH?Qualitative research methods are those that explore broad questionsthrough textual, difficult to encode data such as verbal answers, ratherthan numerical data. These methods usually rely on smaller numbers of individuals giving detailed, free ranging answers to questions. Thisprovides detail about how users feel about a product, how they interact withit, and why they are having that interaction. Methodologies include thingslike Usability Lab Studies, Ethnography, Customer Journey Mapping, andParticipatory Design.STRENGTHS OFQUALITATIVE RESEARCHWEAKNESSES OFQUALITATIVE RESEARCHOne, it’s explorative, which allows it to be used in earlystages of product development. Qualitative research can beas simple as bringing people in for a focus group to talk abouta potential product. The questions don’t need to be fullyformed yet, but can be explored in a conversational format,rather than a strict set of questions or answers.Qualitative research can’t be automated, at least not tothe extent that quantitative can. Because the resulting data isdifficult to encode to a discrete set of answers, a researchermust manually analyze the data that comes in each time.We’ll see more about automating quantitative research lateron in this paper.Additionally, qualitative research answers the question“why?” Simply asking a person whether they preferprototype A or prototype B will give you that preference,but to understand why this is a preference is a deeper, moredetailed type of question. Qualitative research excels atanswering these types of questions.You also have less statistical power to yourfindings. Due to the low number of participants involvedin most qualitative research, you won’t have the sample sizeto make strong, confident assertions about behavior for thewhole population.While you are answering the “Why?” type questions, you canalso tackle more broad and malleable questions than youcan with quantitative research. The ability for a moderatorto hold a conversation with an individual opens opportunityto answer tangential questions, or change questions to bemore appropriate during a research session. Not needing tohypothesize every possible question in advance, but to havea free-flowing conversation is a major strength of qualitativeresearch.Qualitative research can also be relativelyinexpensive. You’ll need to pay for your participants, ofcourse, and potentially for a space to perform research.However, you don’t often need fancy tools to actually performthe research or the analysis.

WHAT IS QUANTITATIVERESEARCH?Quantitative research methods are those that utilize a large number ofresponses to more static questions, in order to analyze that data statistically. The gathered data is likely to come in percentages or averages,and questions are specific with discrete answer choices. Some methodologiesthat fall on the quantitative side of the spectrum are Intercept surveys, TrueIntent studies, and Competitive Benchmark studies.STRENGTHS OFQUANTITATIVE RESEARCHWEAKNESSES OFQUANTITATIVE RESEARCHQuantitative research excels at answering the questions“What?” and “How many?” In a preference question, forexample, it’s easy to use quantitative research to decide ifPrototype A or Prototype B is preferable to an audience, andwhat percentage of the audience prefers each prototype.Can be more expensive, given the common need for onlinesurvey software, statistical analysis software, and dependingon methodology, paying for panelists to take your surveys.Quantitative research relies heavily on providing statisticalpower to your data, typically through testing the p-value ofthe results. The large numbers of participants allow you todraw conclusions about a population from a smaller subset,and be relatively confident that your conclusions are sound.This type of research is also very good when you wantconcrete data about a certain aspect of your site/product,without too much exploration. For instance, you might beinterested in how many people click on a link on your sitewhen attempting a specific task. Using clickstream analysis,you can see exactly the percentage of users who click on thatlink.Benchmarking and tracking metrics are also easy withquantitative research. Finding a baseline for your site or app’sperformance, and then tracking that over time is a numbersgame, and it means you can easily compare your results withother sites/apps or with your past performance.Higher levels of automation are available whenperforming quantitative research. Many online researchsoftware companies provide tools for ongoing voice of thecustomer programs, intercept surveys, or quick feedback thatcan be run and monitored on a weekly or monthly basis.In many cases, quantitative data is useless without someknowledge of statistics, both to perform the statisticalanalysis, and to understand and disseminate the results.Working with hundreds or thousands of responses requiresa different skillset than understanding ten or twelve peoplewho interacted with an app.The responses in quantitative are static in most cases.Because a researcher isn’t typically present with quantitativeresearch – there are too many participants to guide them allindividually – confusing answers remain confusing answers,and can’t be delved into for more detail.

WHILE READING THESE STRENGTHSAND WEAKNESSES of the different types ofresearch, it may have occurred to you that thestrengths of quantitative are the same as theweaknesses of qualitative in many cases. If youthought that, you were right! That’s why at KeyLime Interactive we recommend a blending of theresearch types, either through a hybrid approachincorporating both types of research in onemethodology, or to include each type at differentstages of the product development cycle. You maywant to start with more qualitative methods earlyin your product life cycle, and as the questions getclearer and measurements more specific, start toinclude more quantitative methods. The only way toassure coverage of all major research questions is touse multiple methodologies that cover the spectrumfrom qualitative to ies/For more on our upcoming workin this industry, please contactsales@keylimeinteractive.com.

PROS & CONS OF QUALITATIVE AND QUANTITATIVE IN USER EXPERIENCE RESEARCH IN USER EXPERIENCE RESEARCH, all methodologies fall on a scale from Qualitative to Quantitative. As you move across this gradient, the usefulness, potential questions answered, and results garnered change categorically. It's important to understand the difference between