JAR49(2) 09-023 1/6 05/19/09 8:08 Am - Wharton Faculty

JAR49(2) 09-0231/605/19/098:08 amREVISED PROOFPage:170Using Quasi-Experimental DataTo Develop Empirical GeneralizationsFor Persuasive AdvertisingJ. SCOTT ARMSTRONGThe Wharton Schoolarmstrong@wharton.upenn.eduSANDEEP PATNAIK"Quasi-experimental data" provide a valid and relatively low-cost approach towarddeveloping empirical generalizations (EGs). These data are obtained from studies inwhich some key variables have been controlled in the design. We describe our EGs as"evidence-based principles." Using data from 240 pairs of print advertisements from fiveGallup and Robinsoneditions of the Which Ad Pulled Best series, we analyzed 56 of the advertising principlessandeep.patnaik@from Persuasive Advertising by J. Scott Armstrong (New York: Palgrave Macmillan,gallup-robinson.comforthcoming). These data controlled for target market, product, size of the advertisement,media, and in half the cases, for brand. Aspects of the advertisements differed, however,as in illustrations, headlines, color, and text. The findings from quasi-experimentalanalyses were consistent with field experiments for all seven principles where suchcomparisons were possible. Furthermore, for 26 principles they unanimously corroboratedthe available laboratory experiments as well as the meta-analyses for seven principles. Inshort, quasi-experimental findings always agreed with experimental findings. This isimpressive given that the quasi-experimental analyses—and some of the experimentalanalyses—involved small samples, and often used different criteria.INTRODUCTIONSOURCES OF EVIDENCEProgress in advertising depends upon the use ofNonexperimental studies involve analyses of datacumulative knowledge. This knowledge can beon a set of advertisements to determine whichtransformed into empirical generalizations (EGs).variables are most closely related to success. ThisIn this article, we discuss a type of EG that weis probably the dominant way by which peoplerefer to as "evidence-based principles," or, moredraw inferences about the value of persuasivesimply, as principles.strategies in advertising.We describe the types of evidence that canbe used to develop principles. We then describeIn quasi-experimental studies, the design controls for some (but not all) variables when compar-the quasi-experimental data on full-page printing different advertisements. For example, theseadvertisements that we used to assess advertis-studies might compare advertisements for a givening principles. Finally, we examine the validitybrand and media. Other variables, however, mayof the analyses of these data by comparing thenot be controlled. This approach is seldom used tofindings with those from other types of studiesanalyze persuasive advertising.including laboratory experiments, field experi-In experimental studies, the values of all keyments, and meta-analyses based on experimentalcausal variables are controlled. For example, aevidence.researcher who wants to know whether humor is170 JOURNAL OF ADVERTISING June 2009RESEARCHDOI: 10.2501/S0021849909090230

JAR49(2) 09-0232/605/19/098:08 amREVISED PROOFPage:171DEVELOPING EGs FOR PERSUASIVE PRINT ADVERTISINGIn some cases, the advertisements wereEMPIRICAL GENERALIZATIONcoded by more than one person. For ex-Communicate a unique selling proposition. Ideally, it should be based on an importantbenefit; alternatively, and riskier, it could be based on a feature that clearly impliesa benefit. It is effective if it is unique in the minds of consumers even though other brandscould make the same claim. However, it is especially effective if it cannot be easilymatched by competitors. This generalization, previously regarded as problematic, issupported by recent experimental evidence.ample, advertisements for one principle,"Communicate a Unique Selling Proposition(USP)," were coded by three coders toestablish that the coding related to a USPand not merely to a benefit. The coderswere asked to keep notes regarding uncertainty or other observations relevant totheir coding decision.Coding sheets were prepared (samplepersuasive creates two identical advertise-sheets are provided at advertisingprinci-The WAPB database allows testing of thements, except that one advertisement isdirection and effect size for the ad-ples.com). All coding was done prior tohumorous and the other is not. The re-vertising principles because most of theviewing the criteria, except for two prin-searcher shows the advertisements to ran-conditions were identical for each adver-ciples in the early stages, when the pro-domly assigned subjects in identicaltisement in the pairs. The target market,cedure was being developed. The processsituations and records their reactions. Ex-product, and media were the same. Of themeant that no coders had prior knowl-perimental evidence comes from two types240 pairs of advertisements, 123 were pairededge regarding an advertisement's degreeof studies: laboratory experiments and fieldagainst the same brand. Aspects of the advertisementsof effectiveness.experiments. The former allows for tighterdiffered, however, such as in illustrations, headlines,In a number of instances, the advertisementscontrol, while the latter adds realism.color and text.were coded several times to examine themain principle and the conditions associatedInstructors' guides for each WAPB ediANALYSIS OF THE WAPBtion provided recall scores for advertise-with the principle. For example, while codingQUASI-EXPERIMENTAL DATABASEments in 40 of the pairs and reader-interestfor the principle "Communicate a USP," our ini-We analyzed full-page print advertise-scores for 10 pairs. (Reader-interest scorestial analysis yielded 18 pairs in which onements from Which Ad Pulled Best (WAPB)were not included in the 9th edition.) Weadvertisement in the pair claimed a USP5th through 9th editions (Burton andconsider both of these scores under thewhile the other did not make such a claim.Purvis, 1987-2002). Each edition con-term "recall." Recall scores were basedNext, all advertisements with USP claimstains 50 pairs of advertisements (excepton a subject's ability to identify both thewere carefully coded to check whetherfor the 9th edition, which has 40 pairs).product and advertiser the day followingtheir USPs could have been claimed byThese advertisements had been pre-exposure.other products or services. We found 27pared by leading U.S. advertisers andpairs of advertisements where one of thewere then sent for testing by Gallup &advertisements had a USP that could notRobinson. In effect, they represent quasi-fieldCoding the advertisementshave been claimed by any other productexperiments. The advertisements featured inA team of research assistants coded thewhile the other advertisement stated aWAPB 5th through 8th editions are ar-conditions and actions for each advertise-common USP. In general, advertisementschived at www.advertisingprinciples.ment. In some cases (i.e., "How manywith USPs that were not common werecom. WAPB 9th edition is archived atwords are in a headline"), the coding wasbetter recalled than advertisements usingwww.gallup-robinson.comstraightforward. In many cases, however,a "common" USP. For example, an ad-Criteriathe coding was subjective (i.e., determin-vertisement for Norland Bone Desensi-The WAPB criteria for these quasi-ing whether an advertisement related to atometer that said "Unique design of theexperimental data relate to recall. Recall ishigh-involvement or low-involvementscanner allows scans anywhere on thean intermediate step; if you are not awareproduct). Someone familiar with the prin-surface of the table, and at any angle.of an advertisement, you are unlikely to actciple, therefore, had to be charged withYour patient does not move, the scanneron it. Zinkhan and Gelb (1986) found thatthoughtfully doing the coding. The cod-arm does. . . . No other system offers thishigh "noted" scores were positively re-ers, however, had no knowledge of thefeature" had a recall thrice that for anotherlated to purchase intentions (r 0.52).purpose of the coding.June2009JOURNAL OF ADVERTISINGRESEARCH 171

JAR49(2) 09-0233/605/19/098:08 amREVISED PROOFPage:172DEVELOPING EGs FOR PERSUASIVE PRINT ADVERTISINGTABLE 1Quasi-Experimental Results on Recall Gain for Persuasion PrinciplesNumber ofRecallAdvertisingGainDescription of Principles: Includes Necessary ConditionsPairs.2.126Offer verifiable evidence.2.0445Communicate a Unique Selling Proposition (USP) (one that cannot be claimed by other brands).1.7446Make the first paragraph relevant.1.7310Describe a problem and show how the product solves it.1.7121Include brand and company names (double-branding).1.6510Consider celebrity endorsements for gaining attention.1.6420Provide news, but only if it is real.1.6024Use positive arguments.1.5443Illustrations should support the basic message.1.5224Use descriptive headlines for high-involvement products.1.5036Balance the layout.1.4924Include the brand name in the headline.1.4825For high-involvement products, the reasons should be strong.1.4422Consider mystery advertisements only when the brand or organization does not enhance the claim.1.4418Make the brand or company name prominent if it conveys a good image.1.4314Show the product.1.4212Make elements of an advertisement reinforce one another.1.3912Use product-related questions only if you have good answers.1.3437Provide product information that customers need.1.3437When using metaphors, make them concrete.1.3238Use concrete words.Use familiar words.1.3214.1.317Provide evidence that the product is widely used.37Provide a reason.1.29.1.2816Use a typeface to enhance meaning.1.2558Use informative subheadings for long copy.1.2524Use wordplay if it is clearly related to the product.1.2326Avoid negative words unless the target market believes the opposite.(continued)172 JOURNAL OF ADVERTISING June 2009RESEARCH

JAR49(2) 09-0234/605/19/098:08 amREVISED PROOFPage:173DEVELOPING EGs FOR PERSUASIVE PRINT ADVERTISINGTABLE 1 (cont'd)Number ofRecallAdvertisingPairsDescription of Principles: Includes Necessary ConditionsGain.Describe specific, meaningful benefits.1.2142.1.2126Show that people similar to those in the target market use the product.Match the facial expression of the spokesperson to the product and target market.1.2113.1.1943When using an explicit action step, make it easy.1.1919Lead the reader into the body copy.1.1724Alert the target market early.1.1741Consider using visuals that create favorable associations with the product.Use a single theme—or two in some situations.1.1630.Demonstrate product benefits.1.1573.1.1445Use positive innuendos when their use has a basis.

JAR49(2) 09-023 2/6 05/19/09 8:08 am REVISED PROOF Page:171 DEVELOPING EGs FOR PERSUASIVE PRINT ADVERTISING In some cases, the advertisements were coded by more than one person. For ex- ample, advertisements for one principle, "Communicate a Unique Selling Proposition (USP)," were coded by three coders to