Global Attribution CMO: How is our ad budget allocation changing this year compared to last? VP of Advertising: Were doubling our allotment to digital channels like social media ads and online search ads and paring back our spending on traditional channels like television and magazine ads. Overall in the advertising industry, traditional ad channels are declining and digital channels are growing, and were leading the way. CMO: Great! So are we seeing better results now? VP of Advertising: We are now showing advertisements in over ten different channels, so consumers are being exposed to our advertisements on more media than ever. CMO: Okay, good, but are we getting more bang for the buck? VP of Advertising: We feel that being exposed to more ads in more locations can only help sell customers on our brand. CMO: Thats probably true, but is there any evidence that moving budget away from television to digital channels is bringing in more sales? VP of Advertising: Its impossible to know for sure, but we think keeping ahead of recent trends is a good idea. In the internet era, many customer actions can be measured. As a result, advertisers are under increasing pressure to use this customer data to show that their ads are increasing sales. But even with careful tracking of all possible customer data, problems with attribution can cause faulty conclusions about the effectiveness of various online ads. For example, last-click attribution typically exaggerates the effect of search marketing efforts, and first-click attribution can give highly errant results with small changes in an arbitrary time window assumption. No ready solution to the attribution problem has yet been developed, so marketing analysts must simply keep in mind that their data is not 100% reliable. Even more difficult than the attribution problem within digital marketing is the attribution problem across an entire advertising budget, including both online and offline ad spending. Even if a marketing analyst could be confident in attributing sales to marketing efforts in email, online search, social media, and display advertising, how could she determine the relative effectiveness of marketing efforts in television, billboard, magazine, and catalogs? An analysis that would accurately determine the relative effectiveness of the myriad advertising channels would be extremely valuable to any business, but an analysis of this kind is extremely difficult. The largest marketing research company in the world, The Nielsen Company, along with another marketing research heavyweight, Arbitron, undertook a joint project in 2005 to make such an analysis possible. The project was terminated three years later and deemed an expensive failure (http://magnostic.wordpress.com/2008/02/25/marketing-measurement-misplay-project-apollo-isdead/). In 2013, Peter Danaher and Tracey Dagger, marketing scholars from Monash University, in Melbourne, Australia, published the results of a research project for a large Australian retailer in which they were able to measure the relative effectiveness of advertising expenditures across ten different advertising channels spanning both online and offline advertising activities. In other words, Danaher and Dagger were able to solve the attribution problem, not just for the digital marketing, but for all marketing channels. This case describes the methods they used to collect the data and run this analysis. Collecting Data When a marketing analyst is trying to determine the effect of advertising expenditures on sales, what she is trying to determine is whether seeing an advertisement caused an individual (or several individuals) to make a purchase. Advertising is only effective if it changes individuals behavior. As a result, the only way to reliably determine the effectiveness of advertising is to measure both advertising exposure and purchasing at the individual level. That is, a company would need a list of its customers along with data on their purchasing and amount of exposure to all forms of advertising done by the company. The company could then analyze this data and determine whether customers who saw more television ads subsequently spent more than customers who saw fewer television ads for the company. Collecting such data is challenging. Many marketing research companies collect portions of this data, but none of them collect all of this data at the individual level. To collect this data, Danaher and Dagger used the loyalty program members of the Australian retailer. (The retailer wishes to remain anonymous, but it is an upscale department store analogous to Macys in the United States.) Specifically, they sent an invitation to an online survey to 20,000 randomlyselected members of the loyalty program (hereafter LP) who fit the target market (women between the ages of 25 and 54) on the day after the conclusion of a major four-week-long sale and accompanying advertising campaign. The survey measured LP members exposure to the retailers ads across all 10 advertising channels used by the retailer during the ad campaign for the sale. The LP program maintained a database of each members purchase history, so sales of each LP member could be retrieved from this database and matched to the data on her advertising exposure. The sale began on Wednesday, September 22, 2010 and concluded on Sunday, October 17, 2010. This sale was accompanied by a four-week-long advertising blitz across ten advertising channels, including mass media channels (television, newspapers, radio, and magazines), electronic media outlets (online display ads, Google search ads, and social media ads), and direct media (catalogs, postal mail, and e-mail). Across all media, ads were consistent in their appearance and messaging, announcing massive discounts on a wide range of products or on specific featured items. Table 1 shows the relative spending on these ten advertising channels and various measures of the resulting reach. Table 1 Medium Relative GRPs1 Percentage Spots Insertions Impressions 1 GRP stands for gross ratings points, which is a standard way to measure advertising exposures. GRP is calculated as Reach (%) Average frequency (#). A GRP of 100 indicates enough ad exposures to cover the entire population, Spend Reach Television 222 1,048 99 368 Newspapers 185 516 74 21 Radio 26 160 86 380 Magazines 4 31 16 2 Online display 34 180 61 16 million Search (Google) 3 39 21 15,200 paid clicks Social media 1 5 2 Catalog 100 79 79 Postal mail 72 122 52 E-mail 1 333 87 Measuring an individuals exposure to multiple advertising channels is a difficult task. Market research companies have developed sophisticated measurement techniques for measuring exposure to a single medium, such as Nielsens People Meter panel for television and Arbitrons panel for radio. These companies typically require participants to keep a diary of every exposure to the medium in question. For example, participants in Arbitrons radio panel will record every instance of radio listening for a week, including the radio station listened to and the length of time spent listening. Keeping such diaries is labor-intensive for one medium and thus would be impossible for ten media. As a result, media exposure was measured through the survey sent after the sale and ad campaign concluded. Because the retailer had a known media plan, the survey could be limited to asking about the media on which the retailer had advertised. For example, instead of asking an LP member for every instance of TV viewing during the four-week advertising campaign, the survey asked, In the past four weeks, how many episodes of Desperate Housewives have you seen? For newspapers, LP members were asked, On which days did you read or look into these newspapers in a typical week? To measure exposure to online display ads and social media ads, participants were asked their frequency of visiting the sites on which the retailer had placed banner ads. To measure exposure to Google search ads, the survey asked, About how many times did you do a Google search for [retailer] in the past 4 weeks? To measure exposure to radio ads, the survey asked respondents about their typical weekly radio-listening habits. Because the purpose of the study was to determine how ad exposure influences purchasing, measurements of media exposure must be converted to measurements of ad exposure. Ad exposure was measured using the traditional GRP, with a major difference being that GRP in this case indicates an individuals exposure to ads in that channel rather than the population-level exposure. Individual-level GRP was calculated from the individuals exposure to the medium in question combined with the number of times an ad was shown on that medium. For example, if an individual watched 3 of 4 episodes of Desperate Housewives and the retailer advertised on this show twice, the individuals GRP would be 150 for this show (100 2). The same though this score could come from a reach of 50% and average frequency of 2 or a reach of 100% and frequency of 1. Televisions GRP of 1,048 indicates that people on average saw the advertisement over 10 times. calculation would be carried out for all television shows on which the retailer advertised, and the individuals television GRP would be a summation of the GRP numbers for all television shows on which the retailer advertised. Fitting the Model This case is not meant to provide an in-depth study on statistical modeling, so it will skirt many of the details of the model, but some of the basic aspects of the model must be discussed if the reader is to develop an understanding of this research project and have any hope of replicating it. Table 2 shows a small portion of the data as they were formatted to enable fitting of the statistical model. Table 2 Estimated GRPs (100 = 1 ad exposure) Customer ID Purchase Spending Television Newspaper Radio Magazines Online Display Search (Google) Social Media Catalog Postal mail Email 130464 1 257 1788 159 43 25 240 0 6 100 100 200 179125 1 83 1469 800 190 13 244 100 0 100 100 300 209925 0 0 665 726 162 51 55 0 9 0 100 300 458318 1 511 564 888 186 20 293 0 3 0 100 300 418373 0 0 1969 699 124 48 184 100 2 100 200 200 247912 1 126 1610 609 287 28 241 0 3 200 100 100 234977 0 0 1095 181 297 31 9 0 2 100 100 300 461927 0 0 1748 367 152 2 123 0 0 0 200 300 478111 1 788 1026 331 208 13 219 100 4 100 100 400 162202 1 208 368 731 145 40 143 100 5 0 100 400 The desired end result of the statistical model is measurement of the effectiveness of each advertising channel. That is, we wish to know whether and by how much advertising expenditures in a given channel increased sales. In order for advertising to influence sales, it has to influence an individual to either (1) make a purchase when she otherwise would not have purchased or (2) spend more money than she otherwise would have. To determine whether advertising influenced the first behavior, or purchase incidence, we could run a logistic regression or probit regression model with the Purchase variable as the dependent variable and the GRP data as independent variables. This model would indicate which advertisement channels influenced LP members to shop when they otherwise might not have shopped. But we would not be able to determine whether advertising influenced the amount of money they spent. To determine whether advertising influenced the second behavior, or purchase amount, we could fit a linear regression model with the Spending variable as the dependent variable and the same GRP data as independent variables. But the Spending variable has several 0s in it. Roughly 45% of LP members in the sample made no purchases at all during the sale period. Running a Which of the ten media channels do you think was least reliably measured? Do you think the measurement of ad exposure in this channel was biased (systematically higher or lower than the real number) or just highly variant (sometime higher and sometimes lower than the real number)? 2. Think up another situation in which you would expect to find endogeneity. 3. In general, do you think online media is better for inducing purchase incidence or purchase amount? 4. Could the methodology described in the case be applied to determine advertising effectiveness of a consumer packaged good like Tide? Why/why not