One reason that direct marketing techniques have become so popular for internet advertising is that they offer ample opportunity to test new concepts in an experimental context. For example, Bob just read that using an orange "submit order" button increases both number of orders submitted and the average amount of an order. Bob knows that such "findings" often vary from site to site. Since he does not care at all about any other site, he decides to test this idea using an experiment on his site.

He randomly assigns 500 regular customers to see the orange submit order button when they visit his site, while the remaining regular customers continue to see the regular white submit button.

He looks at sales from the previous month for the 500 test subjects and finds the average sale is $75.50; the average previous month sale for the remaining regular customers is $77.00.

After a one month test, he finds that the test group (the 500 orange button customers) has an average sales amount of $90.00, while the remaining regular customers have an average sales amount of $86.50.

Bob performs a two-tailed difference of means test. Assume his boss uses an alpha level of .05, and Bob finds a p-value of .002.

a) What is your best estimate of the quantitative effects of the orange submit order button? How did you arrive at this answer? (6 points) (I am not looking for a statistical analysis here, but I would like to see a number.)

b) Identify and explain one threat to internal validity which is NOT a problem here, given the design of the experiment. (6 points) In what sense is this threat "controlled for?" (6 points)

c) In general, do you think these results have relatively high or low external validity? Why?????h or low external validity? Why?

Answer rating: 100% (QA)

a The best estimate of the quantitative effects of the orange submit order button can be calculated ... View the full answer