Problem $2$: Do consumers react to faster shipping times?

Definitions

"Visit" $-$ an instance of a person coming to a website. $($A "visitor" is a person who conducts a visit on a website.$)$

"Order" $-$ an instance of someone making a purchase.

"Conversion rate" $-$ the percentage of visits that turn into an order $($aka cvr$).$

Notes about Split Testing $($aka A$/$B testing$)$

A website can be set to give different experiences to different people. In other words, the site

can be "split" into two versions $("$A$"$ and $"$B$")$ that have different characteristics $($e$.$g$.$ layout, products, etc.$).$

These different versions can then be shown to different visitors on a random basis.

For example, we could set our random number generator to aim for $75\%$ of visitors to see version A and

$25\%$ of visitors to see version B$.$

Scenario

We would like to understand how consumers react to seeing different anticipated shipping times. Does showing

a faster shipping time result in more consumers choosing to hit the "buy" button?

A two week split test was set up where visitors were assigned to either an A group, or a B group,

and the number of visitors in each group was accurately recorded by day. The A group sees

a "Free two day shipping" badge. The B group sees our regular "Free shipping" badge.

The additional traffic on the $7$th and $14$th is due to some banner ads running on Weddings.com on those particular days.

A group B group A B

Visits Orders Visits Orders conversion rate conversion rate # of orders in B normalized to number of visits

$6/4/20237,8237962,91028910\mathrm{.}2\%9\mathrm{.}9\%777$

$6/5/20235,6115413,0492629\mathrm{.}6\%8\mathrm{.}6\%482$

$6/6/20235,0925332,77528010\mathrm{.}5\%10\mathrm{.}1\%514$

$6/7/202316,4071,0013,2661916\mathrm{.}1\%5\mathrm{.}8\%960$

$6/8/20234,0724161,98018810\mathrm{.}2\%9\mathrm{.}5\%387$

$6/9/20232,8022681,5121299\mathrm{.}6\%8\mathrm{.}5\%239$

$6/10/20233,2773231,4081349\mathrm{.}9\%9\mathrm{.}5\%312$

$6/11/20238,1598082,7092589\mathrm{.}9\%9\mathrm{.}5\%777$

$6/12/20235,3315172,8022589\mathrm{.}7\%9\mathrm{.}2\%491$

$6/13/20235,2175422,72027210\mathrm{.}4\%10\mathrm{.}0\%522$

$6/14/202315,9221,0993,1192056\mathrm{.}9\%6\mathrm{.}6\%1046$

$6/15/20234,3604152,0911829\mathrm{.}5\%8\mathrm{.}7\%379$

Totals $84,0737,25930,3412,6486,885\mathrm{.}53$

Mean $9\mathrm{.}4\%8\mathrm{.}8\%$

Std$.$ Dev $1\mathrm{.}4\%1\mathrm{.}3\%$

Please show all your analysis, leave formulas in cells, and specify any assumptions you make.

Questions

A$)$ What is the overall conversion rate for the A group?

B$)$ What is the overall conversion rate for the B group?

C$)$ Which version of the shipping badge performs better?

D$)$ What $\%$ more orders can you expect to get by using the version you answered in $($C$)$ above, vs the other version?

E$)$ What is a little strange about this data? Why is this happening?