Independent ttest Module IXC Independent t-test Module IX-C Grenith J. Zimmerman, Ph.D. Associate Dean for Research Copyright 2012 1 Two-Sample Problems Independent t-tests The goal of inference is to compare the responses to two treatments or to compare the characteristics of two populations. We have a separate sample from each treatment or each population. - Each sample is separate. The units are not matched, and the samples can be of differing sizes. 2 Example Exercise and Pulse Rates 3 1 Independent ttest Module IXC H0 : mean exercisers = mean non exercisers n Exercisers Mean SD 29 66 8.6 Non-Exercisers 31 75 9.0 4 Conditions for Comparing Two Means We have two independent Random Samples, from two distinct populations - that is, one sample has no influence on the other--matching violates independence - we measure the same variable for both samples. Both populations are Normally distributed - the means and standard deviations of the populations are unknown - in practice, it is enough that the distributions have similar shapes, the data have no strong outliers and the sample sizes are large. 5 Hypotheses Statistic: Sampling Distribution - Normal - - 6 2 Independent ttest Module IXC 7 Two-Sample t Procedures In order to perform an inference on the difference of two means (1-2), we'll need the standard deviation of the observed difference X1-X2: 8 Two-Sample t Procedures Problem: We don't know the population standard deviations 1 and 2. Solution: Estimate them with S1 and S2. The result is the estimated standard error of the difference in the sample means. 9 3 Independent ttest Module IXC Test Statistic for Separate Variable Estimates Is only approximated by a t-distribution df calculated by a formula Conservative degrees of freedom: smaller of n1-1and n2-1. 10 For normal distributions the mean and standard deviation are independent. 1 2 11 Assuming Equal Variances 12 4 Independent ttest Module IXC Assuming Equal Variances If we can assume equal variances for the two independent populations, we can estimate the standard error using a pooled standard deviation estimate. 13 so Equal sample sizes 14 Two-Sample t Significance Tests Draw a Random Sample of size n1 from a Normal population with unknown mean 1, and draw an independent Random Sample of size n2 from another Normal population with unknown mean 2. To test the hypothesis H0: 1 = 2 , the test statistic is: 15 5 Independent ttest Module IXC Comparing Two Population Standard Deviations Suppose we have independent Random Samples from two Normal populations with unknown means and standard deviations: a sample of size n1 from N (1,1) and a sample of size n2 from N (2,2). To test the hypothesis of equal spread, use the ratio of sample variances, called the F-statistic: 16 Characteristics of F Distributions Density curve for the F (9, 10) distribution 17 H0: 12 = 22 vs H1: 12 22 Rejection Region Acceptance Region Rejection Region 18 6 Independent ttest Module IXC SPSS Test for Equality of Variance Test for Equality of Variances Floor Space Equal Variances assumed Equal variances not assumed F Sig. 1.438 .246 .246 >.05 19 Example Exercise and Pulse Rates A study is performed to compare the mean resting pulse rate of adult subjects who regularly exercise to the mean resting pulse rate of those who do not regularly exercise. n Mean SD Exercisers 29 66 8.6 Non-Exercisers 31 75 9.0 20 Test Statistic 21 7 Independent ttest Module IXC Calculate 22 Calculate t 23 -3.96 -2.00 0 t 2.00 24 8 Independent ttest Module IXC Conclusion Statistical: Reject H0 The means are statistically different. Content area: The exercisers have significantly lower mean resting pulse rates than nonexercisers. 25 9 Module IX-C Independent t-test Slide 1 No Comments Slide2 The independent t-test is used when we wish to compare response to treatment between two randomly divided groups which have been given two different treatments or when we wish to compare two independent groups on some demographic characteristic. Since the samples are separate and the units are not matched, the sample sizes for the two samples for an independent t-test do not have to be the same size, however, if the variability in the two samples is approximately equal, we will get the best results from samples which are about the same size. The size of the smaller sample has the most effect on our ability to find differences. Slide 3 An example of research where an independent t-test would be the appropriate test for analysis is given in this slide. In later slides, we will do the calculations for this problem. A study was performed to compare the mean resting pulse rate of adults who regularly exercise with adults who do not regularly exercise. Slide 4 The table gives us the sample sizes, means and standard deviations for the data that were collected. The null hypothesis would be that the means for the exercisers and non-exercisers are equal. Depending on our knowledge of the effects of exercise on resting pulse rate, we may decide to do a one-tailed or a two-tailed test. Slide 5 The conditions or assumptions for doing an independent t-test are that 1) we are measuring the same quantitative variable on two independent random sample (in many cases, we will be looking at groups that were randomized from a representative sample from some population). 2) Both populations are normally distributed with unknown means and standard errors (in practice, since the t-test is robust to deviations from the assumptions, it is enough that the shapes of the distributions be similar, the data have no strong outliers, and the sample sizes not be too small). Slide 6 The steps in the hypothesis testing process would be: First, state the null and alternative hypotheses. The null hypothesis is that 1 and 2 are equal or that their difference is 0. If we are doing a two-tailed test our alternative hypothesis would be the 1 and 2 are not equal, or that their difference is not 0. The statistic that we use to estimate 1 - 2 is X bar 1 - X bar 2, since X bar1 is an unbiased estimator of 1and X bar 2 is an unbiased estimator of 2. If the distributions we are sampling are normal, the sampling distribution for X bar 1 - X bar 2 is normal. This sampling distribution has mean 1 - 2, which says that the difference in the sample means is an unbiased estimator of the difference in population means. The standard error for the difference in sample means is the square root of the quantity sigma 1 squared/n1 plus sigma 2 squared/n2. You will recognize that sigma 1 squared/n1 is the variance of X bar 1 and similarly for X bar 2. In statistics, variances add, standard deviations do not add; so, we get the standard deviation of X bar 1 - X bar 2 by adding the variances of X bar 1 and X bar 2 and then taking the square root to get the standard error. Slide 7 We can make a Z-variable from X bar 1 - X bar 2 by subtracting the mean 1 - 2 and dividing by the standard error that we have just calculated. Slide 8 In order to test the hypothesis of equality of means, we will need to calculate the standard error of the difference in X bars, but this standard error contains the two unknown parameters sigma 1 and sigma 2. Slide 9 The problem is that we do not know the population standard deviations. When we faced this problem with the one sample t-test, we estimated the population standard deviation by the standard deviation S calculated from the sample data. We could do this again, estimating sigma 1 by S1, the standard deviation from the first sample and sigma 2 by S2, the standard deviation from the second sample. This would give us an estimate of the standard error of the difference in sample means as the square root of the quantity S1 squared/n1 + S2 squared/n2. Slide 10 Substituting this estimate in the denominator of the Z-statistic formula, we get a test statistic for the difference in sample means, X bar 1 - X bar 2. The problem is that this is not a t-statistic. Mathematical statisticians worked for many years trying to determine the sampling distribution of this statistic. What they found was that it could be approximated by a tdistribution with degrees of freedom calculated by a fairly complicated formula. Some text books give the formula and some text books suggest a conservative approach by using the smaller of n1 -1 and n2-1 as the degrees of freedom for the test. The good news is that statistical computer programs for independent t-tests do know and use the correct formula for degrees of freedom and most research projects will use the computer for data analysis. Slide 11 Remember that the mean and standard deviation for the normal distribution are two independent parameters. We can have two normal distributions with the same mean and different standard deviations and we can also have two distributions with the same standard deviations, but with different means. Many times in research projects we have the situation that the characteristic or variable measured on two independent populations will have different means, but the variability or standard deviation will be essentially the same. Slide 12 In this case, we can do an independent t-test, assuming that sigma 1 and sigma 2 are equal. Then, the standard error of the difference in X bars, namely, the square root of the quantity Sigma 1 squared/n1 plus Sigma 2 squared/n2, can have the subscripts 1 and 2 removed from the sigmas and the sigma can be removed from under the square root sign because it is common to both terms. This gives a standard error, under the assumption of equal variances of Sigma times the square root of the quantity 1/n1 + 1/n2. Slide 13 If we can assume equal variances, we only have one parameter to estimate, namely sigma, the common variance for the two populations. We can estimate this value by a pooled standard deviation estimate S sub p which is the square root of the quantity (n1-1) times S1 squared + (n2-1) times S2 squared all divided by n1+n2-2. Slide 14 Let's look at the formula on the last slide and see if we can understand it better. First, the formula for variance is the sum of the squared deviations X-Xbar squared divided by n-1, the number of degrees of freedom. If we multiply S squared by the quantity n-1, we get the sum of the squared deviations. So the numerator of the formula on the previous slide is just the sum of the square deviations from each sample. Since the variances are equal, both of these sums of squares are estimating the unknown population standard deviation sigma. Secondly, n1 plus n2 -2 is really n1-1 the degrees of freedom for the first sample, plus n2-1, the degrees of freedom for the second sample. Finally, if the sample sizes are equal, the pooled standard deviation will be simply the square root of the mean of the two variances. If the sample sizes are different, the most weight in the weighted mean of the variances will be for the sample with the largest size. This makes sense, because the largest sample will give you the best estimate of the parameter sigma. Slide 15 Finally, we present the formula for the t-statistic for independent samples, when we assume that the variances are equal. For testing the hypothesis that 1 = 2, the Numerator = the quantity (X bar1 - Xbar 2) - the quantity (1 - 2) and denominator = the pooled standard deviation times the quantity square root of 1/n1 +1/n2. This t-statistic has n1 + n2 -2 degrees of freedom. We can find the critical values for this statistic using the t-distribution table. Slide 16 How do we know whether we can make the assumption of equal variances when we do our independent t-test. If we have independent random samples from two Normal populations with unknown means and standard deviations, we can test the hypothesis of equal variances sigma1 squared = sigma 2 squared versus the alternative hypothesis sigma 1 squared is not equal to sigma 2 squared using the ratio of sample variances S1 square divided by S2 squared. This ratio of variances has an F distribution which we can use to run the hypothesis test for equal variances. Slide 17 Just as there is a whole family of t-distributions with an index called degrees of freedom, there is a whole family of Fdistributions indexed by two parameters, degrees of freedom for the Variance estimate in the numerator and degrees of freedom for the Variance estimate in the denominator. This graph is of the F distribution with 9 degrees of freedom for S1 squared and 10 degrees of freedom for S2 squared. Slide 18 F distributions are right-skewed with area under the density curve equal to 1, which is total probability. The F distribution being the ratio of two positive numbers, takes on only positive values, The peak of the density curve is near 1. If Sigma 1 squared and Sigma 2 squared are equal (as hypothesized under the null hypothesis) we would expect the F-statistic S1 squared/S2 squared would be close to 1. Large values or very small values would result in the rejection of the null hypothesis. If in calculating the F-statistic, we always put the larger variance in the numerator, we will reject the null hypothesis of equal variances when the F-statistic value is too large. Slide 19 When we do our data analysis on SPSS, the computer output gives an F test for equality of variances and then, gives computer output for both the Independent t-test assuming equal variances and the Independent t-test for separate variance estimates. When we interpret the computer output, we first look at the F-test for equal variances and see if the difference in variances is significant (i.e. <.05). If the difference in variances is significant, we use the separate variances t-test. If the difference in variances is not significant, we use the equal variances t-test. Slide 20 For convenience, we repeat the slide with the data for the comparison of mean resting pulse rate between exercisers and non-exercisers. The mean for the 29 Exercisers is 66 and the standard deviation is 8.6 beats per minute. For the nonExercisers, the mean is 75 and the standard deviation is 9.0 beats per minute. We will assume that the variances for the two groups are equal. Slide 21 The null hypothesis is that the population mean resting pulse rate for exercisers is equal to the mean resting pulse rate for non-Exercisers. We will be doing a two-tailed test, i.e. that the means are not equal. We chose a level of significance of .05. The test statistic will use the pooled standard deviation in the denominator. Slide 22 We first calculate the pooled variance S sub p squared. S sub p square is the quantity (29-1) time 8.6 squared plus (31-1) time 9.0 squared divided by 29+31 -2, which gives us the quantity 2070.88 plus 2430.0 divided by 58, which equals 77.60. Taking the square root to get the pooled standard deviation gives 8.8 beats per minute. Slide 23 We now calculate the t-statistic for this problem. t equals the difference in the sample mean (66-75) minus zero, the hypothesized difference in the population means, divided by 8.8 the pooled standard deviation times the square root of the quantity 1/29 + 1/31. This gives us -9 divided by 8.8 time .258 = -9 divided by 2.27 = -3.96. Slide 24 For = .05 and 58 degrees of freedom, the critical values for 60 degrees of freedom are +/-2.00. The statistic value calculated from our data is -3.96, which puts us in the Rejection Region. Slide 25 Our statistical conclusion is that we Reject the null hypothesis of equal means. The Mean resting pulse rates for Exercisers and non-Exercisers are different. We translate this into the content area by saying that Exercisers have significantly lower mean resting pulse rates than non-Exercisers. After we have collected our data, we know the direction of the difference. Paired t-test Module IX-B Slide 1 No comments Slide 2 In the next two modules, we will consider two sample t-tests. There are two of them, the paired t-test and the Independent t-test. In this module we will discuss the Paired t-test. Slide 3 The paired t-test is the appropriate analysis when we want to compare two measurements of an outcome variable measured on the same person under two different circumstances or when we are comparing an outcome variable between two different subjects who have been matched on potential confounding variables, such as gender, age, diagnosis, or level of severity. The first situation, that is having two measurements of the outcome variable for the same person can happen in one of two ways. A very common research design collects data on the same individual at two different points in time, frequently before and after some type of treatment. The second situation occurs when an individual is tested under two different conditions, such as walking in a lighted environment vs a darkened environment, or walking on a hard surface vs walking on a carpeted surface. In the matched pairs design, subjects are first matched on relevant confounding variables. This makes the two subjects more alike than if they had each been selected randomly from a population. Subjects in each pair are then randomly assigned to two different treatment conditions. Slide 4 Consider a group of stoke patients. If we studied reaction time before and after four treatment sessions, we would have a repeated measures design. Slide 5 We would also have a repeated measures design if we compared how long it takes to walk 6 feet with a walker on a hard surface vs a carpeted surface. Slide 6 Or if we compared the distance that could be walked in three minutes with a wheeled walker vs a standard walker. Slide 7 Each of these studies would analyze their data using the paired t-test on the differences between the two measurements. n, the sample size is the number of pairs in the study. Slide 8 We would have a matched pairs design, if, in a weight loss study, subjects were matched on age, gender and amount of weight that needed to be lost and then randomly assigned to two different weight loss groups. Slide 9 We would also have a matched pairs design if we did a study to determine whether a live review session or a review book was more helpful in passing boards, and subjects were matched on gender, and program GPA. Slide 10 After matching, we would randomly assign the subjects to the two different groups. The analysis would be a paired t-test on the differences between the measurements on the paired individuals. Again, n, the sample size, is the number of pairs in the study. Slide 11 In order for a Paired t-test to be the appropriate analysis for a data set, the data should be a random sample of matched pairs, which is representative of some population. In deciding whether the paired t-test is appropriate, we also need to consider the size of the sample and whether or not there are serious outliers in the sample. If n, the sample size, is less than 15, we should use the t-statistic only if the distribution of the differences is symmetric and there are no outliers. If n is greater than or equal to 15, we can use the tstatistic if the distribution of differences has no outliers and is not strongly skewed. For samples with greater than or equal to 40 pairs of subjects, the paired t-statistic can be used unless the distribution has many outliers. Since we are looking at comparing sample means, the level of measurement needs to be quantitative. Slide 12 This slide looks at the data that is needed to use the paired t-statistic. It is set up as if the research design involves before and after measurements with some intervention between measurements. We have time 1 (before) measurements on n subjects indicated by X11, X12, ..., X1n. The after measurements X21, X22,...X2n are measurements after the intervention for the same group of n subjects. These measurements are paired: X11 and X21 are on the same person, namely subject #1, X12 and X22 on subject #2 and so on. For each of the pairs of measurements, we calculate the difference. The data analysis is actually run on the sample of n differences from the n subjects in the study. Note that if a subject has a before (time 1) measurement, but no after (time 2) measurement, his/her data will not be included in the data analysis. Slide 13 In order to write the t-statistic formula for doing data analysis, we need to define some notation. sub d is the mean difference for the population of paired data; d bar is the mean of the sample of differences; S sub d is the standard deviation calculated from the differences; and n, the sample size is the number of pairs, or correspondingly, the number of differences. Slide 14 The null hypothesis for the Paired t-test is the mean of the differences is equal to 0. This slide uses a two tailed test, and the alternative hypothesis is the mean of the differences is different from 0, but in many circumstances this could be replaced by an appropriate one-tailed test. The statistic that we use for our analysis is d bar, the mean of the sample of differences. Slide 15 When we do the statistical test using the paired t-statistic, we are actually doing a one sample t-test on the sample of differences. The t-statistic in this slide should have a form which is familiar. If we look at our sample of differences, we calculate the t-statistic by subtracting the population mean difference from d bar, the sample mean, and divide by the standard error, which is the standard deviation of the differences divided by the square root of n. The degrees of freedom for this statistic are n-1. Slide 16 Let's look at an example. Suppose that we have a group of 16 stroke patients in our study. We are interested in the outcome or dependent variable, reaction time for responding to a computer command measured in seconds. We will measure our outcome variable before and after four treatment sessions. Our hypothesis is that there will be a decrease in mean reaction time from the before measurement to the after measurement. Slide 17 In symbols our null hypothesis is the after and before are equal or that the mean difference sub d is 0. In this case, it makes sense to use a one-tailed alternative. We do not expect that our treatments will increase the reaction time of our subjects. So, the alternative hypothesis is that the after is less than the before or that the mean difference (if we subtract after - before measurements) is less than 0. It is important when you do one-tailed tests to be consistent in how the hypothesis is stated and the order of the subtraction. Some data is given on this slide for the before and after measurements and the differences have been calculated. Remember, we will be using the sample of differences for our statistical testing. Slide 18 The data set for this problem consisted of data from 16 subjects and the values needed for the analysis were calculated and are given on this slide. The mean of the before measurements was 41.63 seconds, for the after measurements was 37.38 seconds, so it looks like the data supports our alternative hypothesis of a decrease. The analysis will be done on the sample of differences so the important statistics to calculate are the mean and standard deviation of the sample of differences. We find that d bar, the mean of the differences is -4.25 and the standard deviation of the differences S sub d is 3.697. Slide 19 We now substitute these values into the t-statistic formula. Remember that the null hypothesis is that the means are equal, so the difference in means that is subtracted from the quantity d bar 2 is 0. We divided by the standard error of the differences. When we do this, we obtain t=-4.599. Since the sample size for this problem is 16, the degrees of freedom for the t-statistic will be 16-1 = 15. Slide 20 The critical value will be in the lower tail of the t-distribution, since the alternative hypothesis is the sub d is less than zero. If our level of significance is .05, we find that the critical value for this problem is 1.753. Slide 21 Our test statistic value is -4.599 which is much less than the critical value of -1.753 and in the Rejection Region. Slide 22 We now need to interpret this statistical result back into the context of the problem. Remember, we were looking at reaction time for a group of 16 stroke patients before and after 4 treatments designed to improve reaction time. Our conclusion, Reject the null hypothesis, means that there has been a significant decrease in mean reaction time for these stoke patients after 4 treatments. Paired t-Test Module IX-B Paired t-test M odule IX-B Grenith J. Zimmerman, Ph.D. Associate Dean for Research Copyright 2012 1 H ypothesis T esting: Two Sam t-tests ple Paired t-test Independent t-test 2 Paired t-test Repeated M easures/ M atched Pairs Repeated Measures - Data is collected on the same individual at two points in time - Data is collected on the same individual under two different conditions Matched Pairs - Subjects are matched into pairs on confounding variables - Subjects in each pair are randomly assigned to one of two treatments 33 1 Paired t-Test Module IX-B E ple 1 Repeated M xam : easures Pre-test 4 treatments Post-test Independent Variable = Time 44 Hard Surface Carpeted Surface Independent Variable = Type of Surface 55 Wheeled Walker Standard Walker Independent Variable = Type of Walker 66 2 Paired t-Test Module IX-B Analysis of Data Paired t-test on the differences between measurements n=number of pairs 77 E ple 2: M xam atched Pairs 88 Female GPA : 3.2 Female GPA : 3.3 Male GPA: 3.5 Male GPA : 3.6 99 3 Paired t-Test Module IX-B Randomly assign Analysis would be a paired t test Sample size is number of pairs 100 1 Paired t-test The sample data consist of a random sample of matched pairs. n < 15 Use t-statistic only if the distribution of differences is symmetric with no outliers. n 15 Use t-statistic if distribution of differences has no outliers and is not strongly skewed. n 40 Use t-statistic unless the distribution of differences has many outliers. 111 1 H ypothesis T esting for the Paired t-test: The Data Before After Difference X11 - X21 = d1 X12 - X22 = d2 X1n - X2n = dn 122 1 4 Paired t-Test Module IX-B N otation for M atched Pairs = mean value of the differences d for the population of paired data = mean value of the differences d for the paired sample data (equal to the mean of the X1 - X2 values) = standard deviation of the differences d for the paired sample data = number of pairs of data 133 1 Paired t-test Hypotheses: Statistic = 144 1 T Statistic for M est atched Pairs of Sam Data ple where degrees of freedom = n - 1 155 1 5 Paired t-Test Module IX-B E ple: Paired t-test xam Group of 16 stroke patients Outcome: Reaction Time (sec) Measured before and after 4 treatment sessions Hypothesize there will be a decrease in mean reaction time 166 1 Before -7 -5 48 50 +2 40 Sd = Difference 38 33 Data After 45 38 34 -6 (d i d )2 161 177 1 = 3 . 697 188 1 6 Paired t-Test Module IX-B df = 16 - 1 = 15 199 1 df = 15 = .05 200 2 =.05 -4.599 -1.753 Rejection Region Acceptance Region 211 2 7 Paired t-Test Module IX-B Interpretation: There has been a significant decrease in mean reaction time for these stroke patients after 4 treatments. 222 2 8 Health Care Research & Statistics SPSS Assignment 9A ** Look at the SPSS modules for the Paired t-test and the Independent t-test. Use SPSS to analyze the following problems. Include the following steps in the analysis. 1. State the null and alternative hypotheses 2. Make an appropriate data file for the data 3. Complete the appropriate data analysis 4. Did you reject or fail to reject the null hypothesis? What is the basis for this decision? 5. Interpret your results back into the context of the problem. page 495: 32 page 513: 34 page 526: 1, 2, 3 It is often said that when students go from college to graduate school, their GPAs improve. Test this hypothesis using data from the following groups of students. Seniors: 2.9, 3.0, 3.5, 2.6, 3.7, 3.0, 3.3, 3.4, 2.8, 3.1 Graduate students: 3.5, 4.0, 3.3, 3.5, 3.2, 3.5, 3.6