{"id":973,"date":"2020-06-02T07:49:50","date_gmt":"2020-06-02T11:49:50","guid":{"rendered":"http:\/\/pressbooks.library.upei.ca\/montelpare\/?post_type=chapter&p=973"},"modified":"2020-08-30T08:49:53","modified_gmt":"2020-08-30T12:49:53","slug":"the-t-test-for-independent-sample-means-and-pooled-versus-unpooled-variance","status":"publish","type":"chapter","link":"https:\/\/pressbooks.library.upei.ca\/montelpare\/chapter\/the-t-test-for-independent-sample-means-and-pooled-versus-unpooled-variance\/","title":{"raw":"The t-test for Independent Sample Means and Pooled Versus Unpooled Variance","rendered":"The t-test for Independent Sample Means and Pooled Versus Unpooled Variance"},"content":{"raw":"
\r\n

Learner Outcomes<\/p>\r\n\r\n<\/header>\r\n

\r\n\r\nAfter reading this chapter you should be able to:\r\n
    \r\n \t
  1. Compute the significance of the difference between two sample means when the sample variances are different<\/li>\r\n \t
  2. Compute the t-test for independent sample means<\/li>\r\n \t
  3. Compute the t-tests for pooled versus un-pooled variance.<\/li>\r\n \t
  4. Write a SAS program to compute and identify the important elements of the output for the computation<\/li>\r\n<\/ol>\r\n<\/div>\r\n<\/div>\r\n

    Applications of the t-test under different research scenarios<\/h2>\r\nIn statistics, the t-test has a simple approach despite that it uses a variety of error terms in the denominator as shown in Table 30.1, below. Depending on the research design, the error term will differ to ensure that the appropriate variance estimates within each of the samples are included in the analyses. The following equations demonstrate the different error terms related to the types of comparisons.\r\n\r\nTable 30.1 A Summary of t-test Formulae<\/strong>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    t-test descriptions<\/td>\r\nAppropriate t-test formula<\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Evaluation of the single sample mean versus the mean for a population<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n
    The pairwise t-test uses the average difference in the measure of interest, from the pre-test score to the post-test score, and then divided by the standard error of the average difference.\u00a0 The standard error of the average difference is computed by dividing the standard deviation of the average difference by the square root of the number of cases in the pairwise comparison.<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n
    To evaluate the significance of the difference between two mean scores (regardless of the size of \"n\" in each level of the independent variable) we might consider using a pooled t-test for independent variables.<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n
    To evaluate the significance of the difference between two mean scores (regardless of the sample size \"n\") when we consider un-pooled or unequal variances<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nIn Figure 30.1 below we see the two dimensions of variance that can contribute to the differences observed in a t-test calculation. As illustrated, not only does the t-test process the differences between means, but the difference is also influenced by the variability between members within each sample.\r\n

    \"\"<\/p>\r\n

    Figure 30.1 Comparison of Two Independent Means<\/p>\r\nNotice that the decision about which t-test formula to select is dependent on the research design created by the researcher. For example, as noted in table 30.1, to evaluate the significance of the difference between two mean scores (regardless of the size of \"n\" in each level of the independent variables) we could consider using a t-test for independent variables, as noted in Table 30.1 formula 3. However, if the number of participants in the samples being compared is equal then it is more appropriate to use Table 30.1 formula 4.\u00a0 In this latter case, given that the sample size between the groups was equal and it is expected that the variance within the two groups is similar. In this case, we can pool or combine the variance estimates as we assume that the two groups have homogenous variance within and between the two groups.\r\n

    NB The term homoscedasticity is a term that indicates equality of variance between independent variables. The term is more often used to refer to the variance estimates for each independent variable in a statistical model, as in a multiple linear regression equation. Homoscedasticity suggests that the variables have the same variance.<\/span><\/div>\r\n
    Scenario 30.1.1 Comparing 2 groups with unknown variance and different sample sizes<\/strong><\/h6>\r\n
    \r\n\r\nConsider the following scenario in which there are two groups with different sample sizes (number of participants in each group) and the variance is unknown within each of the groups.\u00a0 In this situation, the analysis of data uses the t-test for two independent groups.\u00a0 We can use the formula for the t-test for independent groups (with unequal sample sizes) to compute the significance of the differences in the mean scores from group1 in which the sample size is 10 participants and the mean scores from group2 \u00a0where the sample size is comprised of 8 participants.\r\n\r\nTable 30.2 Comparing Responses for two groups with unequal sample sizes and unknown variance<\/strong>\r\n\r\nData for Group 1\r\n\r\n<\/div>\r\n\r\n\r\n\r\n\r\n
    Participant ID<\/th>\r\n001<\/td>\r\n002<\/td>\r\n003<\/td>\r\n004<\/td>\r\n005<\/td>\r\n006<\/td>\r\n007<\/td>\r\n008<\/td>\r\n009<\/td>\r\n010<\/td>\r\n<\/tr>\r\n
    Score<\/th>\r\n\u00a0234<\/td>\r\n254<\/td>\r\n\u00a0260<\/td>\r\n268<\/td>\r\n253<\/td>\r\n270<\/td>\r\n281<\/td>\r\n287<\/td>\r\n265<\/td>\r\n255<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nData for Group 2\r\n\r\n\r\n\r\n\r\n
    Participant ID<\/th>\r\n001<\/td>\r\n002<\/td>\r\n003<\/td>\r\n004<\/td>\r\n005<\/td>\r\n006<\/td>\r\n007<\/td>\r\n008<\/td>\r\n<\/tr>\r\n
    Score<\/th>\r\n304<\/td>\r\n235<\/td>\r\n212<\/td>\r\n198<\/td>\r\n273<\/td>\r\n289<\/td>\r\n301<\/td>\r\n209<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nThe degrees of freedom for the two-group t-test is df=n1<\/sub>+n2<\/sub>-2, and in this case df=10-8-2=16 so that the t critical value based on an alpha level of 0.05<\/em> is 2.12.\r\n\r\nIn the following SAS code, we compute the difference between the means for the data in Table 30.2.\u00a0 Here we include a grouping variable so that we can distinguish the data for each group before computing the t-test.\r\n
    \r\n

    T-test for the difference between means with unequal sample size<\/p>\r\n\r\n<\/header>\r\n

    \r\n
    OPTIONS PAGESIZE=65 LINESIZE=80;\r\nDATA INDT_TST;\r\nINPUT\u00a0 ID GROUP SCORE @@;\r\nDATALINES;\r\n001 1 234\u00a0002 1 254\u00a0003 1 260\u00a0004 1 268\u00a0005 1 253\u00a0006 1 270\u00a0007 1 281\u00a0008 1 287\u00a0009 1 265\u00a0010 1 255\u00a0011 2 304\u00a0012 2 235\u00a0013 2 212\u00a0014 2 198\u00a0015 2 273\u00a0016 2 289\u00a0017 2 301\u00a0018 2 209\r\n;\r\nPROC SORT DATA=INDT_TST; BY GROUP;\r\nPROC TTEST; CLASS GROUP; VAR SCORE;\r\nRUN;<\/div>\r\n<\/div>\r\n<\/div>\r\nThe output for the PROC T-TEST procedure for this independent t-test analysis is shown below. The INDEPENDENT t-test Procedure.\r\n\r\n\r\n\r\n\r\n\r\n
    GROUP<\/td>\r\nN<\/td>\r\nMEAN<\/td>\r\nSTD<\/td>\r\nSTD ERR<\/td>\r\nMINIMUM<\/td>\r\nMAXIMUM<\/td>\r\n<\/tr>\r\n
    1<\/td>\r\n10<\/td>\r\n262.7<\/td>\r\n\u00a015.17<\/td>\r\n4.79<\/td>\r\n234.0<\/td>\r\n287.0<\/td>\r\n<\/tr>\r\n
    2<\/td>\r\n8<\/td>\r\n252.6<\/td>\r\n44.02<\/td>\r\n15.56<\/td>\r\n198.0<\/td>\r\n304.0<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n\r\n\r\n\r\n\r\n\r\n
    GROUP<\/td>\r\nMEAN<\/td>\r\nSTANDARD DEVIATION<\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Diff (1-2)<\/td>\r\n10.08<\/td>\r\n31.26<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    group<\/td>\r\nMethod<\/td>\r\nMean<\/td>\r\nStd\u00a0Dev<\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    1<\/td>\r\n<\/td>\r\n262.7<\/td>\r\n\u00a015.17<\/td>\r\n<\/tr>\r\n
    2<\/td>\r\n<\/td>\r\n252.6<\/td>\r\n44.<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/td>\r\nPooled<\/td>\r\n10.0750<\/td>\r\n31,26<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/td>\r\nSatterthwaite<\/td>\r\n10.0750<\/td>\r\n47.37<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n
    Method<\/td>\r\nVariances<\/td>\r\nDF<\/td>\r\nt\u00a0Value<\/td>\r\nPr\u00a0>\u00a0|t|<\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Pooled<\/td>\r\nEqual<\/td>\r\n16<\/td>\r\n0.68<\/td>\r\n0.51<\/td>\r\n<\/tr>\r\n
    Satterthwaite<\/td>\r\nUnequal<\/td>\r\n8.33<\/td>\r\n0.62<\/td>\r\n0.55<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nEquality of Variances\r\n\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n
    Method<\/td>\r\nNum\u00a0DF<\/td>\r\nDen\u00a0DF<\/td>\r\nF Value<\/td>\r\nPr\u00a0>\u00a0F<\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Folded F<\/td>\r\n7<\/td>\r\n9<\/td>\r\n8.42<\/td>\r\n0.0049<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\nTo determine if the mean scores in each group were significantly different we typically compare the t-observed to the t critical values, generally available in a reference table. For example, the t critical value for \u03b1 = 0.05, where df=9 is 2.262 for a two-tailed test and 1.833 for a one-tailed test. \u00a0In the SAS output shown here, a t-critical is not given, however, the probability of achieving the t-value that was computed is reported and this is the indicator of significance. That is to say, when the Pr\u00a0>\u00a0|t| is greater than 0.05, as it is in this instance, we would accept the null hypothesis that there is no difference between the mean scores in each group.\r\n\r\nAdditionally, in the SAS output, we observe a t-value for both a pooled variance estimate and for an un-pooled variance estimate, where the Satterthwaite t Value estimates the unequal\/un-pooled variance. As a general rule because the Folded F stat is a test of unequal variances, when the folded F statistic<\/strong> is large and the p-value is <0.05, as shown in the SAS output above, then we refer to the Satterthwaite unequal variances estimate to determine the decision rule regarding the comparison of means via the t-test.\r\n\r\n30.2 On the importance of p-values\u00a0\u00a0\u00a0<\/strong>\r\n\r\nIn the following data set there were 2 groups of 15 individuals.\u00a0 A test was conducted and each individual produced a score.\u00a0 The means were then computed for the scores in each group and a t-test was used to determine if there was a significant difference between the means for each group.\u00a0 The null hypothesis was given as:\u00a0 H0<\/sub>: mean for group1 = mean for group2\r\n\r\nData in Scenario 1:<\/strong>\r\n
    \r\n\r\nThe data for each group is shown here using the format (id, group, score): <\/strong>\r\n\r\n001 01 12,<\/strong> 002 01 25, 003 01 26, 004 01 23, 005 01 14, 006 01 15, 007 01 17, 008 01 11, 009 01 18, 010 01 14, 021 01 25, 023 01 28, 025 01 26, 027 01 23, 029 01 24 011 02 15, 012 02 34, 013 02 39, 014 02 35, 015 02 34, 016 02 33, 017 02 15, 018 02 31, 019 02 13, 020 02 20, 022 02 16, 024 02 22, 026 02 27, 028 02 26, 030 02 25\r\n\r\n<\/div>\r\nThe results of the t-test computation using SAS are shown here:\r\n\r\nThe UNIVARIATE Procedure \u2013 Data for the total group for Dependent Variable Score<\/strong>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    N<\/strong><\/td>\r\n30<\/td>\r\nSum Weights<\/strong><\/td>\r\n30<\/td>\r\n<\/tr>\r\n
    Mean<\/strong><\/td>\r\n22.8666667<\/td>\r\nSum Observations<\/strong><\/td>\r\n686<\/td>\r\n<\/tr>\r\n
    Std Deviation<\/strong><\/td>\r\n7.7135498<\/td>\r\nVariance<\/strong><\/td>\r\n59.4988506<\/td>\r\n<\/tr>\r\n
    Skewness<\/strong><\/td>\r\n0.25907362<\/td>\r\nKurtosis<\/strong><\/td>\r\n-0.8503857<\/td>\r\n<\/tr>\r\n
    Uncorrected SS<\/strong><\/td>\r\n17412<\/td>\r\nCorrected SS<\/strong><\/td>\r\n1725.46667<\/td>\r\n<\/tr>\r\n
    Coeff Variation<\/strong><\/td>\r\n33.7327251<\/td>\r\nStd Error Mean<\/strong><\/td>\r\n1.40829508<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\nThe t-TEST Procedure <\/strong>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    group<\/strong><\/td>\r\nN<\/strong><\/td>\r\nMean<\/strong><\/td>\r\nStd\u00a0Dev<\/strong><\/td>\r\nStd\u00a0Err<\/strong><\/td>\r\nMinimum<\/strong><\/td>\r\nMaximum<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    1<\/strong><\/td>\r\n15<\/td>\r\n20.0667<\/td>\r\n5.8244<\/td>\r\n1.5039<\/td>\r\n11.0000<\/td>\r\n28.0000<\/td>\r\n<\/tr>\r\n
    2<\/strong><\/td>\r\n15<\/td>\r\n25.6667<\/td>\r\n8.5161<\/td>\r\n2.1988<\/td>\r\n13.0000<\/td>\r\n39.0000<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/strong><\/td>\r\n<\/td>\r\n-5.6000<\/td>\r\n7.2955<\/td>\r\n2.6639<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    group<\/strong><\/td>\r\nMethod<\/strong><\/td>\r\nMean<\/strong><\/td>\r\n95% CL of the Mean<\/strong><\/td>\r\nStd Dev<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    1<\/strong><\/td>\r\n<\/td>\r\n20.0667<\/td>\r\n16.8412<\/td>\r\n23.2921<\/td>\r\n5.8244<\/td>\r\n<\/tr>\r\n
    2<\/strong><\/td>\r\n<\/td>\r\n25.6667<\/td>\r\n20.9506<\/td>\r\n30.3827<\/td>\r\n8.5161<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/strong><\/td>\r\nPooled<\/strong><\/td>\r\n-5.6000<\/td>\r\n-11.0568<\/td>\r\n-0.1432<\/td>\r\n7.2955<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/strong><\/td>\r\nSatterthwaite<\/strong><\/td>\r\n-5.6000<\/td>\r\n-11.0893<\/td>\r\n-0.1107<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n
    Method<\/strong><\/td>\r\nVariances<\/strong><\/td>\r\nDF<\/strong><\/td>\r\nt\u00a0Value<\/strong><\/td>\r\nPr\u00a0>\u00a0|t|<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Pooled<\/strong><\/td>\r\nEqual<\/td>\r\n28<\/td>\r\n-2.10<\/td>\r\n0.0447<\/td>\r\n<\/tr>\r\n
    Satterthwaite<\/strong><\/td>\r\nUnequal<\/td>\r\n24.746<\/td>\r\n-2.10<\/td>\r\n0.0459<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nEquality of Variances<\/strong>\r\n\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n
    Method<\/strong><\/td>\r\nNum\u00a0DF<\/strong><\/td>\r\nDen\u00a0DF<\/strong><\/td>\r\nF Value<\/strong><\/td>\r\nPr\u00a0>\u00a0F<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Folded F<\/strong><\/td>\r\n14<\/td>\r\n14<\/td>\r\n2.14<\/td>\r\n0.1675<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nSo for this output, we would <\/span>reject<\/strong> the null hypothesis and suggest that the mean for <\/span>Group 1<\/em> was significantly different than the mean for <\/span>Group 2<\/em>. <\/span>However, what would happen if in one of the groups we changed one of the scores by 5 points.\u00a0 Notice in the following data set for Scenario 2, all of the scores are exactly the same, except that we changed the data for participant 1 from 12 to 17.<\/span>\r\n\r\n<\/div>\r\nData in Scenario 2:<\/strong>\r\n
    \r\n\r\nThe data for each group are shown here using the format (id, group, score):<\/strong>\r\n\r\n001 01 17,<\/strong> 002 01 25, 003 01 26, 004 01 23, 005 01 14, 006 01 15, 007 01 17, 008 01 11, 009 01 18, 010 01 14, 021 01 25, 023 01 28, 025 01 26, 027 01 23, 029 01 24, 011 02 15, 012 02 34, 013 02 39, 014 02 35, 015 02 34, 016 02 33, 017 02 15, 018 02 31, 019 02 13, 020 02 20, 022 02 16, 024 02 22, 026 02 27, 028 02 26, 030 02 25\r\n\r\n<\/div>\r\nThe t-test output for Scenario 2 uses the exact same data set, except that the score for participant 1 in Group 1 was changed from a score of 12 to a score of 17. Notice the highlighted t values and the highlighted confidence intervals.\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    group<\/strong><\/th>\r\nMethod<\/strong><\/th>\r\nMean<\/strong><\/td>\r\n95% Confidence Limits for the <\/strong>Mean<\/strong><\/td>\r\n\u00a0Std Dev<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    1<\/strong><\/th>\r\n\u00a0<\/strong><\/th>\r\n20.4000<\/td>\r\n17.3755<\/td>\r\n23.4245<\/td>\r\n5.4616<\/td>\r\n<\/tr>\r\n
    2<\/strong><\/th>\r\n\u00a0<\/strong><\/th>\r\n25.6667<\/td>\r\n20.9506<\/td>\r\n30.3827<\/td>\r\n8.5161<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/strong><\/th>\r\nPooled<\/strong><\/th>\r\n-5.2667<\/td>\r\n-10.6175<\/td>\r\n0.0841<\/td>\r\n7.1538<\/td>\r\n<\/tr>\r\n
    Diff (1-2)<\/strong><\/th>\r\nSatterthwaite<\/strong><\/th>\r\n-5.2667<\/td>\r\n-10.6597<\/td>\r\n0.1264<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n
    \r\n\r\n\r\n\r\n\r\n\r\n\r\n
    Method<\/strong><\/th>\r\nVariances<\/strong><\/td>\r\nDF<\/strong><\/td>\r\nt\u00a0Value<\/strong><\/td>\r\nPr\u00a0>\u00a0|t|<\/strong><\/td>\r\n<\/tr>\r\n<\/thead>\r\n
    Pooled<\/strong><\/th>\r\nEqual<\/td>\r\n28<\/td>\r\n-2.02<\/td>\r\n0.0535<\/td>\r\n<\/tr>\r\n
    Satterthwaite<\/strong><\/th>\r\nUnequal<\/td>\r\n23.85<\/td>\r\n-2.02<\/td>\r\n0.0552<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\nIn this second example analysis, we would accept<\/strong> the null hypothesis indicating that there was no difference between the means. This decision is based on the comparison of the p-value to the accepted demarcation point of p<0.05.\r\n\r\nDespite that we have a demarcation point for the probability of the observed t-test, we need to consider the range of scores that we could have seen.\u00a0 The confidence interval provides such information for us.\u00a0 In the first example, the 95% confidence interval tells us that we are 95% confident that the difference between means could be somewhere between -11.09 and -0.11.\u00a0 However, in the second example the 95% confidence interval tells us that we are 95% confident that the difference between means could be somewhere between -10.66 and 0.126.\r\n\r\nSullivan and Feinn[2]<\/a> provide two quotes to support the need to look beyond the simple comparison of research findings to a p-value. The first quote is by Gene Glass who said,\u00a0\u201c<\/em>Statistical significance is the least interesting thing about the\u00a0<\/em>results. You should\u00a0 describe the results in terms of measures\u00a0<\/em>of magnitude \u2013 not just, does a treatment affect people, but\u00a0 <\/em>how much does it affect them.<\/em>\u201d.\u00a0 <\/em><\/strong><\/span>\r\n\r\nThe second quote is by Jacob Cohen, who said,\u00a0\u201c<\/span><\/strong><\/em>The primary product of a research inquiry is one or more\u00a0<\/em>measures of effect size, not P values.<\/em>\u201d. <\/em><\/span><\/strong>\r\n\r\nTwo important points of interest arise from this comparison.\u00a0 The first being that the intervals are not only similar in size but that they are similar in the bandwidth on the number line[latex]\\;\\rightarrow\\; [\/latex] lower limits being (-11.09 and -10.66) and upper limits being (-0.11 and 0.126).\u00a0 However, the second point of interest is that we only had to change one score from the entire set of 30 scores and only by 5 points in order to change from showing a significant difference to a non-significant difference.\r\n\r\nIf you consider the standard deviation for the scores in Group 1<\/em> in each trial, you will notice that a 5-point change is less than the score by which we expect any score to vary from the mean.\u00a0 That is, in the two examples the standard deviation is 5.82 and 5.46, respectively. Therefore, changing a score by less than the computed standard deviation was sufficient to cause a decision to change from significant<\/strong> to not significant<\/strong>.\r\n\r\nConsider that this was a study in which you invested millions of dollars.\u00a0 If you only relied on the p-value then you would be happy with scenario 1 (a significant difference was found) but you would be tremendously disappointed with scenario 2, and you would unnecessarily throw away valuable information.\u00a0 So a guiding principle may be that despite the reported value of p for any comparison, consider also the standard deviations and the standard errors along with the computed confidence intervals before reporting the findings.<\/span>\r\n\r\n30.3 Estimating the Effect Size<\/strong>\r\n\r\nWe can compute the effect size \u2013 where the effect size is defined as the magnitude of the difference between the two means when the difference is adjusted by the standard deviation for the mean of interest.<\/strong> The formula is simply the difference between the two means in a scenario that compares two groups divided by the standard deviation of the group of interest. So how do I establish the group of interest? The confusion in using this formula is often in which standard deviation to select. One way is to simply select one group to be the standard reference group and the other group to be the group of interest.\r\n\r\nThe Effect Size<\/em> formula: <\/strong>[latex]ES = {\\left(\\overline{x_{1}} - \\overline{x_{2}}\\right)\\over{s_{1}} }[\/latex]\r\n\r\nThe <\/span>effect size<\/em> formula is often interpreted using Cohen\u2019s criteria where an effect size score of 0.2 is considered as a small but noticeable effect, while an effect size score of 0.5 is considered to be a medium effect size, and an effect size score of 0.8 is considered to be a large effect size.<\/span>\r\n\r\nWe can also calculate the confidence interval for the difference between two means in any scenario where we compute the t-observed score. The formula to compute the confidence interval for a mean difference for two independent samples is shown here. The elements for this calculation are produced from the SAS output using PROC UNIVARIATE or PROC MEANS and substituted into the equation. The formula for confidence intervals in a t-test for independent samples is:\r\n\r\n[latex]\\left(\\overline{x_{1}} - \\overline{x_{2}}\\right) \\pm t_{0.05} \\times \\sqrt{\\frac{s^2_{1}}{n_{1}}+ \\frac{s^2_{2}}{n_{2}}} [\/latex].\r\n\r\nWhere the t0.05<\/sub> is the critical value for t<\/strong> for the degrees of freedom in the study. Considering that we have 50 cases in our sample our degrees of freedom value will be: df =\u00a0 (n1<\/sub>\u2013 1) + (n2<\/sub> \u2013 1) and the critical value of t0.05<\/sub> = 2.01\r\n\r\nThe decision rule concerning a confidence interval in a t-test for independent samples is to determine if the range of the confidence interval from the lowest value to the highest value includes 0. If 0 is included in the range then we accept the null hypothesis that the mean for group 1 = the mean for group 2.\r\n\r\n30.4 The degrees of freedom and critical values<\/strong>\r\n