MPH 560 Module 5 Comparing Groups Example

Is the Difference Real? Comparing Means and Proportions Between Two Groups

Student Name

Master of Public Health Program, Aspen University

MPH 560: Applied Biostatistics for Public Health

Instructor Name

Month Day, Year

Is the Difference Real? Comparing Means and Proportions Between Two Groups

Many public health questions compare two groups: people who received a program and those who did not, or two neighborhoods, or two time periods. The appropriate test depends on the type of outcome. This paper explains the independent-samples t test for comparing means and the chi-square test for comparing proportions, using two composite studies whose calculations are shown.

Choosing a Test

For a numerical outcome, such as blood pressure, comparing two independent groups usually calls for a t test when data are roughly normal. For a categorical outcome, such as vaccinated or not, the chi-square test compares proportions. Paired data, such as before-and-after measures on the same people, require paired tests instead (Whitley & Ball, 2002).

Example 1: Comparing Means

In a composite trial, 80 adults with high blood pressure were randomly assigned to a community health worker program or usual care. After six months, the program group of 40 had a mean systolic pressure of 131.5 mm Hg (SD 13.2, and the usual care group of 40 had a mean of 138.2 with a standard deviation of 14.1.

The t Test Calculation

The pooled standard deviation is 13.66, and the standard error of the difference is 13.66 times the square root of 1/40 plus 1/40, or 3.05. The difference of 6.7 mm Hg divided by 3.05 gives t of about 2.19 with 78 degrees of freedom. The two-sided P value comes to roughly .03, and the difference's 95% interval runs from 0.6 to 12.8 mm Hg.

Example 2: Comparing Proportions

In a composite study of text message reminders, 132 of 200 children whose parents received reminders were vaccinated by the due date, compared with 104 of 200 in the comparison group, 66% versus 52%.

The Chi-Square Calculation

The chi-square test compares observed counts with those expected if vaccination were unrelated to group. For a two-by-two table, the statistic can be computed as the total times the squared difference of cross-products, divided by the product of the row and column totals. Here it equals about 8.1 with one degree of freedom, a P value of about .004 (Bewick et al., 2004).

Results Summary

The table summarizes both analyses.

Interpreting the Results

Both differences are statistically significant at the .05 level. The blood pressure interval, 0.6 to 12.8 mm Hg, is wide: the true effect could be trivial or clinically important, so a larger trial would help. The reminder effect of 14 percentage points is meaningful for a low-cost intervention. Reporting effects with intervals conveys this better than P values alone.

Assumptions

The t test assumes independent observations, approximately normal distributions within groups or reasonably large samples, and similar variances; a version that does not assume equal variances is available. For the chi-square test, each observation must be independent and every cell should have an expected count of roughly five or more; with small counts, Fisher's exact test is used instead. Checking assumptions before testing, with histograms, box plots and a comparison of group standard deviations, takes only minutes and prevents misleading results.

Multiple Comparisons

If the trial had also compared weight, cholesterol, glucose and other outcomes, some would reach significance by chance. The Bonferroni method guards against this by dividing the significance level by the number of tests, so with five outcomes each would be tested at .01. It is simple but conservative, reducing power (Bland & Altman, 1995).

Beyond Two Groups

Comparing three or more groups calls for analysis of variance for means or a larger chi-square table for proportions. Adjusting for other variables, such as age, requires regression methods, which appear later in the course. Running many separate two-group tests instead of one analysis of variance inflates the chance of false positives, which is another form of the multiple comparison problem.

Paired Comparisons

If the health worker trial had measured the same people before and after the program, a paired t test would compare each person's change, removing variation between individuals and often increasing power. Using an independent-samples test on paired data wastes information and can mislead.

Effect Sizes

Effect sizes express differences in standard units. For the blood pressure trial, dividing the 6.7 mm Hg difference by the pooled standard deviation of 13.66 gives about 0.49, a medium effect by common conventions. Effect sizes help compare results across studies that use different measures.

Reporting Comparisons

Reports should give group sizes, summary statistics for each group, the difference with its confidence interval, the test used and the exact P value. Readers can then judge both the size and the precision of the effect, rather than seeing only whether a threshold was crossed.

Common Mistakes

Frequent mistakes include using a t test on highly skewed data from small samples, applying a chi-square test with very small expected counts, running many tests without adjustment and describing a non-significant difference as proof of no effect. Each can be avoided by checking data and assumptions before testing.

Practical Interpretation

For a program manager, the key questions are how large the effect is, how certain the estimate is and whether the benefit justifies the cost. The reminder study suggests that inexpensive texts could raise vaccination substantially, which is useful for decisions even before a larger study.

Randomization and Inference

The health worker comparison is especially credible because participants were randomly assigned. In observational comparisons, such as clinics that chose to send reminders, differences might reflect other characteristics of those clinics, and a significant test would not show that reminders caused the difference.

Conclusion

The t test and chi-square test answer whether differences between two groups are larger than chance would easily produce. In the composite examples, a community health worker program lowered mean systolic pressure by 6.7 mm Hg and reminders raised vaccination by 14 points, both statistically significant. Choosing the right test, checking assumptions, reporting intervals and guarding against multiple comparisons make such findings trustworthy.

References

Bewick, V., Cheek, L., & Ball, J. (2004). Statistics review 8: Qualitative data, tests of association. Critical Care, 8(1), 46-53. https://doi.org/10.1186/cc2428

Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: The Bonferroni method. BMJ, 310(6973), 170. https://doi.org/10.1136/bmj.310.6973.170

Whitley, E., & Ball, J. (2002). Statistics review 5: Comparison of means. Critical Care, 6(5), 424-428. https://doi.org/10.1186/cc1548