PAC 302 Module 2 Scores, Norms and Statistics Example

From Twenty-Eight to Eighteen: Norms, Standard Scores and Whether a Client's Change Is Real

Student Name

Psychology and Addiction Studies Program, Aspen University

PAC 302: Assessment Procedures in Addiction Studies

Instructor Name

Month Day, Year

From Twenty-Eight to Eighteen: Norms, Standard Scores and Whether a Client's Change Is Real

Dana is thirty-six and has been in outpatient treatment at Lakeview Counseling and Recovery, the fictional Grand Rapids agency in these papers, for eight weeks for alcohol use disorder and low mood. At intake she completed a twenty-one-item depression questionnaire and scored twenty-eight. At her eight-week review she scored eighteen. Her counselor wrote in her chart that she was "much improved." Her psychiatrist asked whether the change was real or within the error of the test. This paper uses Dana's scores to explain raw scores, norms and the statistics needed to answer that question. Dana, her scores and the questionnaire's manual values are invented for teaching; the research is real.

Raw Scores and Norms

A raw score is the number a person earns on a test, such as the sum of item ratings. By itself it has little meaning. Dana's twenty-eight means something only in comparison with others. Norms provide that comparison: the scores of a reference group, the norm group, who took the test under standardized conditions. The test manual reports that the general adult norm group had a mean of twelve and a standard deviation of eight. A clinical cutoff of twenty separates scores typical of people seeking treatment for depression from scores typical of the general population, and the manual reports a reliability of 0.90.

Norms must fit the person. A norm group should be similar to the person being tested in characteristics that affect scores, such as age, language and setting, and recent enough to reflect current performance.

Standard Scores and Percentiles

Standard scores place a raw score on a common scale. A z-score expresses distance from the mean in standard deviation units: the raw score minus the mean, divided by the standard deviation. A T-score converts the z-score to a scale with a mean of fifty and a standard deviation of ten. Percentiles indicate the percentage of the norm group scoring at or below a given score, and in a normal distribution they can be read from the z-score.

When the Normal Curve Fails

Percentile conversions assume that scores are normally distributed. Micceri (1989) examined hundreds of large-sample distributions of achievement and psychological test scores and found that very few were normal. Most were skewed, lumpy or had heavier tails than the normal curve. Depression questionnaires in general populations are typically skewed, with most people scoring low and a long tail of high scores. For Dana's questionnaire, percentiles based on the normal curve are approximations; the manual's own percentile table, if available, is more accurate.

When Norms Grow Old

Norms also age. Flynn (1987) compiled data from fourteen nations showing that average scores on intelligence tests had risen substantially over a generation, in some countries dramatically. The gains meant that a person tested against norms collected decades earlier would score higher than against current norms, appearing more able relative to peers than they were. The finding, now known as the Flynn effect, applies most directly to intelligence tests, but its lesson extends to any instrument: the date of the norms matters, and older norms can distort interpretation. Dana's questionnaire was normed eleven years ago, which is acceptable but worth noting.

Dana's Scores Converted

To get the standard error of measurement, take one minus the reliability, find its square root and multiply by the standard deviation: 0.10 has a square root of about 0.316, and eight times that is about 2.53. Multiplying by 1.96 gives a 95% confidence interval of about five points on either side of each score.

Is the Change Reliable?

Jacobson and Truax (1991) proposed two criteria for judging change in individual clients. The first is the reliable change index: the change from pretest to posttest divided by the standard error of the difference, a figure built from the standard error of measurement by squaring it, doubling it and taking the square root. If the index exceeds 1.96, the change is unlikely to reflect measurement error alone. For Dana, 2.53 squared is 6.40, doubled is 12.80, and its square root is about 3.58. Her change of ten points divided by 3.58 gives an index of about 2.79. Her improvement exceeds what measurement error would produce.

Is the Change Meaningful?

Jacobson and Truax's second criterion is clinical significance: whether the client moved from the range typical of a dysfunctional population into the range typical of a functional one. One way to judge this is with a cutoff between the two distributions. Dana's intake score of twenty-eight was above the questionnaire's clinical cutoff of twenty, and her eight-week score of eighteen is below it. By both criteria, her change is reliable and clinically significant. Her counselor's note that she was much improved is supported, with the caution that her final score sits only two points below the cutoff and its confidence interval crosses it.

Choosing the Right Comparison

The calculations depend on which norms are used. Compared with the general adult norm group, Dana's intake score was two standard deviations above the mean. Compared with a clinical norm group of adults in outpatient treatment, if the manual provided one, the same score might sit near the middle of the distribution. Both comparisons are legitimate, but they answer different questions. The general norms ask how unusual her symptoms are in the population; clinical norms ask how she compares with others seeking help. A report should say which norm group was used and why.

What the Numbers Cannot Say

The statistics establish that Dana's reported symptoms changed more than measurement error would explain and that she now scores in the range typical of people not seeking treatment. They do not explain why: reduced drinking, therapy, a new job or the passage of time could each contribute. They also rely on her self-report. Her counselor's interview and her own account of how she feels remain essential.

Conclusion

Dana's scores illustrate the statistics behind assessment. Norms give raw scores meaning, standard scores and confidence intervals express them precisely, Micceri warns that percentiles assume a normal curve real data often lack and Flynn shows that norms grow outdated. Jacobson and Truax's criteria show that Dana's ten-point drop is reliable and clinically significant, answering her psychiatrist's question with numbers a reader can check.

References

Flynn, J. R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological Bulletin, 101(2), 171-191. https://doi.org/10.1037/0033-2909.101.2.171

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12-19. https://doi.org/10.1037/0022-006X.59.1.12

Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(1), 156-166. https://doi.org/10.1037/0033-2909.105.1.156