PAC 499 Module 5 Gathering and Organizing Evidence Example

Two Cycles In: Organizing Retention Data and Interview Codes for the Bluegrass Peer Call Pilot

Student Name

Psychology and Addiction Studies Program, Aspen University

PAC 499: Senior Capstone

Instructor Name

Month Day, Year

Two Cycles In: Organizing Retention Data and Interview Codes for the Bluegrass Peer Call Pilot

Jordan's capstone pilot at Bluegrass Recovery Services, the outpatient program in Lexington, Kentucky, invented for this course along with Jordan, began on August 1. Each new client receives a call from one of three peer recovery specialists the day after intake. This report, written in mid-November, describes the evidence gathered through two of three monthly cycles, how it is organized and what has changed so far.

Evidence Collected Against the Plan

How the Data Are Managed

The intake coordinator prepares a monthly file listing each client by a study number, intake date, whether the client came to any scheduled appointment in the window from day twenty-five to day thirty-five and whether the peer call was completed. Names and other identifiers stay with the coordinator, in keeping with the special protection federal rules give addiction treatment records. Jordan keeps the files received unchanged in one folder and works only on copies. Each month, Jordan checks the file against the peers' call logs, using the study number; in September, two records showed a completed call that the log did not record, and the coordinator corrected one entry and confirmed the other. Every correction is noted in a data log with the date and reason. Interview recordings are kept on the program's secure computer, transcribed without names and then deleted.

Retention by Month

Reading the Run Chart

Jordan plotted monthly retention on a run chart, a line graph of a measure over time with the median drawn across it. Perla et al. (2011) explain that run charts help improvement teams tell real change from ordinary variation, provided they read them with rules rather than by eye. They describe four signals. For Bluegrass, a shift would mean retention landing above fifty-two percent, or under it, for six straight months. A trend would mean five months that each improve on, or each fall short of, the month before. Runs are the unbroken stretches a chart makes on one side of its middle line, and a chart with unusually few or unusually many of them points to change. An astronomical point is a month so unlike the rest that it stands out on sight. The baseline months sit tightly around a median of fifty-two percent. August and September are both above it. Two points cannot meet any of these rules, so the chart so far is encouraging but not evidence of change. If October is also above the median, the pilot will still be short of a shift, a limit Jordan will state in the final report.

The First Cycle and What Changed

McCarty et al. (2007) describe NIATx, a network in which addiction treatment agencies made small changes, tested them in rapid cycles and kept what worked, with the aim of reducing waiting times and improving continuation in treatment. Participating agencies reported shorter waits and better continuation, and the authors credited the approach of quick, small tests led by staff. Jordan's pilot follows the same logic. Before August, Jordan predicted that peers would reach nine of ten new clients. They reached eighteen of twenty-four, three in four. The call logs showed why: most missed calls went unanswered, and peers said several clients later told them they ignore calls from numbers they do not know. In the act step, the supervisor and peers agreed that intake staff would tell each new client to expect a call the next day from a named peer and hand them a card with the peer's name and number. In September, peers reached twenty of twenty-three. For October, Jordan predicted that the reach rate would stay above eighty-five percent.

Building the Interview Codebook

Jordan followed the early phases set out by Braun and Clarke (2006), reading each of the first five client transcripts twice before coding and noting first impressions in a memo. Coding then labeled every passage relevant to the research questions, staying close to what clients said. The codes were gathered into a working codebook, which will grow as the remaining interviews are coded.

Work Remaining and Risks

Three client interviews and all three peer interviews remain, as do October retention data and the analysis. The main risk is time: the semester ends in mid-December, and October data arrive in early December. To protect the schedule, Jordan will finish coding all interviews before the October data arrive and will draft the method and context sections of the final report now. A second risk is that the remaining clients interviewed may be those most positive about the call. Jordan has asked the supervisor to invite clients who were not reached as well as clients who were.

Conclusion

After two cycles, the evidence is organized and checked, the call has already changed in response to what the first cycle showed and the interview data are coded with a defined codebook. Retention in August and September sits above the baseline median, but read with Perla and colleagues' rules, two points are not yet a signal. The remaining work is planned, and the risks to finishing are known.

References

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

McCarty, D., Gustafson, D. H., Wisdom, J. P., Ford, J., Choi, D., Molfenter, T., Capoccia, V., & Cotter, F. (2007). The Network for the Improvement of Addiction Treatment (NIATx): Enhancing access and retention. Drug and Alcohol Dependence, 88(2-3), 138-145. https://doi.org/10.1016/j.drugalcdep.2006.10.009

Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety, 20(1), 46-51. https://doi.org/10.1136/bmjqs.2009.037895