DPH 870 Module 5 Critical Appraisal of Key Studies Example

Weighing the Evidence: Critical Appraisal of Key Studies on Community Health Workers and Hypertension

Student Name

Doctor of Public Health Program, Aspen University

DPH 870: Evidence-Based Practice and Advanced Research Methods

Instructor Name

Month Day, Year

Weighing the Evidence: Critical Appraisal of Key Studies on Community Health Workers and Hypertension

Not all studies deserve equal weight. Critical appraisal examines how well a study was designed and conducted, so that a literature review can rely more on stronger evidence and interpret weaker evidence cautiously. This paper appraises three key studies for a review of evidence on community health workers in hypertension care, using established tools, and considers how appraisal shapes the review's conclusions.

Reading for Bearings

Greenhalgh (1997) advises readers first to establish what a paper is about: why the study was done, what question it asked, what type of study it was and whether that design suited the question. Answering these questions before examining details prevents readers from being impressed by statistics in a study that could not answer its question.

Appraisal Summary

The table summarizes the appraisal of each study.

Appraising the Randomized Trial

The Cochrane risk of bias tool examines random sequence generation, allocation concealment, blinding, incomplete outcome data and selective reporting (Higgins et al., 2011). In the trial of community health worker support across primary care practices, randomization and allocation appear sound and outcomes were prespecified. Participants and workers could not be blinded, which may affect self-reported outcomes more than measured ones such as hospital days (Kangovi et al., 2018).

Appraising the Systematic Reviews

AMSTAR 2 assesses systematic reviews on 16 items, including protocol registration, search comprehensiveness, justification of exclusions, risk of bias assessment and appropriate synthesis, with some items treated as critical (Shea et al., 2017). The earlier review of community health workers for hypertension stated a clear question and searched several databases but predates current reporting standards and included studies of varied quality (Brownstein et al., 2007). The review from lower-income countries searched broadly but combined very different programs (Jeet et al., 2017).

Appraising the Capstone's Own Design

The capstone uses a matched observational design, so its own risk of bias should be anticipated. ROBINS-I assesses nonrandomized studies of interventions across domains including confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes and selection of reported results (Sterne et al., 2016). Confounding by motivation is the capstone's greatest risk.

From Studies to Certainty

Appraising single studies is not enough; the review must judge the certainty of the body of evidence. Under GRADE, certainty is graded from high to very low; trials begin high and observational studies low, and ratings fall for bias, inconsistent results, indirect evidence, imprecision or suspected publication bias (Guyatt et al., 2008). For blood pressure control, evidence appears to be of moderate certainty.

Relevance to the Capstone

Appraisal also considers applicability. The randomized trial took place in an academic health system with strong infrastructure; the capstone's program is run by a health department. Differences in setting, staffing and population may limit how directly the trial's results apply, a point the review will make explicitly. The two reviews drew on programs in many countries, which broadens their relevance but also mixes very different health systems.

Consistency of Findings

Despite different designs and settings, the three studies point in the same direction: community health workers can improve outcomes for people with chronic conditions, including blood pressure. Consistency across designs strengthens confidence even when individual studies have limitations.

Using Appraisal in the Review

Appraisal results will shape the review in three ways: stronger studies will receive more weight in the synthesis, limitations will be discussed where they affect conclusions and gaps in high-quality evidence, such as for health department-run programs, will be highlighted as the capstone's contribution.

Common Appraisal Errors

Common errors include treating all published studies as equally trustworthy, confusing reporting quality with study quality, applying a trial checklist to an observational study and summarizing results without judging them. Choosing the right tool for each design avoids the last two. Another is judging a study by its journal rather than its methods.

Appraising Qualitative Studies

The review also includes qualitative studies describing participants' and workers' experiences. These are appraised with a checklist asking whether aims, methods, sampling, data collection, analysis and reflexivity are appropriate and clearly reported. Qualitative studies help explain mechanisms that trials measure only indirectly.

Consistency Between Appraisers

A classmate independently appraised the three key studies using the same tools. The two appraisers agreed on overall judgments for all three but differed on two AMSTAR 2 items, which were resolved by discussion and by rereading the review methods. Reporting this process strengthens confidence in the judgments.

Publication Bias

Studies with positive results are more likely to be published. Small, positive studies dominate parts of the community health worker literature, raising the possibility that effects are overstated. The review will note this risk and, where enough studies exist, examine whether smaller studies report larger effects.

Appraising Outcome Measurement

How blood pressure was measured matters. Studies using standardized research measurements provide stronger evidence than those relying on routine clinic readings, which vary in technique. The appraisal notes the measurement method for each study, since the capstone itself relies on routine readings.

Recording Appraisal

Appraisal judgments are recorded in a table for each study, with the tool used, answers to each item and reasons for the overall judgment. This record appears in an appendix, allowing the committee to see how conclusions about study quality were reached.

Appraisal and Local Relevance

A study can be methodologically strong yet less relevant to a local question. The appraisal therefore records both internal validity and applicability, so that the review can weigh strong but distant evidence against weaker but more local evidence.

Appraisal as a Skill

Appraisal improves with practice. Working through the tools for three studies built familiarity that made appraising the remaining 28 studies faster and more consistent. Keeping notes on recurring issues, such as unclear allocation or missing data handling, helped standardize judgments across the review.

Conclusion

Critical appraisal lets the capstone's review weigh evidence rather than merely list it. Using the Cochrane tool for the trial, AMSTAR 2 for the reviews and ROBINS-I for the capstone's own design, and summarizing certainty with GRADE, the review can state what is known with appropriate confidence and where stronger local evidence is needed.

References

Brownstein, J. N., Chowdhury, F. M., Norris, S. L., Horsley, T., Jack, L., Zhang, X., & Satterfield, D. (2007). Effectiveness of community health workers in the care of people with hypertension. American Journal of Preventive Medicine, 32(5), 435-447. https://doi.org/10.1016/j.amepre.2007.01.011

Greenhalgh, T. (1997). How to read a paper: Getting your bearings (deciding what the paper is about). BMJ, 315(7102), 243-246. https://doi.org/10.1136/bmj.315.7102.243

Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(7650), 924-926. https://doi.org/10.1136/bmj.39489.470347.AD

Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., Cochrane Bias Methods Group, & Cochrane Statistical Methods Group (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343, Article d5928. https://doi.org/10.1136/bmj.d5928

Jeet, G., Thakur, J. S., Prinja, S., & Singh, M. (2017). Community health workers for non-communicable diseases prevention and control in developing countries: Evidence and implications. PLOS ONE, 12(7), Article e0180640. https://doi.org/10.1371/journal.pone.0180640

Kangovi, S., Mitra, N., Norton, L., Harte, R., Zhao, X., Carter, T., Grande, D., & Long, J. A. (2018). Effect of community health worker support on clinical outcomes of low-income patients across primary care facilities: A randomized clinical trial. JAMA Internal Medicine, 178(12), 1635-1643. https://doi.org/10.1001/jamainternmed.2018.4630

Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, Article j4008. https://doi.org/10.1136/bmj.j4008

Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., Carpenter, J. R., Chan, A.-W., Churchill, R., Deeks, J. J., Hróbjartsson, A., Kirkham, J., Jüni, P., Loke, Y. K., Pigott, T. D., ... Higgins, J. P. (2016). ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355, Article i4919. https://doi.org/10.1136/bmj.i4919