EDP 812 Module 2 Research Methods in I/O Psychology Example

Did the Onboarding Work? A Quasi-Experimental Design for a Contact Center

Student Name

Doctor of Education Program, Aspen University

EDP 812: Foundations of Industrial and Organizational Psychology

Instructor Name

Month Day, Year

Did the Onboarding Work? A Quasi-Experimental Design for a Contact Center

After examining the history of selection research, Gabriel, the organizational development specialist at a composite credit union in Boise, turned to a narrower question. The credit union had developed a four-week onboarding program for new member service representatives, with a peer mentor, graduated call difficulty and weekly check-ins with a supervisor. Leadership wanted to know whether it would reduce turnover in the first six months. Both Gabriel and the credit union are fictional. This paper designs a study to answer the question.

The Research Question

The research question is whether six-month turnover is lower for representatives who complete the new onboarding program than for similar representatives given the previous one-week orientation. The hypothesis is that six-month turnover will be lower among those receiving the new program. Secondary questions concern whether the program improves new hires' confidence and their early performance.

Choosing a Design

Scandura and Williams (2000) reviewed research methods used in management studies and described a trade-off among three goals: generalizability to other settings and people, precision in measuring and controlling variables and realism of the setting. No single design maximizes all three. Laboratory experiments are precise but unrealistic; field studies are realistic but less precise; surveys can be generalizable but weak on causal inference. They found that management research relied heavily on surveys and called for more varied designs.

Grant and Wall (2009) argued that quasi-experiments, which compare groups or periods without random assignment, are underused in organizational research. Organizations rarely allow random assignment, but well-designed quasi-experiments, using features such as comparison groups, multiple pretests and staggered introduction, can rule out many alternative explanations and approach the strength of true experiments while keeping the realism of field settings.

The Chosen Design

The credit union has two contact center sites. The new program will begin at the downtown site in January and at the Meridian site in July. From January to June, new hires at Meridian, still receiving the old orientation, serve as a comparison group. From July, the Meridian start provides a replication: if turnover drops there too after the program begins, the case for the program strengthens. Turnover for hires in the previous two years at both sites provides a baseline.

Measures and Sources

Podsakoff et al. (2003) reviewed research on common method bias, the inflation or distortion of relationships between variables that occurs when they share a method or source, for example when one survey asks employees about both their onboarding experience and their intention to quit. The design avoids this by drawing the main outcome, turnover within six months, from human resources records. Early performance comes from quality assurance scores assigned by evaluators who do not know which program a representative received. New hires' confidence is measured by survey at weeks two and eight, analyzed as a secondary outcome with its limits acknowledged.

Threats to Validity

Several threats remain. History: an event such as a pay increase at one site could change turnover. The credit union agrees not to change pay or schedules differently across sites during the study, and the team will record any changes that occur. Selection: hires at the two sites might differ; the analysis will compare their characteristics at baseline. Diffusion: supervisors at Meridian might adopt parts of the new program early; the team will check by asking supervisors monthly. Regression to the mean: turnover may have been unusually high before the study; the two-year baseline helps assess this.

Ethics

Participation in the program is part of employment, but the survey is voluntary, with informed consent. Responses will be confidential, stored separately from personnel records and reported only in aggregate. Supervisors will not see individual responses. An institutional review board will examine the protocol, since results may be published as part of Gabriel's doctoral work. Participants may decline the survey without any effect on their employment, and the consent form will say so plainly.

What the Design Cannot Show

Without random assignment, the design cannot rule out every alternative explanation. It also cannot say which part of the program, the mentor, the graduated calls or the check-ins, produces any effect. A later study could vary components.

Analysis Plan

The main analysis will compare six-month turnover among hires who received the new program with turnover among hires who received the old orientation, controlling for site, hire month and prior contact center experience. Support for the hypothesis would require lower turnover at the downtown site after January relative to Meridian during the same months, and a similar drop at Meridian after its July start. A drop at both sites at the same time, regardless of the program, would point instead to an outside cause. Grant and Wall (2009) emphasize that this kind of prediction, a specific pattern of results that alternative explanations would not produce, is what gives quasi-experiments their strength.

Practical Constraints

The design reflects compromises. Leadership wanted the program at both sites at once; Gabriel negotiated a six-month delay at Meridian by pointing out that the program's materials were still being refined and that a staggered start would show whether the investment paid off. Sample size is limited: about forty hires per site per year means the study may detect only a fairly large effect. The team will report the size and uncertainty of any difference rather than relying only on statistical significance.

Conclusion

Scandura and Williams's trade-offs explain why no design is perfect, Grant and Wall show how a quasi-experiment can support strong inference in a real organization and Podsakoff and colleagues show why outcomes should come from sources other than the survey. A staggered design across two sites, with turnover drawn from records, gives the credit union a credible answer to its question.

References

Grant, A. M., & Wall, T. D. (2009). The neglected science and art of quasi-experimentation: Why-to, when-to, and how-to advice for organizational researchers. Organizational Research Methods, 12(4), 653-686. https://doi.org/10.1177/1094428108320737

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879-903. https://doi.org/10.1037/0021-9010.88.5.879

Scandura, T. A., & Williams, E. A. (2000). Research methodology in management: Current practices, trends, and implications for future research. Academy of Management Journal, 43(6), 1248-1264. https://doi.org/10.5465/1556348