Police departments in the United States face substantial staffing issues resulting from attrition and increased competition for qualified individuals to fill vacancies. As a result, many police departments’ recruiting and hiring units are unsuccessful at meeting hiring goals. This issue can be compounded as agencies attempt to target specific groups in their hiring efforts. Either way, the inability to adequately hire enough individuals to meet demand reflects limitations in interest among job seekers as well as obstacles within departments’ hiring processes. Given the challenges, what are law enforcement agencies to do?
To address the problem, many departments seek advice from other agencies. Sometimes advice is solicited from nearby agencies; other times, it is requested from agencies that seem more successful at addressing hiring issues. But direct advice from another agency is often simply anecdotal and can be unreliable—for instance, what worked in New York might not work in Chicago, what worked in a sheriff’s department might not work in a police department, and what worked in the 1990s might not work in 2020. To overcome this problem, law enforcement agencies can turn to a variety of available materials that make recommendations based on scientific evidence. Commonly discussed strategies to address hiring shortfalls include
■ collaborating with other agencies;
■ increasing community engagement;
■ streamlining recruiting and hiring;
■ putting someone in charge of recruiting;
■ enhancing the use of the web and social media;
■ mentoring applicants; and
■ enhancing the department’s image.1
Sometimes, recommendations like these are referred to as best practices. Unfortunately, there is no agreement about what is required for a practice to be referred to as a “best practice,” and, often, best practice simply means a practice accepted by the field. In other cases, recommendations may be referred to as evidence-based practices or EBPs, suggesting that there is some evidentiary basis that indicates that the recommendations are likely to be effective. In recent decades, law enforcement agencies have benefited from increased access to EBPs. Key policing tasks, like crime control, have been studied scientifically in some detail, and practices to address selected issues are now guided by a reasonable amount of quality research.
Assessing Evidence Quality
In other areas, such as recruiting and hiring, however, less evidence is available. While criminal justice researchers have called for more use of EBPs to guide other aspects of police organizational management, less scientific information is available to address many pressing organizational concerns. Compounding the issue of limited evidence is the fact that some available evidence, although technically scientific in nature, lacks sufficient quality or applicability to be useful. This deficiency relates to an additional problem that, in many fields including criminal justice, there may be limited agreement as to what evidentiary basis should be required to either support a recommendation or to refer to a practice as an EBP. In short, as with any evidence used in investigating crimes, scientific evidence varies markedly in quality. As a result, so does the value of the recommendations that follow, whether they are labeled best practices, EBPs, or otherwise.
Systems for classifying the quality of evidence based on the level of rigor in the underlying science have been developed in some fields. For example, in medicine, evidence can be classified into as many as 10 different levels ranging from weak evidence generated with little scientific rigor (e.g., expert opinion without explicit critical appraisal) to very strong evidence generated through very rigorous scientific methods (e.g., systematic reviews of multiple randomized control trials).2 This type of taxonomy allows for the development of guidelines to help practitioners understand the choices available when they are seeking a solution to a particular medical issue. For example, one example of guidelines from the field of medicine differentiates among three types of evidence:
1. Evidence supporting strong recommendations that should be followed unless there is a clear and compelling rationale for a different approach
2. Evidence supporting recommendations that should be generally followed while keeping abreast of new evidence that may change the recommendation
3. Evidence supporting optional recommendations that should be treated only as one of many possibilities
Absent a classification system based on the value of the underlying evidence that supports recommendations, decision makers may struggle to evaluate and choose between alternatives. This can be challenging in cases where evidence is coming from multiple sources, and the issue is particularly problematic when the available evidence includes different, conflicting recommendations. Faced with such confusion, decision makers may choose between recommendations based on intuition, or, more problematically, they may discount the evidence altogether, making decisions with little regard to recommendations.
Available Recruiting and Hiring Evidence
The absence of a system to evaluate the validity of the available (albeit limited) evidence on law enforcement recruiting and hiring practices creates a challenge for the field. For example, a continued issue in law enforcement in the United States is the challenge that agencies face when attempting to diversify. In the context of gender, agencies have made little progress in almost 30 years. Today, female representation in U.S. policing hovers at about 13 percent, roughly the same level seen in the early 1990s. To address this problem, many sources stress the need to market law enforcement jobs by emphasizing the service aspect of working in law enforcement.3 However, recent research investigating this claim casts doubt on the value of this practice for specifically recruiting women.
Researchers theorized that people who are interested in service may already be more likely than others to consider law enforcement careers. If this is true, agencies may need to target individuals who are not pre-disposed to careers in law enforcement to expand the size of their applicant pools. To test these assumptions, researchers conducted a field experiment and showed that service messages do not effectively motivate individuals to apply for policing jobs.4 Perhaps more importantly, the research demonstrated that messages emphasizing the challenging nature of working in policing and messages emphasizing that police work can be a long-term career more than tripled the likelihood that individuals would apply. The impact was greater for women and minorities, who were more than four times as likely to apply after having received either the challenge or career message. In addition, there was no indication that the quality of applicants changed as a result of the different marketing efforts.
Importantly, this work was conducted using a field experiment (or randomized control trial), which represents the gold standard in social science research. Following the guidelines presented from the medical field above, the evidence from this work was generated with sufficient scientific rigor to support a strong recommendation that, according to the previously listed standards, should be followed unless there is a clear and compelling rationale for using a different approach. Given these findings, this practice is likely to increase applications overall as well as substantially increase applications from diverse candidates.
To date, few other issues in recruiting and hiring in law enforcement agencies have been studied with similar levels of scientific rigor. However, the limited available research conducted this way uncovers strategies that are more likely to achieve important goals than strategies that are untested or strategies that are tested in more limited ways. As an example, another experimental study showed strong evidence that simplifying hiring processes and establishing norms suggesting that the process is quick and easy reduces voluntary drop off of police applicants.5 Specifically, the study showed that simplifying the process applicants use to submit their personal history sheet (the third step in the hiring process studied) and suggesting that most applicants complete the step within two weeks resulted in an 8 percent increase in the submitted materials and a 56 percent increase in the likelihood that applicants submitted the materials within two weeks. Based on the experimental design of the study, this evidence supports a recommendation for simplifying hiring processes that should be followed absent clear and compelling reasons to do something differently.
A larger body of research on recruiting and hiring has been completed using less rigorous, yet still informative, scientific methods. One such example, based on correlational analysis of historical data, showed that using expedited testing procedures, where applicants complete more than one test per day, results in less overall attrition.6 The research also indicated that applicants who were allowed to take multiple assessments on the same day were more likely to be hired and appointed to the training academy. The same study also showed that simplifying hiring processes by reducing the number of steps required and using standardized application materials that applicants may have encountered when applying to other agencies also reduced attrition among applicants. Given that this work was completed using less rigorous methods, the recommendations that follow do not carry the same weight as those from the experimental studies above. However, this level of evidence situates these recommendations as something agencies should generally follow unless new evidence becomes available that changes the recommendations.
Research conducted on specific aspects of recruiting and hiring in law enforcement may be unavailable. . . however, research conducted in other fields can address this limitation.
Unfortunately, some research fails to reach even this lower threshold. Overall, research into recruiting and hiring practices in law enforcement agencies has rendered a variety of findings, yet many of the studies available have been conducted using scientific methods that are less scientifically rigorous than the discussed correlational study. Often, work based on these more limited methods supports recommendations that would be viewed only as possibilities to be considered if no better recommendations exist. For example, a recent review of available research on academy training demonstrates this issue, finding that only 12 percent of available police academy training research was conducted with enough scientific rigor to be considered strong evidence.7
In other cases, research conducted on specific aspects of recruiting and hiring in law enforcement may be unavailable to justify recommendations to agencies. However, turning to research conducted in other fields can address this limitation. Despite limited work on several recruiting and hiring issues in the law enforcement context, there is a substantial body of general research about recruiting and hiring that has been conducted in both the public and private sectors. While the recommendations from these works do not include strategies specifically shown to impact hiring in law enforcement agencies, they are instructive in that they have been shown to impact similar issues in hiring processes in other organizations.
For example, organizational scientists using laboratory experiments on hiring decisions have studied the impact of descriptive norming on hiring decisions. Descriptive norms are typical patterns of behavior, and the presence of descriptive norms has been shown to support conformity among individuals. The theory suggests that individuals may be nudged into behaviors by being presented with information that others are also engaging in the behavior. In the context of underrepresentation, there may be a reactive effect (technically known as reactance) where information about the extent of underrepresentation may actually encourage behaviors that further limit representation. In other words, focusing on the underrepresentation of a particular group could encourage behaviors that serve to keep the group from growing. The research conducted showed men in hiring positions attempt to hire more men when faced with the knowledge that others have attempted to hire more women, but men do not try to hire more women when they are aware that others have tried to hire more men.8 The work also shows that women in hiring positions do not exhibit this type of reactance.
While this study was conducted as a lab experiment and the decisions made by experiment subjects were not contextualized around law enforcement, the findings are still largely instructive. The reactance behavior exhibited by men is understood to apply to a variety of selection contexts. Although information about what others do would typically nudge men to behave similarly, the finding of reactance suggests that men may be less suitable for working in hiring or recruiting positions in organizations with gender imbalances. This may be particularly important if these organizations are attempting to diversify. The findings suggest that women may be better suited to work in recruiting and hiring roles in these circumstances.
Other Considerations
IACP Resources■ Best Practices Guide: Recruitment, Retention, and Turnover ■ Center for Police Research and Policy ■ “Police Personnel Retention Challenges” (article) |
While issues about the quality of evidence are important to understand when thinking about EBPs, the other concern noted earlier, applicability, also requires substantial consideration. The strongest evidence in other fields supports practices that should be adhered to unless there is a compelling reason to do otherwise. Law enforcement agencies are not unique among organizations in that multiple factors often influence adopted practice and effectiveness is only one concern. In all organizations, certain factors can serve as hurdles to implementing even the most effective practices. Issues such as legal context, resources, community concerns, and goals make some practices more viable for some organizations than others. In the end, these concerns reflect the potential impact of decisions and are often guided by value considerations that cannot be resolved scientifically. In this regard, law enforcement administrators carry the ultimate burden of making the decisions that are best for their organizations. Still, the available evidence that demonstrates the effectiveness of any practices under consideration should always be among the factors that law enforcement administrators consider in their deliberations, and an understanding of the relative strength of evidence supporting these recommendations is paramount to understanding their value. 🛡
Notes:
1 See, e.g., IACP, Law Enforcement Recruitment Toolkit (Washington, DC: Office of Community Oriented Policing Services, 2009), 7–14; Jeremey M. Wilson et al., Police Recruitment and Retention for the New Millennium: The State of Knowledge (Santa Monica, CA: Rand Corporation, 2010), 60–86; Law Enforcement Best Practices: Lessons Learned from the Field (Washington, DC: Office of Community Oriented Policing Services, 2019), 93–110.
2 Oxford Centre for Evidence-Based Medicine, The Oxford 2011 Levels of Evidence, V. 2.1. While attempts at developing evidence reliability scales such as those used in medicine have been undertaken by criminal justice researchers, no such system has been widely adopted.
3 See, e.g., Ellen Scrivner, Innovations in Police Recruitment and Hiring: Hiring in the Spirit of Service (Washington, DC: Office of Community Oriented Policing Services, 2001).
4 Elizabeth Linos, “More Than Public Service: A Field Experiment on Job Advertisements and Diversity in the Police,” Journal of Public Administration Research and Theory 28, no. 1 (January 2018): 67–85.
5 Elizabeth Linos and Nefara Riesch, “Thick Red Tape and the Thin Blue Line: A Field Study on Reducing Administrative Burden in Police Recruitment,” Public Administration Review 80, no. 1 (January/February 2020): 92–103.
6 Linos and Riesch, “Thick Red Tape.”
7 Brendan McGinley et al., “Police Recruit Training Programmes: A Systematic Map of Research Literature,” Policing: A Journal of Policy and Practice (May 2019).
8 Maliheh Paryavi, Iris Bohnet, and Alexandra van Geen, “Descriptive Norms and Gender Diversity: Reactance from Men,” Journal of Behavioral Public Administration 2, no. 1 (2019).
Please cite as
Anne Li Kringen, “Assessing the Value of Evidence: Understanding Research on Recruiting and Hiring,” Police Chief 87, no.4 (April 2020): 34–38.
