Staffing issues have plagued police executives across the globe in their search to fill vacancies with qualified applicants capable of dealing with the complex realities of modern policing.1 These challenges reveal just how critical a component the hiring process is for the longevity and sustainability of a police force. Within the hiring process, recent efforts have highlighted the recruitment challenges, but less attention has been given to candidate selection procedures.2 Agency administrators know that the single greatest asset (both material and intangible) is their personnel. With new officer training costs climbing to over $200,000 in some agencies, the accurate selection of police candidates pays not only monetary dividends but also increases the overall performance and professionalism of an agency. With this valuable information in mind, what price would hiring officials place on predictive data for the successful selection of future officers? The ability to leverage data into the personnel decision-making process would be priceless both in terms of human and operational costs.
A review of various police hiring models reveals multiple conventional methods of personnel selection, including written tests, pre-employment interviews, and assessment centers. Very little data exists as to the validity of these methods when employed within an organization.3 The methods utilized by a specific agency vary greatly and are influenced by the familiarity and comprehension of what each technique can achieve. Recent calls are being made to evaluate candidates “holistically” and to use a “highly structured assessment process.”4 Each selection tool taps into specific data and characteristics of candidates by objectively employing different selection criteria. Among these methods, the most costly, both financially and administratively, is the assessment center. An assessment center is defined as “a standardized evaluation of behavior based on multiple inputs” and utilizes measures to assess the whole person in a highly structured manner.5
Assessment Centers
Assessment centers have their roots dating back to WWI and WWII, when the British military utilized them for the selection of military officers. Assessment centers were widely used throughout military and civilian work environments before starting to emerge in policing in the 1970s.6 Assessment centers have shown to be useful for prediction, identification, and as an intervention technique for personnel development.7 Being able to have predictive personnel selection capabilities in a policing context is especially critical for a variety of reasons. Two of the most compelling would include (1) the current societal expectations of police officers to possess a diverse skillset while handling a wide range of tasks and (2) the hefty amounts of money and time required for training a police officer.
When properly formatted and executed, an assessment center will produce a standardized evaluation of a participant’s behavior formulated from multiple inputs.8 This personnel assessment offers critical data in the prediction and selection of future police personnel. Understanding the behaviors of potential officers, their unique personality traits, and the impact of these items on future officer success is paramount for the sustainability of the policing profession. Unveiling the complexity of the process, along with complimentary assessment tools, is of utmost importance for administrators and personnel decision-makers in hiring, training, and equipping the most successful candidates.
Background
In 2015, the Wauwatosa, Wisconsin, Police Department identified the need to more accurately select successful police officer candidates. The department estimates its total cost to recruit, select, and train an officer at $120,000. Traditionally, the Wauwatosa Police Department utilized a combination of a cognitive test (written examination) and a department interview for selection of candidates for hire. This method resulted in 72 percent of candidates passing the police and fire commission interview, background investigation, academy, and field training and completing their probationary period. This passing percentage was the measure of “success” that the police department utilized in evaluating their hiring and selection process. This measure is generalizable to most police agencies since the most critical goal in the recruitment, selection, and training process is to produce an officer capable of performing the duties required by their agency. The overall sentiment of the administration was that the “traditional method” was not accurately identifying the candidates who would be a successful officer because these methods did not analyze the most critical job-relevant behavioral components.
Beginning in 2016, the Wauwatosa Police Department initiated an assessment center as the primary selection tool for entry-level police officers. Individuals are pre-screened from a pool of applicants and invited to the fitness assessment. Those who pass the fitness assessment are stratified, and invitations to the assessment center are offered based on this ranking. The assessment center uses several behavioral exercises, such as leaderless group discussions, oral presentations, situational videos, deductive reasoning exercises, and interactions, that are designed to elicit identifiable behaviors from candidates. Assessors score each exercise on multiple dimensions such as decision making, communication, initiative, adaptability, judgment, interpersonal sensitivity, and maturity. Not all dimensions are scored for each exercise. The dimensional rating is achieved by the integration of all assessors coming to a consensus for each dimension. The dimensional scores are combined for an overall assessment center rating (OAR), and candidates who achieve or exceed a predetermined OAR are allowed to continue in the hiring process.
Data
Data were collected from all the applicants who were evaluated in each of the 11 assessment centers since its implementation in 2016. This included all pre-employment application data and the assessor ratings of candidates within the assessment center. These data were coupled with information relating to candidates who were eliminated from the hiring process at any point up to completion of their probationary period. This includes candidates who passed the assessment center (passing OAR) but were removed from the process either during the background investigation, police and fire commission interviews, recruit academy (if applicable), or field training program or for cause during the remainder of their probationary period.
Within the assessment center, the individual exercises are not given a numerical value; instead, the exercises are designed to elicit behaviors that are evaluated based on different dimensions using a numerical scale (1–5). For purposes of this data analysis, all of the dimension scores within each assessment center exercise were aggregated, and the mean (average) was reported.
“At the end of the day, what matters most is judgment.”
In addition to the data on assessment center ratings, demographic data about applicants were collected and used for analysis. These variables included applicants’ age (in years); gender, ethnicity (white/non-Hispanic, Hispanic/non-white, and non-white/non-Hispanic); education level (90 college credits or less, associate’s degree, and bachelor’s degree or higher); prior law enforcement experience (no previous experience, some experience but less than two years, and two years or more); and military veteran’s status (veteran or non-veteran). This information was collected to identify the successful attributes of a candidate, aid in recruitment strategies, and minimize selection bias.
The goal of the analysis is to understand the impact of the various assessment center dimensions and demographics on overall success. Overall success was defined as the completion of the probationary period.
Results
Two different binary logistical regression (LOGIT) models were used to evaluate the data. The first LOGIT model analyzed the assessment center dimensions and demographic data’s impact on overall success. Judgment showed as the only significant dimension within the assessment center. Along with judgment, age was also related to overall success. The impact of judgment on overall success is positive, while the impact of age is negative, such that the older a candidate is, the worse he or she performs. The overall model is highly significant and explains over 46 percent of the variance in success.
For the second LOGIT, all of the assessment center exercises were analyzed in conjunction with the same demographic data as the first model. The situational video and oral presentation exercises were the two significant predictors of overall success. Age remained significant as well, still having a negative impact. The situational video and oral presentation exercises have a positive effect on candidate success. The overall model is highly significant and explains approximately 33 percent of the variance.
Table 1: LOGIT Output |
||
|---|---|---|
| Variables | LOGIT 1 | LOGIT 2 |
| Constant | -20.362 (6.616)** |
-9.789 (4.093)* |
| Judgment | 6.729 (2.032)*** |
|
| Situational Video | 2.109 (.809)** |
|
| Oral Presentation | 1.681 (.676)* |
|
| Age | -.146 (.065)* |
-.146 (.062)* |
| Nagelkerke R2 | .463 | .326 |
| Chi-Square | 23.755*** | 15.620*** |
| aDependent Variable Overall Hiring Success b Standard Errors are given in parenthesis *Significant at the 0.05 level **Significant at the 0.01 level ***Significant at the 0.001 level |
||
Three separate Univariate Analysis of Variance (ANOVA) models were run to investigate demographic differences in judgment, situational video, and oral presentation scores. The first ANOVA evaluated the judgment dimension. Significant differences in judgment scores were found for education level, prior law enforcement history, military veteran’s status, gender, and ethnicity. Specifically, those candidates with more education (especially a bachelor’s degree), those with least two years of prior law enforcement experience, those with military experience, females, and those of Hispanic origin all scored higher on judgment.
Table 2.1: ANOVA 1 |
|
|---|---|
| Source | Type III Sum of Squares |
| Corrected Model | 4.911b*** |
| Intercept | 29.409*** |
| Gender | .922* |
| Education | 1.019* |
| L.E. History | 1.305* |
| Ethnicity | 1.152* |
| Veteran Status | .496± |
| Age | .063 |
| a Dependent Variable Judgment b R Squared = .200 ± Significant at the 0.10 level *Significant at the 0.05 level **Significant at the 0.01 level ***Significant at the 0.001 level |
|
The second ANOVA model evaluated the situational video exercise. Significant differences in situational video exercise scores were found for prior law enforcement history and gender. Specifically, those with at least two years of previous law enforcement experience outperformed those with less than two years and those with no prior experience. In terms of gender, females outperformed males.
Table 2.2: ANOVA 2 |
|
|---|---|
| Source | Type III Sum of Squares |
| Corrected Model | 6.981b** |
| Intercept | 30.654*** |
| Gender | 1.471* |
| Education | .852 |
| L.E. History | 4.023*** |
| Ethnicity | .175 |
| Veteran Status | .624 |
| Age | .002 |
| aDependent Variable Situational Video bR Squared = .189 *Significant at the 0.05 level **Significant at the 0.01 level ***Significant at the 0.001 level |
|
The third ANOVA model evaluated the oral presentation exercise. The demographic factors that showed significant differences in the oral presentation exercise scores include prior law enforcement history and ethnicity. Again, individuals with at least two years of previous law enforcement experience outperformed those with no prior law enforcement experience. There were no significant differences in scores for those with at least two years of experience, and those with less than two years of experience. In terms of ethnicity, Hispanic/non-white individuals outperformed white/non-Hispanics in this study. There was no significant difference between Hispanic/non-white and non-white/non-Hispanic subjects within the oral presentation exercise.
Table 2.3: ANOVA 3 |
|
|---|---|
| Source | Type III Sum of Squares |
| Corrected Model | 7.657b* |
| Intercept | 25.805*** |
| Gender | .006 |
| Education | 1.233 |
| LE History | 1.830± |
| Ethnicity | 1.542 |
| Veteran Status | .907 |
| Age | .513 |
| aDependent Variable Oral Presentation bR Squared = .147 ± Significant at the 0.10 level *Significant at the 0.05 level **Significant at the 0.01 level ***Significant at the 0.001 level |
|
The overall success rate of the assessment center method is 84 percent, which comprises candidates who are hired and successfully complete their probationary period. The assessment center method showed a 12 percent increase in overall success compared to the previous selection method of a written cognitive test and interview (72 percent success).
Discussion and Implications
When making any substantial procedural change, having empirical research supporting the belief that the new process is working allows administrators to breathe a sigh of relief. The police department could see that the assessment center process was delivering better-qualified candidates but statistically proving it was an important step in validating the new strategy’s success. In addition to the assessment center process resulting in a 12 percent increase in officer retention, this additional commitment of time and money in the selection phase provides crucial insight into successful officer characteristics. When the authors began to dig into what personal characteristics aided candidates in being successful, again, it was no surprise to find that judgment ranked at the top in their study.
The assessment center process defines judgment as the ability to gather factual information without jumping to conclusions, the ability to avoid acts that may be seen as compromising one’s integrity, utilizing multiple approaches in developing solutions, and the avoidance of bias or prejudice during the fact-finding phase. All of these capabilities are hallmarks of effective, professional police officers.
These attributes are elicited through a situational response exercise, where applicants view short video scenarios and then detail in a written report what steps they would take to resolve the issue. The fact this exercise shows to be a significant predictor of overall success aligns with the call of others to utilize similar exercises in assessing the whole person.9 As with most situations in policing, there are no singularly right answers. Instead, the exercise is designed to see to what extent candidates can think outside the box to solve problems.
The second assessment center exercise that best produced relevant data on a candidate’s overall success was the oral presentation. Candidates are provided the opportunity to select three random law enforcement speech topics from a total of eight. They can then choose the one that they feel most confident presenting. They develop a three- to five-minute speech on the selected topic that they present to the assessors. This exercise allows assessors to evaluate a candidate’s ability to communicate relevant, factual information in an oral manner—a requirement in law enforcement. Candidates who excel in this exercise generally speak on the topic from their personal experience, allowing assessors to observe any signs of potential bias or moral defects. Candidates who ignore the topic and present on an unrelated tangent are not as successful, demonstrating a lesser ability to develop multiple approaches to uncomfortable or challenging situations. Knowing the importance of judgment in law enforcement, it is not surprising that both of these exercises have judgment as a graded dimension.
Once judgment was found to be significant, the study began to focus on background traits and characteristics that successful candidates share. Again, many of the characteristics should come as no surprise. Applicants with bachelor’s degrees or more outpaced those with associate’s degrees. Those with military experience performed better in exercises that evaluated judgment than those without. Some of the more interesting characteristics were that women performed better than men and less than two years of law enforcement experience revealed the same success rate as those without any prior experience. In this study, Hispanics performed better than non-Hispanics, with all other races being equal. Age was also a significant factor in that the older a candidate was, the worse they did in the process. It is important to note that the characteristics discussed here are for demographic purposes only and cannot be used when forming hiring decisions.
Conclusion
Police leaders and directors in charge of recruitment can leverage this information in their efforts on reaching candidates most likely to be successful officers. Chiefs faced with making decisions should seek to include and select those candidates who demonstrate excellent judgement. There is little doubt that the face of law enforcement is changing. With some questioning the police’s intentions and others questioning the profession’s hiring practices, it should be reassuring to know that police hiring practices have evolved, resulting in the recruitment and selection of diverse candidates who become successful police officers.
At the end of the day, what matters most is judgment. This study has shown that candidates who demonstrate the ability to think on their feet, without bias or prejudice, become successful police officers. Further, this study shows that the applicants most likely to excel in that area are a diverse group of people. By targeting recruitment efforts towards those demographics and leveraging data obtained in the hiring process, agencies are more likely to hire officers who exhibit integrity, fairness, and unbiased behavior.
Notes:
1Alex Neicu, “The Right Person for the Right Position: Changing Hiring Practices to Meet 21st Century Demands,” Police Chief 85, no. 9 (September 2018): 30–38.
2Police Executive Research Forum (PERF), The Workforce Crisis, and What Police Agencies Are Doing About It (Washington. DC: PERF, 2019).
3Anne Li Kringen, “Assessing the Value of Evidence: Understanding Research on Recruiting and Hiring,” Police Chief 87, no. 4 (April 2020) 34–38.
4Lance Anderson, Shane Pittman, and Michael McLenagan, “Ensuring That Hiring and Promotional Procedures Support Organizational Values,” Police Chief Online, July 15, 2020.
5International Task Force on Assessment Centers, “Guidelines and Ethical Considerations for Assessment Center Operations,” Journal of Management 41, no. 4 (May 2015): 1248.
6Marvin D. Dunnette and Stephan J. Motowidlo, Police Selection and Career Assessment (Washington, DC: U.S. Government Printing Office, 1976), 208.
7International Task Force on Assessment Centers, “Guidelines and Ethical Considerations for Assessment Center Operations.”
8International Task Force on Assessment Centers, “Guidelines and Ethical Considerations for Assessment Center Operations.”
9Anderson, Pittman, and McLenagan, “Ensuring That Hiring and Promotional Procedures Support Organizational Values.”
Please cite as
Joseph E. Roy, David A. Cefalu, and Pavan R. Chennamaneni, “Data-Driven Personnel Selection: An Exploratory Analysis of the Assessment Center and Hiring Process,” Police Chief Online, January 6, 2021.
