By Cassi L. Fields, PhD, Fields Consulting Group Inc., McLean, Virginia; and Karen J. Kruger, Esq., Funk & Bolton, P.A., Counsel, Maryland Chiefs of Police Association, Board Member of the IACP Legal Officers’ Section
he U.S. Supreme Court issued a decision in May 2010 holding that the city of Chicago is liable under Title VII for the adoption of a testing process that had a disparate impact on African Americans applying for firefighter positions and also for the application of that process when it selected applicants from the list generated by that flawed test some years later.1 The Court decided that during the entire life of an eligibility list and one year after the list’s expiration, any candidate may sue for discrimination if each hire causes or exacerbates adverse impact. In the past, this was not the case because, when an eligibility list was initially published, it was reviewed at that time for adverse impact and potential adverse impact. Now, plaintiffs can evaluate the list after its expiration to determine how it might have ultimately impacted those who might claim discrimination.
In January 2010, the Equal Employment Opportunity Commission filed a complaint in the U.S. District Court for the District of New Jersey on behalf of candidates in the New Jersey Civil Service Commission who were seeking promotion to the rank of sergeant through the Police Sergeant exam. The complaint alleges that the state’s pass/fail testing process had a negative, disparate impact on African Americans and applicants of Hispanic origin.2
In Ricci v. DeStefano,3 another case involving firefighters, the city of New Haven, Connecticut, was found to have violated Title VII when it rejected successful promotional test results achieved by nonminorities in favor of minorities, in its effort to avoid a disparate impact against those minorities. Essentially then, a jurisdiction cannot discard an eligibility list solely because it does not produce a diverse group of candidates. Similarly, the city of Milwaukee in 2007 was found to have engaged in unlawful discrimination for using promotional practices that favored women and minorities in promotions from police lieutenant to captain.4
These cases could leave law enforcement managers with the sense that they are in no-win positions when it comes to avoiding liability under Title VII with regard to promotional testing. But there could be a solution.
One large, metropolitan, public safety department, whose policy is to use promotional exams to promote its supervisors and managers, started using a comprehensive, nine-step promotional program. The purpose of the nine-step approach was multifold. The agency desired to increase transparency of the entire promotional process; obtain buy-in and support from the candidates; create a highly reliable and valid promotional process that identifies the most qualified candidates; and create methods to reduce adverse impact. The resulting comprehensive approach was designed to motivate candidates; reduce their anxiety about the promotional process; improve their test-taking skills; and assist with both their career development and the agency’s development.
One of the most challenging aspects of using promotional exams in public safety departments is ensuring that the process is valid and produces a diverse group of candidates. Further, the prevailing legal guidelines and court rulings make it more difficult for a law enforcement executive to make selections from a valid test if those selections result in adverse impact. Since the inception of the 1964 and 1991 Civil Rights Acts, court decisions with regard to employment discrimination have added to all test users’ burden to defend its testing processes. The current rash of employment discrimination lawsuits in public safety selection and promotional exams proves this point.
Many observers of these cases have experienced the frustrations that led to litigation—namely, “How do I use a promotional test and not get sued for adverse impact?” or “What strategies can I use to ensure the diversity of my workforce while avoiding lawsuits for civil rights violations?” Many police chiefs have experienced the feeling of being between a rock and a hard place when implementing selection or promotional systems.
The challenge facing test creation concerns not only validity but also whether the test user considered alternatives that have less adverse impact than the test that was previously used. The nine-step approach accomplishes both goals: it increases validity and facilitates a real-time search for alternatives that reduce adverse impact. The process comprises
- test orientation,
- test preparation,
- construction and administration of a valid practice testing process,
- candidate feedback regarding practice assessment center performance,
- analysis of practice testing process,
- design of alternatives targeted to reduce or eliminate adverse impact,
- construction and administration of a valid final testing process,
- design of a promotional list with administrative banding, and
- design of an unbiased selection system from the administrative bands.
Disparities in Test Performance
There are three sets of theories that attempt to explain differences in test performance among population subgroups.
Cognitive ability. One set of theories rests on the premise that tests of cognitive ability are valid. This approach then concludes that observed differences in cognitive ability among population subgroups can be attributed to heredity. 5 In general, these theories do not hold promise for immediately addressing test performance differences or minimizing test litigation since this area of research concludes that cognitive ability is genetic and highly persistent, stable, and virtually unchangeable.
Test bias. Another set of theories focuses on test bias or test flaws that may contribute to test performance disparities. These theories suggest that test bias is either partially or entirely the cause of test performance disparities. These biases and flaws include, but are not limited to, test unreliability, invalid cognitive load, and invalid passing or weighting schemes.6 In general, these theories hold little promise for eliminating adverse impact or litigation since a perfectly reliable test has yet to be achieved, and the research regarding test bias has not produced conclusive evidence that test bias is the cause of adverse impact.
Environment and experience. The nine-step approach, instead, focuses on test performance moderators originating from the environment that some research has shown impacts test performance and explains subgroup test performance disparities. These test performance moderators include test preparation, study skills, test-taking motivation, test anxiety, and test practice. Each of these moderators is shaped by test takers’ educational and socioeconomic experiences.7 The main difference between this set of explanatory theories and the other theories is that test performance moderators can be controlled. In the nine-step approach, these moderators are equalized across groups and consequently close test performance gaps.
Disparities in Cognitive Ability
One comprehensive promotional strategy was designed with the assumption that subgroups within most public safety departments have similar cognitive ability. This assumption is based upon several facts. First, each applicant for a sworn position is subjected to rigorous and consistent hiring standards. Second, after hire, employees are put through rigorous, uniform training that they must pass in order to be sworn into their positions. All anecdotal data of subgroup training and overall job performance in public safety departments support the premise that the cognitive abilities among the members are similar; that is, that subgroups do not perform noticeably differently on performance measures in training or on the job.
Disparaties in Culture
Since cognitive ability is likely similar across public safety personnel, it stands to reason that cultural influences contribute significantly to subgroup differences in test performance. These cultural influences might be the main cause of any observed statistical differences in test performance between subgroups or may exacerbate those differences. Specifically, some candidates may have lower-quality primary and secondary school experiences. This supports conclusions presented earlier that, in general, some candidates will have
- less effective test preparation strategies;
- less effective test-taking strategies;
- less motivation to succeed due to
- lower self-efficacy,
- lower expectancy,
- less belief in testing processes, and
- stereotypical beliefs about minority test performance; and
- higher test anxiety than other candidates.
Overcoming Test Moderators
The ingredients of one approach designed to have high validity and low adverse impact include the following components.
- A test orientation that is clear and transparent regarding what candidates should expect in their promotional system is scheduled.
- A test preparation course that provides candidates with a complete understanding of the content and the format of their practice examination is offered. A test preparation course includes test practice with elements to address test preparation, test-taking skills, test-taking motivation, and test anxiety. It also assists in reducing stereotype threat. If the pretest training also incorporates technical training in the relevant knowledge, skills, and abilities, this can decrease subgroup test score differences.
- A valid practice test that exposes candidates to the type of final test they will take, in terms of content and format, is given. In some cases, it may be necessary to alter the final test significantly if the practice test results indicate low validity and/or high adverse impact. This strategy will be successful only if during candidate orientation, this potential is clearly communicated. Otherwise, the entire transparent model will be lost.
- Candidate feedback regarding their practice test performances and methods to improve test scores is solicited.
- A valid final test that is constructed based on the results of the practice test is given. The intent should be to improve reliability and validity while decreasing adverse impact.
- An unbiased practice and final test ensure validity and are necessary precursors to any pretest training that might serve to close subgroup gaps in test performance. The test is reliable, places job-related cognitive processing demands on the test taker, and possesses job-related test component weights and passing scores.
- A promotional registry strategy that allows for nonrank-ordered selection is instituted. Administrative banding is a transparent and practical strategy.
- A selection strategy from the promotional registry that is structured, unbiased, and allows for the consideration of other job-related factors is followed.
Costs and Benefits
This comprehensive, nine-step approach is entirely transparent. All stakeholders who value transparency can be told and can understand every step of this process. The current trend toward finding alternative weighting and passing strategies after a test is administered is not necessary using this procedure. Further, weighting and passing are based upon the results of the practice test, giving the weights and the passing scores more practical and scientific credibility.
Any department that utilizes this approach is making a noticeable attempt to find an alternative selection procedure that is valid and possesses low or no adverse impact. This will be appreciated by all stakeholders, and, alternatively, the courts that may be inclined to believe that discrimination is intentional if a department does not take active steps to alleviate adverse impact.
The cost of a program to address adverse impact is less than the potential costs of protracted litigation. Anecdotal evidence shows that, per rank, litigation costs can total up to $4 million. Further, in some jurisdictions, litigation for one law enforcement rank can take up to 10 years, inhibiting promotions in that department during that time.
Socially and politically, this comprehensive approach does not appear to offend the rank and file. For example, African American candidates do not feel stereotyped or labeled, and Caucasian candidates do not feel that African Americans are being given an unfair advantage.
The practice exam allows for an extensive evaluation of all subgroup performances so that adverse impact can be minimized across all groups. If a problem is observed in the practice exam (for example, low reliability, low validity, or high adverse impact), creative alternatives can be substituted for the problematic components.
Administrative banding is a transparent approach that candidates can understand. It is a rule-based approach that prevents an immediate injunction against the test if the test has adverse impact, because the test is not the only consideration for promotion.
This comprehensive approach addresses the issue of group-specific adverse impact. When researchers have given exams to one candidate population in a public safety department, then administered almost that exact same exam in another public safety department, one candidate population typically will have no adverse impact, and the other will. The practice exam allows an analysis of subgroup differences for one group of applicants in advance so that proactive steps can be taken, if needed, to address those gaps in the final process.
Some other ancillary benefits include
- the levels of anxiety observed in the candidates appeared much reduced (for example, candidates showed up at the assessment center minutes before the start of their final test as compared to hours before their practice test);
- most candidates took information given during feedback and utilized it in the final test, including using their time limits more effectively and providing more detail in their responses;
- most candidates appeared highly motivated to succeed and reported to test administrators that they were confident that they could perform well; and
- candidates reported, in writing, favorable perceptions about the fairness and job-relatedness of the promotional process.
The techniques used in this comprehensive promotional program may finally provide a reasonable solution to departments that have struggled with promotional testing issues. ■
1Lewis et al. v. City of Chicago, Illinois, 560 U.S. ____ (2010); 130 S. Ct. 2191; WL 2025206 (May 24, 2010).
2United States v. State of New Jersey & New Jersey Civil Service Commission, Case No. 2:33-av-00001, United States District Court for the District of New Jersey (filed January 7, 2010), http://www.justice.gov/crt/emp/documents/newjerseycomp.pdf (accessed August 19, 2010).
3Ricci v. DeStefano, ___U.S. ____, 129 S. Ct. 2658 (2009).
4Alexander et al. v. City of Milwaukee et al., 474 F. 3d 437 (7th Cir. 2007).
5John P. Campbell, “Group Differences and Personnel Decisions: Validity, Fairness, and Affirmative Action,” Journal of Vocational Behavior 49 (October 1996): 122–158; Linda S. Gottfredson, “Why g Matters: The Complexity of Everyday Life,” Intelligence 24, no. 1 (1997): 79–132, http://www.udel.edu/educ/gottfredson/reprints/1997whygmatters.pdf (accessed August 19, 2010); Ulric Neisser et al., “Intelligence: Knowns and Unknowns,” American Psychologist 51, no. 2 (February 1995): 77–101, http://www.gifted.uconn.edu/siegle/research/Correlation/Intelligence.pdf (accessed August 19, 2010); J. Philippe Rushton and Arthur R. Jensen, “Thirty Years of Research on Race Differences in Cognitive Ability,” Psychology, Public Policy, and Law 11, no. 2 (2005): 235–294, http://psychology.uwo.ca/faculty/rushtonpdfs/PPPL1.pdf (accessed August 19, 2010); and Paul R. Sackett and Steffanie L. Wilk, “Within-Group Norming and Other Forms of Score Adjustment in Preemployment Testing,” American Psychologist 49, no. 11 (1994): 929–954.
6John E. Hunter and Ronda F. Hunter, “Validity and Utility of Alternative Predictors of Job Performance,” Psychological Bulletin 96, no. 1 (1984): 72–98, http://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf (accessed August 19, 2010); and Paul R. Sackett et al, “High-Stakes Testing in Employment, Credentialing, and Higher Education: Prospects in a Post-Affirmative-Action World,” American Psychologist 56, no. 4 (April 2001): 302–318.
7Ann Marie Ryan, “Explaining the Black-White Test Score Gap: The Role of Test Perceptions,” Human Performance 14, no. 1 (2001): 45–75.
Please cite as:
Cassi L. Fields and Karen J. Kruger, "A Comprehensive Approach to Eliminating Adverse Impact in Public Safety Promotional Exams," Chief's Counsel, The Police Chief 77 (October 2010): 12–14, http://www.nxtbook.com/nxtbooks/naylor/CPIM1010/#/12 (insert access date).