Ability-tests.ppt 493dp

Ability Tests

Sensory (e.g., hearing, vision)

Motor/Physical (e.g., dexterity, strength, agility)

Cognitive (e.g., intelligence, aptitude)

Cognitive Ability (e.g., ability to learn, or potential to learn, and acquire new knowledge and skill)

Spearman, C. (1904) ‘General intelligence,’ objectively determined and measured. American Journal of Psychology, 15, 201-293. Also, differentiated general intelligence from specific (s) abilities (e.g., Spatial, Perceptual, Mechanical, Verbal, Numerical)

Cognitive Ability (cont.)

• About 50% of the variance in cognitive ability is due to g • Roughly 8-10% due to verbal, quantitative, and spatial abilities • Differences in specific abilities (verbal and numerical abilities, spatial, perceptual, mechanical are due to differences in “investment” (or focus, interest) in each area. Role of education, opportunity,

The Wonderlic Personnel Test • Wonderlic Personnel Test  Developed in 1938, in wide use thereafter  Is a 50 multiple-choice item test taken in 12 minutes  Content—vocabulary, “commonsense” reasoning, formal syllogisms, arithmetic reasoning and computation, analogies, perceptual skill, spatial relations, number series, scrambled sentences, and knowledge of proverbs.  Primarily measures verbal comprehension, with As ofdeduction 2011, the tests been istered to over 200 andhave numerical fluency being the million people next two factors in order of importance.

Surveys have reported the use of CA tests to range from 11% to 27%

~ Differential Aptitude Tests ~ Designed to measure an individual's ability to learn or to succeed in a number of different areas such as mechanical reasoning, verbal reasoning, numerical reasoning, and space relations. Verbal Reasoning Numerical Ability Abstract Reasoning Mechanical Reasoning Space Relations Language Usage

Differential Aptitude Test (DAT)

Verbal Reasoning ..… is to water as eat is to …..

..... is to night as breakfast is to …..

A. B. C. D. E.

A. B. C. D. E.

continue ----- drive foot ----- enemy drink ----- food girl ----- industry drink ----- enemy

….. is to one as second is to ….. A. B. C. D. E.

two ----- middle first ----- fire queen ----- hill first ----- two rain ----- fire

supper ----- corner gentle ----- morning door ----- corner flow ----- enjoy supper ----- morning

Differential Aptitude Test (DAT) Numerical Ability Add 30 20

A. 8 B. 15 C. 16 D. 26 N. none of these

Add 13 12

A. 14 B. 16 C. 25 D. 59 N. none of these

Differential Aptitude Test (DAT) Abstract Reasoning PROBLEM FIGURES

ANSWER FIGURES

A

B

C

D

E

Differential Aptitude Test (DAT) Abstract Reasoning (cont.) PROBLEM FIGURES

ANSWER FIGURES

A

B

C

D

E

Differential Aptitude Test (DAT)

Mechanical Reasoning

A

B

Which weighs more? (If equal, mark C.)

Mechanical Reasoning (cont.)

Mechanical Reasoning Sample Test

Differential Aptitude Test (DAT) Space Relations

A

B

C

D

Example Items Similar to Items on the Minnesota Clerical Test Name Comparison Neal Schmitt ____________________________ Hubert Feild ____________________________ Chris Riordan ____________________________ Tim Judge ____________________________ Murray Barrick ____________________________

Frank Schmidt Herbert Field Kris Reardan Jim Fudge Mick Mount

Number Comparison 84644 ____________________________ 179854 ____________________________ 123457 ____________________________ 987342 ____________________________ 8877665994 ____________________________

84464 176845 12457 987342 8876659954

Cognitive Ability (cont.) ~ Measurement (Reliability) ~ Cognitive Ability Tests are among the most reliable assessments used in organizational settings Reliability estimates: • Overall, about .88 - .90 • Test-retest (average interval 24 weeks) .83 (.65 - .95 range) • Correlation between different tests (varying item type, content, format) = .77 _________________________________________ • Most reliable – Verbal and Numerical Abilities • Less reliable – Spatial, Perceptual, Mechanical

Cognitive Ability (cont.) ~ Measurement (Validity) ~ Cognitive Ability Tests are among the best predictors of job performance across jobs and settings  Individuals with high CA possess high levels of: • Declarative Knowledge (facts, procedures, rules) • Procedural Knowledge (what to do) Cognitive Ability Performance

Job Knowledge ()

(mostly supervisor ratings)

Job

Cognitive Ability Tests (Managerial Performance) Significant correlations of “g” with managerial performance • Uncorrected = .25 to .35 • Corrected = .43 to .53 (Ghiselli, 1972; Hunter & Hunter, 1984)

Verbal Comprehension ---- .18 Numerical Ability ---------- .42

R = .52 (Verbal Reasoning and Numerical Ability alone)

Visual Speed/Accuracy --- .41

Verbal Reasoning:

Space Visualization ------- .31

• Top managers - 98 percentile

Numerical Reasoning ---- .41 Verbal Reasoning --------- .48 Word Fluency ------------- .37 Symbolic Reasoning ------ .31

• Middle managers - 95th percentile Numerical Ability: • Top managers -85th percentile • Middle managers -59th percentile From: Grimsley & Jarrett (1973, 1975)

The Validity of Mental Ability Tests • Project A  A multiple-year effort to develop a selection system appropriate for all entry-level positions in the U.S. Army  Involved the development of 65 predictor tests that could be used as selection instruments  Produced results indicating that general mental ability tests are valid selection instruments across a large variety of military jobs

Project A Validity Coefficients

Validity Across Jobs

Cognitive Ability Validity Versus Other Tests

Other Selection Devices Test

Corr. with Performance

Corr. with CA

Incremental Validity

Job Knowledge

.40 - .50

.63 - .80

0 - .04

Work Sample

.31 - .43

.30 - .50

.02 - .05

Assessment Center (Problem Solving)

.31 - .39

.28 - .32

.02 - .05

Situational Judgment Tests

.19 - .26

~ .32

0 - .01

Assessment Center (Overall)

~ .44 (Mechanical combination) ~ .31 (Holistic)

.43

.13

Interview

.44 (high structure)

.12 - .16*

.09 - .12

Biodata

.34

.37

.02

Source: Ones, Dilchert & Viswesvaran (2012)

Cognitive Ability and Interviews (Meta-analysis = .42 correlation) Roth & Huffcutt (2013) • Theoretical Reasons for a Relationship • Involve technical, problem solving, learning about policies & procedures (describe past accomplishments) • Require understanding of meaning, organization of answers, processing speed • Impression management Earlier meta-analysis reported an overall corrected correlation of . 40 (Huffcutt, Roth, & McDaniel, 1996) More recent meta-analysis reported a noticeably lower corrected correlation of .27 (Berry, Sackett, & Landers, 2007).

Non-Cognitive Scales and Incremental Validity Incremental validity

So-called “compound” personality measures (included here) are NOT highly related to CA and provide decent incremental validity

Trait EI assesses such things as self-esteem, stress management, adaptability, & emotional stability

Cognitive Ability Predictive Power Across Time Consistent tasks (behavior becomes automatic, effortless, routine) Predictive value of CA drops over time Inconsistent tasks (job duties differ; tasks are “resource” dependent) Predictive value remains stable over time

Validity Generalization or Situational Specificity Can Cognitive Ability Tests Generalize Across Jobs or Does Individual Validity Studies Need to be Conducted?

Situational Specificity Validity coefficients for the same combination of mental ability tests and job performance measures differ greatly for studies in different organizations These differences were thought to be caused by undetermined organizational factors that affected the correlation between selection instruments and criteria Selection specialists concluded that a validation study is necessary for each selection program developed.

Validity Generalization

Meta-analyses indicate the robust nature of general cognitive ability tests

Evidence for Validity Generalization

Validity Generalization (cont.) • Implications for Selection  Conducting validity studies within each organization is not needed  No organizational effects on validity; the same

predictor (selection instrument) can be used across all organizations  It is necessary only to demonstrate through job analysis that jobs are similar to the job in the validity generalization study

 Task differences among jobs have little

effect on the validity coefficients of mental ability tests. 

Differing information-processing and problemsolving demands of the job, not task differences themselves

Cognitive Ability Tests (cont.) » Strengths • •

High reliability Criterion-related validity for wide range of jobs (especially high level positions and high complexity jobs such as managerial positions)

•

Group istration

•

Ease of Scoring

•

Relatively low cost (e.g., versus personality test)

 Weaknesses • Likelihood for adverse impact (minorities score lower than nonminorities) • Females score lower on tests of specific abilities (e.g., mechanical ability) • Overall, best to use in combination with other tests/inventories

Assessing Emotion Scale (AES) 33 Items arranged on a 5-point scale (“1” strongly disagree to “5” strongly agree) • I find it hard to understand the non verbal messages of other people • I am aware of the non verbal messages I send to others • By looking at their facial expressions, I recognize the emotions people are experiencing • I know why my emotions change • I know what other people are feeling just by looking at them • When another person tells me about an important event in history her life, I almost feel as though I experienced this event myself • When I feel a change in emotions, I tend to come up with new ideas • I help other people feel better when they are down • It is difficult for me to understand why people feel the way they do

Group Differences in Cognitive Ability Scores

Types of Physical Ability Tests Basic Ability Tests: Measures a single ability that consists of medical-related information (e.g., aerobic capacity, heart rate) ADA issue: Considered as a medical test. Must be given post offer. • Can be used for multiple jobs • Safe to ister • Relatively inexpensive Physical Ability Tests (e.g., lifting, running, jumping) and Job Simulation Tests (those related to essential job tasks such as lifting/carrying objects, stair climbing, carrying a fire hose, climbing a fence) • Content valid

Determining Physical Requirements of Jobs • Job Analysis • Gather ergonomic, physiological, and biomechanical data (if needed) • Assess the role of work conditions on task performance (e.g., temperature, cramped spaces, PPE)

Physical Ability Tests • Reasons for Physical Ability Testing  More female applicants for male-dominated jobs  Reducing the incidence of work-related injuries, lost work days  To determine the physical status of job applicants • Legal Issues in Testing Physical Abilities  Adverse impact for scores on physical ability tests 



Tests must clearly be linked to critical job tasks that require physical abilities in their completion (Test must mirror the job demands; Key role of on-site observation) Question is whether the tasks can be modified to reduce or eliminate these physical demands (i.e., reasonable accommodation for disabled applicants).

Texas city hit with police sex discrimination suit CHRISTOPHER SHERMAN, Associated Press Updated 7:13 p.m., Tuesday, July 3, 2012 McALLEN, Texas (AP) — The Justice Department sued the city of Corpus Christi on Tuesday, alleging the Police Department discriminated in hiring women by using a physical ability test few female applicants have been able to . Federal prosecutors say only about one in five women who took the test between 2005 and 2009 ed it, compared with about two-thirds of the men. The last two years the rates for men and women increased due to a change in the cutoff scores, but the gap between men and women persisted. The complaint filed in federal court in Corpus Christi says the department hired 12 female entry-level officers and 113 males from 2005 to

http://www.cctexas.com/? fuseaction=main.view&page=2478

Consent Decree (Settlement) The consent decree requires that Corpus Christi no longer use the physical abilities test challenged by the United States for selecting entry-level police officers.   It also requires the city to develop a new selection procedure that complies with Title VII.   Additionally, the consent decree requires the city to pay $700,000 as back pay to female applicants who took and failed the challenged physical abilities test between 2005 and 2011 and are determined to be eligible for relief.  Also under the consent decree, some women who took and failed the challenged physical abilities test between 2005 and 2011 may receive offers of priority employment with retroactive seniority and benefits.  Applicants interested in priority employment must the new, lawful selection procedure developed by Corpus Christi under the decree and meet other qualifications required of all applicants considered for entry-level police officer positions.  

Physical Abilities Fleishman’s 9 Categories of Physical Abilities

• Static strength • Dynamic strength • Explosive strength

Hogan’s 3 Categories of Physical Performance Muscular strength

• Trunk strength • Extent flexibility • Dynamic flexibility • Gross body coordination • Gross body equilibrium • Stamina

Movement quality Cardiovascular endurance

Physical Ability Categories (7)

Source: Baker & Gebhardt (2012). The assessment of physical capabilities in the workplace.