Does “behavioral interviewing” of job candidates really work?

Dex replies:

I’m going out on a limb here, but my guess is you didn’t get the job.

It’s not going to help much if I tell you that a job interview is intended to help determine whether there’s a good fit between candidate and employer, and that in the long run, you’re better off if you don’t get hired for a job where there’s not a good fit. Granted, in the current lousy economy, a job is a job. However, we need to take the big picture view.

For those who haven’t interviewed for a job in a while, “behavioral interviewing” is a technique in which the interviewer asks how the candidate behaved in real-life situations having some relevance to the job being applied for. Before we can talk about whether it works or not, we need to talk about the alternatives, that is, the range of methods an employer can use to assess whether a candidate is qualified for a job and is better suited than others. It’s not easy. As you say, most selection methods assume that the past offers some guide to the future. If you want to hire an accountant, you’ll probably make the assumption that someone with a strong college math and business education including accounting courses will do a better job than someone who dropped out of high school to be second trombone in a jazz band. That judgment could be wrong, but it’s one that most employers (and most people, for that matter) would make.

The three basic methods for selection are (1) background investigation and reference checks, (2) testing, and (3) interviewing.

Background Investigation and Reference Checks

Background checks are used extensively, potentially covering areas such as educational history, past employment, and references listed on your resume. Some employers check for a criminal record or IRS charges, review your credit report, and perhaps even talk with co-workers or neighbors not listed on your resume. The intention is to verify factual information that the candidate provided and to uncover any damaging information that wasn’t provided. Background checks seem like an elementary precaution, particularly when you’re hiring someone for a sensitive position, but it’s surprising how many employers do them in a slapdash manner. No doubt you’ve seen news accounts of people with criminal records being hired for airport security jobs, and of others in prominent positions who were found to have lied on their resumes. Usually when the truth comes out they get fired, to the embarrassment of all concerned. A proper background check during the initial hiring process would have saved everyone a lot of grief.

Useful as they can be, background checks are limited in scope. Federal and state laws in the U.S. protect the rights of individuals during background checks. Candidates have sued former employers for slander, libel, or defamation of character based on comments that may have prevented the individual from getting a job. Consequently, many companies now won’t give out any information other than name, employment dates, job title, and perhaps salary information. Background checks seldom cover prior job performance. Most human resource managers view reference letters as having little value beyond simple fact-checking, and studies show they have only a moderate correlation with future performance. More on this later.

Testing

Tests are commonly classified based on what they measure: cognitive (mental) ability, physical abilities, personality and interests, or achievement.

• Cognitive (mental) ability tests include general intelligence tests as well as specific aptitude tests, such as verbal comprehension, reasoning ability, memory, and numerical ability. If you are applying for a job that involves lots of thinking, say in a management consulting firm, a prospective employer may ask you to take such a test. The results of cognitive tests have a very high correlation with future job performance.
• Tests of physical ability might include muscular tension, muscular power, muscular endurance, cardiovascular endurance, flexibility, balance, coordination, dexterity, reaction time, etc. If you’re are applying for a job with the police force or fire department, expect a barrage of physical ability tests.
• Achievement tests measure what a person has learned, either in school or on the job. Sometimes, achievement tests provide a professional credential (passing the bar exam or getting certified as an actuary or accountant, for instance) and are administered not by the prospective employer but by a professional organization.
• Tests of personality and interests tend to cover attitude, motivation, conscientiousness, and agreeableness (these are all defined psychological terms). Many companies require such tests of applicants for managerial jobs. Can applicants distort their responses and “outsmart” the test (or the tester), to make a better impression? They can try. Most companies warn applicants that distortions can and will be detected, with consequences. But there is no agreed-upon method for detecting faked answers in personality and interest tests. At bottom, such tests assume most people will answer honestly out of self-interest: Why would you want to fool people to get a job where you’ll be miserable?

Some companies screen for substance abuse. A few require tests of dubious validity, such as polygraphs or handwriting. Under the Americans with Disabilities Act, pre-employment medical exams are prohibited until a job has been conditionally offered.

Interviews

The interview is intended to obtain information through oral responses to oral questions. Generally, the interview is only one of several selection tools, but it is by far the most common. Not all companies use tests or even reference checks, but it would highly unusual for a manager not to interview a prospective employee.

Interviews can be structured or unstructured. The unstructured (sometimes called “nondirective”) interview has no set format. The interviewer can pursue whatever points of interest happen to come up. Often there is little to distinguish such an interview from an ordinary conversation. Unstructured interviews, not surprisingly, have moderate to low correlation with future job performance.

In contrast, in a structured (sometimes called “directive”) interview, the employer establishes a sequence of questions in advance. The candidate’s responses can then be rated–often formally, say on a 1-to-5 scale, based on appropriateness.

Structured interviews are generally job-focused but can use different types of questions to assess a candidate’s fitness for the position. The classifications below are mostly academic. In most real-life situations, a variety of types of questions would be used. (To avoid pronoun problems, I’ll assume the candidate is female and the interviewer is male.)

• Situational. The candidate is asked about her behavior in a hypothetical job-related situation. For instance, a candidate for a supervisor position might be asked how she would deal with a subordinate who was late three days in a row.
• Behavioral. This is the type of question you’re talking about. The candidate is asked about her behavior in an actual past situation. For example, a candidate for a call center position might be asked, “Tell me about a time you were speaking with an irate person, and how you turned the situation around.” The logic here is that past behavior is a good predictor of future actions.
• Relational. The candidate is asked a series of job-related questions that don’t involve scenarios. For example, “What courses did you like best in school?”
• Stress interview. The interviewer tries to make the applicant uncomfortable by a series of questions that might be rude. The idea is to identify candidates who might be overly sensitive, or those with low or high stress tolerance. For example, an interviewer might pry intensively into why the candidate left past jobs. The stress interview is a high-risk approach for an employer. Apart from possibly antagonizing the interviewee, the technique can give a false reading–the typical candidate already has some baseline anxiety about an interview.

Each type of question has its place, but behavioral questions are especially useful. The answers give the interviewer lots of information, since (in theory) they’re based on reality. It’s harder to fool an interviewer about something that supposedly actually happened.

You say it’s possible to find typical questions online and concoct responses to them, and that “unsubstantiatable and self-serving stories are the most effective tools in landing the job.” Some comments:

• A person who wants to lie in response to a question about past behavior can lie when asked any type of question. There’s no reason to single out behavioral questions. In fact, behavioral questions tend to be much harder to fake. A candidate might be able to manufacture an answer to one or two such questions, but the third or fourth will usually expose any deceptions. A good interviewer might go back to an example given in an earlier response and delve deeper. For instance, if the interviewer starts by asking how the candidate handled a situation, he may follow up by asking why she chose a particular approach, how others reacted, how were things resolved, and so on.
• Self-confidence might be a positive trait, but self-aggrandizement is NOT the most effective tool in landing a job. I’ve rejected candidates because they were self-centered “stars,” and I felt they wouldn’t be good team players in a situation that required working closely with others.
• While it’s possible for a great actor to fake a few answers, why bother? If you’re qualified for the job and are caught lying, you’ll have blown your chances. If you weren’t qualified for the job, interviewed on a lark, and managed to fool the interviewer, your lies would be exposed after a brief period on the job, and lying in the interview is grounds for dismissal. How then do you explain in the next interview why you were fired after only a month? If you’re desperate for a job in a crummy economy, lying may get you a few paychecks. But from a long-term perspective, that kind of deception is likely to be self-destructive.

Structured interviewing has done much to redeem a job selection technique that was once held in low esteem. Fifteen or twenty years ago, managers thought interviews were unreliable, low in validity, and biased. Interviewing can be costly, both in time and travel. Since interviews are subjective, rejected applicants sometimes sue, especially if asked irrelevant questions. Today the situation has improved. To avoid discrimination lawsuits, many larger companies today train managers in how to ask relevant questions in a structured format, often asking behavioral questions. The focus is on gathering specific information relevant to the job. Consequently, the reliability and validity of interviews have improved markedly. The goal of most interviews now is “controlled subjectivity.” Testing might supplement interviews when rating some abilities. Companies often use multiple trained interviewers per candidate, to avoid subjective errors.

Research suggests that structured interviews have validity about double that of unstructured interviews. There are conflicting results on whether situational interviews or behavioral interviews yield higher mean validity–and I’m not convinced that most interviewers stick to asking only behavioral or situational or relational questions. But structured interviews, regardless of the type of question asked, are more valid than unstructured interviews.

Is there specific research that links the selection process to job performance? Yes, there are several studies. Wayne F. Cascio, in his textbook Managing Human Resources, cites F.L. Schmidt and J.E. Hunter, reporting in the Psychological Bulletin (1998). They show the degree of correlation between various candidate selection methods and later job performance. The highest correlations come from work sample tests, general mental ability tests, and structured employment interviews, all with correlation coefficients of 0.5 or above (the range is -1.0 to +1.0)–which isn’t bad, statistically.

I wasn’t able to locate a study that distinguished between pure behavioral and situational interviews–and, as I say, I think the distinction is mostly academic. But the high correlations for structured interviews should be enough to answer your question.

By the way, the next tier in terms of validity includes peer ratings, job knowledge tests, integrity tests, and job tryout procedures. Unstructured interviews have low correlations, around 0.25, while years of education, interests, age, and handwriting tests have correlations near zero.

I suppose to a job candidate a structured interview may seem more threatening than the old-fashioned, shoot-the-breeze variety. After all, the interviewer is trying to evaluate your strengths and weaknesses. The interview is a test, you’re being rated, and it’s possible for you to do poorly. But you need to understand that from an employer’s perspective, structured interviews are a way to lend some system to a process that in the past was often a shot in the dark. As such, they can work to the benefit of job seekers as well as employers. They make it less likely that an employer will misjudge and select an ill-suited candidate over someone better qualified.

You’re right about one thing–it’s possible to interview well but not do the job well, and vice versa. Structured or not, interviewing remains a subjective process. The interviewer, not unreasonably, makes judgments based on nonverbal cues. A person who is nervous, jumpy, and jittery in the interview may come across the same way with customers or subordinates or supervisors. A candidate’s behavior under the stress of an interview can be a good indicator of such traits as introversion/extroversion, agreeableness, etc. On the other hand, an interview probably will not give good clues to such important job-related traits as conscientiousness or emotional stability.

An interview can be undermined if the interviewer fails to understand the job he’s interviewing for or attaches too much weight to a candidate’s nonverbal behavior and personal characteristics (like attractiveness, gender, race). The order in which candidates are interviewed affects how they are rated. An employer who has interviewed several total losers might regard an average candidate as great by comparison.

Most companies–certainly the larger ones–are aware of these weaknesses and try to compensate for them. It’s to their advantage to do so. Hiring the wrong person can be an expensive, sometimes catastrophic mistake. The company saves a great deal of money if interviewing is focused and structured, and if the hiring process is as error-free as possible.

It doesn’t hurt to prepare for a structured interview as you would for any test. As you say, it’s easy to obtain sample questions online, and while they won’t necessarily be the ones you’ll be asked, thinking about them will prepare you for the interview ordeal. Having responses for some of the more common questions makes it less likely you’ll stutter or stammer when the interviewer puts you on the spot. But I wouldn’t advise finding some totally fictional experiences to pass off as your own. There’s a good chance you’ll be found out, and even if you’re not, you may wind up in a job for which you’re not suited.

In short, while structured (behavioral and situational) interviews aren’t perfect, they rank reasonably high, statistically speaking, as a guide to selection and as a predictor of future performance. Rather than try to subvert what is basically a reasonable process, you’d be better off trying to understand it and making it work to your advantage.

Resources:

Cascio, Wayne F., Managing Human Resources (6th edition), McGraw Hill (2003)

Dessler, Gary; Human Resource Management (9th edition), Prentice Hall (2003).

Mathis, Robert L., and Jackson, John. H., Human Resource Management (10th edition), Thomson Learning (2003).

Dex

Send questions to Cecil via cecil@straightdope.com.

STAFF REPORTS ARE WRITTEN BY THE STRAIGHT DOPE SCIENCE ADVISORY BOARD, CECIL’S ONLINE AUXILIARY. THOUGH THE SDSAB DOES ITS BEST, THESE COLUMNS ARE EDITED BY ED ZOTTI, NOT CECIL, SO ACCURACYWISE YOU’D BETTER KEEP YOUR FINGERS CROSSED.