Reliability and validity: what HR professionals need to know

Why these two words matter when you choose a test

A test gives you a number that informs a decision about a person. That number is only useful if two things hold. It has to be consistent: the same person measured twice under the same conditions should get roughly the same result. And it has to mean something for the decision: a higher score should go with more of what matters in the role.

The first is reliability, the second validity. A test can be very reliable and still measure something irrelevant to the job. The reverse does not work: a test with inconsistent results cannot predict anything consistently either. Reliability is a precondition for validity, not a substitute for it.

Reliability: how consistent is the score?

Reliability is expressed as a coefficient between 0 and 1. There are several kinds, each answering a different question.

Test–retest reliability

The same people take the test twice. A high correlation means scores are stable over time, which matters for traits expected to be stable, such as personality. The interval matters: too short and people remember their answers, too long and real change gets mixed in.

Internal consistency

Do the items of a scale measure the same thing? The usual index is Cronbach's alpha; McDonald's omega is often more suitable when items reflect the construct with different strength. A commonly quoted rule of thumb is that around .70 can do for research at group level, while decisions about individuals call for clearly higher values. Test review models set their own thresholds, depending on what is at stake; see how test quality criteria work.

Inter-rater reliability

When assessors rather than a scoring key produce the score, as in interviews and assessment centre exercises, the question is whether different assessors reach the same rating. It is reported as an intraclass correlation or Cohen's kappa. Structured questions, behavioural rating scales and assessor training raise it.

The standard error of measurement

A reliability coefficient describes the test across a group. For an individual, the standard error of measurement (SEM) is more useful: the standard deviation of the scale multiplied by the square root of one minus the reliability. It shows how far an observed score can lie from the score the person would get without measurement error.

Validity: does the score mean what you need it to mean?

Validity is the more important of the two. Current professional standards do not treat it as a fixed property of a test, but as the degree to which evidence supports a particular interpretation of scores for a particular use. So the useful question is not whether a test is valid, but: valid for what, for whom and in which language?

Content validity

Do the items cover what matters in the role, and only that? Subject-matter experts usually judge this, for instance by comparing the tasks in a skills test with the job analysis.

Construct validity

Does the test measure what it claims to measure? Evidence includes the factor structure, correlations with similar and dissimilar measures, and whether the structure holds in each language version. Without it, you do not really know what the name of a scale stands for.

Criterion and predictive validity

Do scores relate to outcomes that matter, such as job performance, training results or retention? A predictive design tests people at hiring and measures the outcome later. A concurrent design tests current employees, which is quicker, but that group has already been selected, which tends to make correlations smaller. The criterion matters too: a vague or inconsistent performance rating limits how high any validity coefficient can be.

How big is good? Reading validity coefficients

Criterion validity is usually reported as a correlation, and in personnel selection those correlations are modest. That is normal: performance depends on much that a test cannot capture, from the team and the manager to life outside work. A correlation around .30 is a meaningful result in this field, not a weak one.

Sackett, Zhang, Berry and Lievens (2022) re-examined the meta-analytic evidence and arrived at more conservative estimates than older, still widely quoted figures, mainly because they judged earlier corrections for range restriction too large. Among their estimates:

• Structured interviews: about .42
• General cognitive ability: about .31
• Conscientiousness: about .19

Three points follow. These are averages across many studies and roles, corrected for statistical artefacts, so a specific test in your setting may do better or worse. All are clearly above zero: applied consistently across many hiring decisions, even modest validity improves the average outcome compared with unstructured judgement. And methods that measure different things add information. Cognitive ability and conscientiousness are only weakly related, so a combination tends to predict better than either alone. The same reasoning supports using cognitive ability tests alongside a structured interview rather than instead of it.

Reading a technical manual: questions to ask

A figure in a brochure means little without its context, and a good technical manual provides that context. Whether you read one or talk to a vendor, these questions help.

• Which kind of reliability is reported, and is the standard error of measurement given?
• How large was the sample, and who was in it: country, language, job level? Coefficients from a few dozen people are unstable.
• Which language version does the evidence concern? A translation needs its own data before you assume it works the same way.
• What was the criterion in validity studies, how was it measured, and was the design predictive or concurrent?
• For which use does the evidence hold: selection, development, a particular group of roles?
• How recent is it, and what does the manual say has not been studied?

"The test is validated", without saying for what, with whom and with which figures, does not answer the question. Incomplete evidence is acceptable, especially for new tests, as long as it is stated openly. On Match.hr, each test's psychometric label shows reliability with the sample it is based on, the intended use and the language versions analysed.

How this connects to norms and fairness

A score only becomes interpretable next to a comparison group, so the next question is which norm group fits your candidates and how recent it is; see norm groups explained. And a test can predict well on average while working differently for some groups, for instance when an item depends on language or cultural knowledge that is not part of what it should measure. Our page on fairness and bias covers how that is examined.

Consistent scores read as bands, evidence for your specific use, a fitting norm group and fairness analyses where lawful: together they make a test a sound basis for a considered decision by a person, not a replacement for it.