How to judge the quality of a psychological test

Why independent criteria help when you choose a test

Choosing a test from a brochure is risky. Marketing material tends to quote the best coefficient from the most favourable sample and to stay silent about what was never studied. A fixed set of criteria turns an impression into specific questions: what is the test for, how precise are the scores, who are candidates compared with, and is there evidence that the scores relate to what matters at work?

Two frameworks are widely used in Europe. COTAN, the test committee of the Dutch Association of Psychologists (Nederlands Instituut van Psychologen, NIP), has documented tests since 1959 and published ratings since 1969. The Test Review Model of the European Federation of Psychologists' Associations (EFPA) was built partly on the COTAN system and on British and Spanish criteria. Neither gives a test a universal seal of approval: both ask whether the evidence supports a particular use, with particular people, in a particular language.

The seven COTAN criteria in plain language

COTAN reviews each test on seven criteria. Two reviewers assess the test independently, and each criterion gets its own rating; there is no overall score. In the 2010 edition of the review system, each rating is insufficient, sufficient or good.

• Principles of test construction: are the intended use, the target groups and the construct described and grounded in theory? What a test claims to measure determines which evidence it needs.
• Quality of the test materials: are items, instructions and scoring standardised, and is the content free of items that could offend particular groups?
• Quality of the manual: does it give you what you need to administer the test and interpret the scores correctly, including the limits of that interpretation?
• Norms: are the norm groups large enough, representative of the people you compare candidates with, and recent? A norm table without a date or a description of the group cannot be judged.
• Reliability: how precise are the scores, according to which type of study and which sample?
• Construct validity: is there evidence that the test measures what it claims to measure, for example from its factor structure and its relations with other measures?
• Criterion validity: do scores relate to outcomes such as job performance, training results or turnover, and how well was that outcome measured?

An insufficient rating can mean that the evidence was weak or that it was never provided. COTAN treats both alike, because the burden of proof lies with the test author. For you the difference can matter: a new test with little data is not the same as a test that fell short. Either way it is a warning sign, and if you use the test anyway you should be able to explain why.

COTAN ratings are given only by COTAN itself and are published in the online COTAN Documentatie, which Boom uitgevers publishes on behalf of the NIP. If a vendor mentions a COTAN rating, look it up there and check which version of the test and manual it covers.

The EFPA Test Review Model and related frameworks

EFPA's Test Review Model is a structured review form with notes for reviewers, based on criteria from the British Psychological Society, the Spanish psychological association and COTAN. Version 4.2.6 dates from 2013; version 5.0, a major revision, was published in August 2025.

The model first describes the instrument and then evaluates the rationale and documentation, test materials, norms, reliability, validity, support for fair use and the quality of digitally generated reports, ending in an overall evaluation. Validity evidence is grouped by source: test content, internal structure, relations with other variables and other sources.

The model is meant to be adapted to national arrangements and applied by expert reviewers; associations such as the British Psychological Society have based their test reviews on it. EFPA states that ratings given with the model do not imply its endorsement of a test, and that advertising must not suggest otherwise. A vendor that calls its test approved by EFPA has misread the model.

DIN 33430 in Germany

DIN 33430, the German standard for job-related aptitude assessment, was last revised in 2016. It does not rate individual tests but covers the whole process: clarifying the assignment, analysing the requirements of the role, choosing methods, administration and interpretation, documentation, and the qualifications of the people involved. An annex sets out what the documentation of a psychometric test should contain, which makes it a useful checklist outside Germany too.

A checklist for reading a technical manual

Whichever framework you use, the evidence is in the technical manual. Ask for it before you buy, and read it with your own decision in mind.

• Intended use: does the manual name your use, and does it say what the test is not meant for?
• Norm groups: size, composition, country, language and year of data collection. Do they resemble your candidates? See how norm groups work.
• Reliability: which type, from which sample, and is the standard error of measurement reported? The basics are in reliability and validity explained.
• Validity: is there evidence for the factor structure, for relations with comparable tests and, for selection, for a relation with job-related outcomes? How large were the samples?
• Fairness and languages: do items and scales work the same way across groups, and was your language version adapted and studied, not only translated?
• Version and independent review: does the evidence cover the version you will use, and is there a review you can look up yourself?

How to read it. A reliability of .86 is solid, but for an important decision about an individual it sits between COTAN's anchor points of .80 and .90, so look at the standard error of measurement and avoid hard cut-offs on small score differences. The norm group fits finance roles in the Netherlands, not sales roles or a German version. The criterion study points the right way, but with 85 people the confidence interval is wide. Conclusion: usable in selection alongside other information, and a person makes the decision.

When you develop or publish a test yourself

The same criteria work as a design brief: define the construct and the intended use, try out items, and collect data on reliability, structure and norms in the target population, as described under test development. EFPA explicitly encourages developers and publishers to evaluate their own tests with its model. That gives you a structured self-assessment, not a review: a COTAN rating is given only by COTAN, through its own procedure.

The Psychometric Lab at Match.hr reviews tests before they are listed and can work with you on test validation. The Lab's criteria follow the structure of the COTAN and EFPA review models. The Lab is not affiliated with or endorsed by either organisation, and a review level such as Validated is not an official rating. If you want to offer your instrument to employers, publishing your test starts with that review.