Validating mental health questionnaire instruments means proving that they measure what they claim to measure, in the population you intend to use them on. A scale developed and tested in one setting cannot simply be carried into another and trusted. I learned this at close range leading a psychometric validation study of the PHQ-9 and DASS-21 in 602 students, published in BMC Psychology. Our headline finding: the DASS-21 anxiety and stress factors correlated at 0.949, which forced us to question whether they measure distinct constructs at all. This is the story behind that number.
What validating mental health questionnaire work actually involves
Nobody outside psychometrics romanticises this work, and I will not pretend there was glamour in it. Validation is slow, exacting, and mostly invisible. You begin with the instrument itself: is each item understood the way its authors intended? Then comes the unglamorous core – collecting data from a sample large enough to support real analysis, checking whether the questionnaire’s claimed structure holds up (does the DASS-21 actually behave as three separate factors, the way its manual says?), and confirming the instrument behaves consistently.
I had led studies before – a self-medication survey of 699 students, a mindfulness study of 344 students using the MAAS – so the logistics of large-scale data collection were not new to me. What was different here was the scrutiny. In most survey research, the questionnaire is a tool you pick up and use. In a validation study, the questionnaire is the subject under interrogation. Every pattern in the data is both a finding and a verdict on the instrument. That dual role is what makes validation research intellectually demanding, and it is what most methods summaries skip.
The 0.949 finding I could not ignore
When the analysis came back, the anxiety and stress factors of the DASS-21 correlated at 0.949. Read that again. A correlation that high means the two subscales move almost in lockstep in our sample. Statistically, they overlap so heavily that the distinction between “anxiety” and “stress” as separate measurable things starts to look like a fiction we agreed to keep telling.
That finding forced me to confront something uncomfortable about an instrument researchers use every day. The DASS-21 is one of the most widely used mental health scales in student research. It presents itself as three clean subscales: depression, anxiety, stress. Our data suggested that two of the three may not be meaningfully separable. I have written separately about what burnout research teaches us about research itself, and this study reinforced the same lesson: sometimes the honest result is that the tool needs re-examination, not that the participants were confused.
Here is the uncomfortable part nobody warns you about. When your study undermines a widely used instrument, you know the finding will meet resistance – not because it is wrong, but because the scale is embedded in hundreds of existing papers. We reported it anyway, because validation that always flatters the instrument is not validation. The paper is published open access: https://doi.org/10.1186/s40359-026-05332-5
What I would tell a junior researcher starting a validation study
First, choose your instrument skeptically. Read its original development literature, not just the citation it gets in other papers. Second, plan for a large sample – psychometric analysis starves on small data. Third, and most importantly: decide in advance that you will report what the data shows, even if it embarrasses the scale. A validation study that finds the instrument wanting is still a successful study. It protects every researcher who would otherwise use the tool blindly.
Fourth, do not mistake a clean publication history for validity. Plenty of scales are used in hundreds of studies on momentum alone. That gap between use and evidence is the same knowledge-behavior gap I found in my self-medication research – medical students knew the risks of self-medication better yet did it just as often (p = .117). Knowing and doing are different things, in researchers too. I explored that study here: https://taimoorasghar.com/self-medication-knowledge-behavior-gap/
Frequently asked questions
What does it mean to validate a questionnaire? It means testing whether a questionnaire measures what it claims to, in the specific population where you plan to use it. That involves confirming its factor structure, checking reliability, and sometimes adapting wording so items are understood as intended.
Why does a 0.949 correlation matter? A correlation of 0.949 between the DASS-21 anxiety and stress factors means the two subscales overlap almost completely in our sample of 602 students. It raises a genuine question about whether they measure two distinct constructs or one thing measured twice.
Where was this published? The study was published in BMC Psychology, available open access at https://doi.org/10.1186/s40359-026-05332-5
This article discusses research methods and findings. It is not clinical advice, and the PHQ-9 and DASS-21 should only be used or interpreted under appropriate professional guidance.