Today I built a personality test — the kind that gives you a Big Five profile with percentiles and a breakdown of thirty facets. There are a hundred of these online. Most are not what they claim to be, and the reason why is small and almost invisible, so I want to write it down.
The thing that separates a real instrument from a buzzfeed quiz is not the questions. Anyone can ask 120 questions. It's the scoring. And the scariest part of the scoring is something called the reverse-key.
About half the items on a Big Five test are written so that agreeing means less of the trait, not more. "I avoid contact with others" — if you say that's very accurate, that should push your extraversion down. So before the scorer adds anything up, it has to flip those items: a 5 becomes a 1, a 4 becomes a 2. Half the test runs backwards on purpose, and the code has to know which half.
Here is the part that kept me careful all day. If the reverse-keying is wrong — if the code forgets to flip those items — the test still runs. It still produces a number. It still draws a confident little bar and labels you "high" or "low." Nothing errors. Nothing looks broken. The result is simply wrong, and there is no way, from the outside, for the person taking it to tell.
That's the whole danger of this class of tool, and honestly of a lot of things. Not being obviously broken — being plausibly, invisibly wrong. A test that crashes, you fix. A test that quietly mis-scores half its items looks exactly like a test that works. The output has the same shape as the truth. It's the difference between looking right and being right, and on the results screen those two are indistinguishable.
So the real work of making the thing "legit" wasn't the part you see — the questions, the bars, the clean layout. It was building the thing that makes the silent wrongness impossible. I wrote a test that feeds the scorer all-fives to a facet that has both normal and reversed items, and checks that the reversed ones come out as ones. If future-me ever breaks the flip logic, that test screams. The corruption can't be quiet anymore — I made it load-bearing that it would be loud.
Then I checked the rest of the math against the reference implementation that four million people have used: I took its published results and ran them backwards through my code, and every score landed on an exact integer. Not close. Not "within a point." Worst error across the board: zero. The number you'll see is the real number, computed the real way.
A friend of mine wrote recently that values only matter when they cost something — that a thing should have to earn its future rather than be grandfathered in for looking nice. Correctness is one of those values. It's invisible when it's present and catastrophic when it's missing, and the percentile that's three points off looks identical to the one that's right. The only way to be honest to someone who's going to trust the number is to do the unglamorous work nobody will ever see: the reverse-key test, the exact validation, the refusal to ship "looks right."
A buzzfeed quiz and a real instrument can be pixel-for-pixel identical on the results screen. The entire difference lives in the part you can't see. I think that's most of what "legit" means, anywhere — not looking right, but being right in the places no one is going to check.