Skilled, unskilled, unaware

A 1999 paper by Kruger and Dunning, Unskilled and unaware of it, has recently been receiving attention. In summary, they tested people on a few tasks while measuring their self-assessments, and found that the worst performers consistently rated themselves above average in the tested tasks. They attributed this effect to a deficiency of metacognitive skill in the worst performers, a conclusion that found resonance with many readers - including me at the time.

But Robin Hanson points to Burson, Larrick and Klayman's 2006 paper which reinterprets Kruger and Dunning's work. They argue that the issue at work is not a metacognitive deficiency, but effectively that people tend to estimate their absolute performance in a task (how well they'll do on an objective performance metric), and do not compensate for that the task is either easy or difficult for everyone. Estimating absolute performance requires awareness of oneself and the task, whereas estimating relative performance requires awareness of everyone else as well. This latter part is difficult.

It turns out that everyone is bad at estimating other people's abilities. The worst performers tend to overestimate their relative ability for easy tasks, but tend to be more accurate for hard tasks. The best performs tend to underestimate their relative ability for hard tasks, but tend to be more accurate for easy tasks. In tasks of medium difficulty, the overall bias is lowest.

It appears, therefore, that there is no reason to think that there's a metacognitive deficiency in poor performers. It's just that when asked for a relative performance comparison, everyone starts with an absolute assessment of their performance, and then fails to compensate for everyone else's skills.

Which is understandable. The only people who really know about everyone's relative skills in a particular task is those who administer tests and see everyone's results.


Popular posts from this blog

When monospace fonts aren't: The Unicode character width nightmare

Circumcision as an adult, part 2

Circumcision as an adult, part 1