Klingon Warships

You need not have attended a Star Trek Convention to remember the dreaded Klingons. Here is a little Klingon trivia I have been thinking about lately: Faced with impending attack and nowhere to run, Klingon warriors engaged a cloaking device rendering their spacecraft invisible. When you're the most reviled being in the galaxy, that's a pretty good tool to have, isn't it?  

Sometimes I think our mental models, our maps of the world and ourselves, are like Klingon warships. Invested in our livelihood and the stories we tell ourselves, our models are self-protective. In their drive for preservation, our stories and models trump even scientific fact. I offer one example from Nobel-prize winning economist, Daniel Kahneman's 2011 book, Thinking, Fast and Slow.

Several years ago, Kahneman consulted with a prestigious group of investment advisors whose clients were among the wealthiest people in the United States. In the course of his work, Kahneman acquired a spreadsheet summarizing the investment outcomes of 25 anonymous advisors, for each of eight consecutive years. Note that advisors' year-end bonuses depended primarily on their outcome score.

To determine whether there were persistent differences in skill among the advisors and whether the same advisors consistently achieved better returns for their clients, Kahneman ranked the advisors by their performance in each year and made the appropriate comparisons. He then computed correlation coefficient between the rankings in each pair of years: year 1 with year 2, year 1 with year 3, and so on up through year 7 with year 8. The result was 28 correlation coefficients, one for each pair of years.

What happened next was not a shock to Kahneman whose theories about the persistence of skill foretold the outcome. Are you ready? The average of the 28 correlation coefficients was .01. In other words, there was no relationship between investing skill and portfolio performance. It was impossible to predict the former from the latter.

"Our message to the executives was that, at least when it came to building portfolios, the firm was rewarding luck as if it were skill. This should have been shocking news to them, but it was not. There was no sign that they disbelieved us. How could they? After all, we had analyzed their own results, and they were sophisticated enough to see the implications, which we politely refrained from spelling out. We all went on calmly with our dinner, and I have no doubt that our findings and their implications were quickly swept under the rug and that life in the firm went on just as before" (pp. 216).

Kahneman explains our tendency to excuse statistical fact from individual cognition as an artifact of System 1 thinking, that is, our drive to create a story based on often-faulty memory to give meaning to otherwise unpredictable, disconnected, and therefore psychologically untenable circumstances. Over time, these stories become, and then reinforce, our mental models. And like Klingon warships, those models cloak themselves in invisibility, protecting themselves from existential threat.

So what is it that educational leaders need to know that their mental models have rendered invisible? What maps need to be redrawn? Clearly, throwing out extant models and leaving everything to chance is foolish. We can indeed draw a clear and convincing arrow from degrees of skill and varying outcomes in many things. Playing a musical instrument, shooting a basketball, and driving a car come to mind. If you don't believe me, experience a middle-school band concert; then go to a concert performed by a professional symphony.

Kahneman's work does suggest, however, that in highly-skilled professions like developing school principals and teachers and educating children, critical contributors to outcomes may not be as apparent as one might think. And then there is the matter of chance.

Since focusing on chance and things over which we have no control gets us nowhere, we can and must think strategically about the educational outcomes we seek, as well as on how we intend to get them. From our thinking, including the results of correlational research, we will build better models; but first we must see the models. What Klingon warship will you uncloak today?