Is there a difference? More than we usually realize.
Nassim Nicholas Taleb describes the issue nicely in The Black Swan
The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship upon them. Explanations bind facts together. They make them all the more easily remembered; they help them make more sense. Where this propensity can go wrong is when it increases our impression of understanding.
We are primed for making sense of the world, to see patterns in nature and in data. Success often depends on accurate anticipation of what will happen next. We look for patterns in things to use as a means of forecasting. But sometimes that beneficial habit betrays us. We see patterns where there are none.
Kaiser Fung, a statistician, makes the observation that in most news articles, there is a point where the reporter transitions from reporting the story to telling a story. They shift from the agreed facts to an imputation of cause that is not there. It is the point where we transition from facts to speculation and it is important to distinguish the two.
Fung offers an example from a Los Angeles Times article, Who’s teaching L.A.’s kids? The article starts out reporting the results from “a statistical approach known as value-added analysis, which rates teachers based on their students’ progress on standardized tests from year to year.” For example, the first finding reported is:
Highly effective teachers routinely propel students from below grade level to advanced in a single year. There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.
After a series of factual findings, the reporter then transitions.
Nevertheless, value-added analysis offers the closest thing available to an objective assessment of teachers. And it might help in resolving the greater mystery of what makes for effective teaching, and whether such skills can be taught.
On visits to the classrooms of more than 50 elementary school teachers in Los Angeles, Times reporters found that the most effective instructors differed widely in style and personality. Perhaps not surprisingly, they shared a tendency to be strict, maintain high standards and encourage critical thinking.
But the surest sign of a teacher’s effectiveness was the engagement of his or her students — something that often was obvious from the expressions on their faces.
The transition is subtle and not obvious unless you are looking for it. In a matter of column inches we have gone from the certainty of “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%” (an objective and empirical fact), to the speculation that “the surest sign of a teacher’s effectiveness was the engagement of his or her students.” The performance gap is measured empirically and objectively but the proposed cause of the gap (engagement) is not. Instead the reporter connects outcomes to a cause which he attributes to engagement which he can ascertain from “the expressions on their faces.” We have moved away from the factual story to the entertainment of speculative storytelling.
It seems obvious when pointed out but people most often go with the flow without recognizing the phase change. It is important to acknowledge that the reporter might be correct, that the teacher’s capacity to engage students might indeed be the cause of their improved results. But that is an undocumented and unproven assumption which we need to treat differently from the documented facts.
Accurate facts are the lifeblood of good decision-making. Without them, everything is faith and hope. Not bad things in themselves but as the adage has it, Hope is not a strategy.
Most routine problems arise not because we don’t know what needs to be done but because we don’t do what we know needs to be done. Ignorance of the facts is not at fault but will. And so we tell stories. We link the facts together in a sequence from beginning to end, from A leading to Z, each set of facts causing the next action. Tied together the individual facts begin to create a sense of inevitability, increasing our confidence and our willingness to act.
But a well told story is not the same thing as an accurate story. An accurate story means we know a series of facts AND we know the causal relationship between those facts and the final outcome we are trying to attain. Most often what we know are some select facts but then we fill in the blanks about causes with what makes most sense to us. These unacknowledged assumptions are often the genesis of both failure and unintended consequences.
Those filled-in blanks covering missing facts and unknown causations are assumptions that undermine our decision-making. Some facts simply can’t be known and we do have to make assumptions. But we have to do it consciously with a plan to mitigate the consequences if our assumption turns out to be wrong. It is the unrecognized assumption, usually created in the storytelling process, which ends up sinking the ship.
The most common form of error is the classical logical fallacy of post hoc ergo propter hoc, (after this, therefore because of this). We see one thing correlated with another and we leap to the conclusion that the one thing caused the other. You go into a wealthy neighborhood and see many luxury cars. You assume that luxury cars must cause wealth. Put that way, it is patently absurd but that is the logic that underpinned US housing policy in the latter two decades of the 20th century and which ended so disastrously in 2008. The argument was in part that homeownership ought to be encouraged through relaxed mortgage requirements and other programs because many positive social outcomes (mortality, morbidity, education attainment, lower teen pregnancy, higher savings rates, higher incomes, etc.) were correlated with home ownership. It was assumed that home ownership fostered desirable traits such as diligence, planning, and responsibility. If you increased home ownership, you would increase those traits and therefore would correspondingly increase life expectancy, education attainment, etc.
In hindsight, the error is obvious; the traits themselves facilitate both homeownership and the other desirable life outcomes. Classic post hoc ergo propter hoc.
Once you know the facts and have an acceptable level of confidence in how the facts are linked together to yield the outcome you are seeking, you then have a basis for a good decision.
Getting people to accept that decision is in part a storytelling exercise. People respond to stories. Inspired storytelling that follows established facts is likely to yield good outcomes. Inspired storytelling in the absence of facts or despite facts is a recipe for disaster.
So when someone comes to you with a proposition, a proposal or an argument, look for the hinge point where they shift from telling the story to storytelling, where they depart from facts and venture into the realm of assumptions and speculation. Likewise, when presenting a recommendation to a diverse audience, ask yourself how grounded in facts is your argument, and how much are you asking them to take on the faith of a shared assumption. The more diverse the audience, the greater the number of different stakeholders, the more certain it is that someone will call you on your own assumptions.
Decision Clarity Consulting tools and methodologies help distinguish convincing storytelling from convincing facts and help bring the two into alignment with one another.
Posted in Examples, Featured | No Comments »
