...helping clients in complex environments make better decisions, faster.

CALL NOW: 404.898.9096


Posts Tagged ‘Outcomes’

Measuring expert opinion

The value and dangers of forecasting.  Everybody’s An Expert: Putting predictions to the test by Louis Menand.

Well documented and researched.


There is a correct answer to that question, but it’s unlikely we’ll ever know what it was.

From Why Do Education and Health Care Cost So Much? by Megan McArdle.  A great example of the challenges related to causal density.  We may accurately identify all the causes of an outcome but still not be able, because of poor understanding of the relationships between root causes, to predict outcomes.  Absent accurate prediction, we don’t really understand the nature of a problem at all.

So how do we explain health care and college cost inflation? Well, health care economist David Cutler once offered me the following observation: In health care, as in education, the output is very important, and impossible to measure accurately. Two 65-year-olds check into two hospitals with pneumonia; one lives, one dies. Was the difference in the medical care, or their constitutions, or the bacteria that infected them? There is a correct answer to that question, but it’s unlikely we’ll ever know what it was.

Similarly, two students go to different colleges; one flunks out, while the other gets a Rhodes Scholarship. Is one school better, or is one student? You can’t even answer these questions by aggregating data; better schools may attract better students. Even when you control for income and parental education, you’re left with what researchers call “omitted variable bias” — a better school may attract more motivated and education-oriented parents to enroll their kids there.

So on the one hand, we have two inelastic goods with a high perceived need; and on the other hand, you have no way to measure quality of output. The result is that we keep increasing the inputs: the expensive professors and doctors and research and facilities.

I would quibble with McArdle.  There are actually two problems.  It is true that it is hard to measure education and health outcomes and that is a challenge.  But even if we were able to measure with great precision and accuracy, that is still not the same as forecasting.  Measuring is a predicate to forecasting.

If we precisely and accurately measure our initiating action X, we want to know with some level of accuracy and certainty that X will lead to Y, the outcome we desire.  If we cannot predict the outcome, it means we don’t understand the relationship between and among the various causes.

Decision Clarity Consulting has no affiliation with the Scheier Group, owner of the DECISION CLARITY®, and the Scheier Group does not sponsor, endorse, or control any material on this site. Decision Clarity Consulting, in accordance with an agreement with Scheier Group, performs no work with non-profit or government entities.