Hmmmm ....... makes sense to me!
Proof Negative
Posted on August 10, 2014
DisconfirmationI have regularly argued that in investing, as in most things in life, disconfirmation is more valuable than confirmation (see here, for example). In other words, we learn more from what doesn’t work than from what does. That’s largely because induction is the way science advances.
We want deductive proof, but have to settle for induction. That’s because science never fully proves anything. It analyzes the available data and, when the force of the data is strong enough, it makes tentative conclusions. But these conclusions are always subject to modification or even outright rejection based upon further evidence gathering. The great value of data is not so much that it points toward the correct conclusion (even though it does), but that it allows us the ability to show that some things are conclusively wrong.
In other words, confirming evidence adds to the inductive case but doesn’t prove anything conclusively. Correlation is not causation and all that. Thus disconfirming evidence is immensely (and far more) valuable. It allows us conclusively to eliminate some ideas, approaches or hypotheses.
That said, we don’t like disconfirming evidence and we tend to neglect the limits of induction. Few papers get published establishing that something doesn’t work. Instead, we tend to spend the bulk of our time looking (and data-mining) for an approach that seems to work or even for evidence we can use to support our preconceived notions.
We should be spending much more of our time focused upon a search for disconfirming evidence for what we think (there are excellent behavioral reasons for doing so too). But we don’t, as illustrated by the following question (a variation of the Wason selection task).
Most people answer with E and 4, but that’s wrong. For the posited statement to be true, the E card must have an even number on the other side of it and the 7 card must have a consonant on the other side. It doesn’t matter what’s on the other side of the 4 card. But we turn the 4 card over because we intuitively want confirming evidence. And we don’t think to turn over the 7 card because we tend not to look for disconfirming evidence, even when it would be “proof negative” that the given hypothesis is incorrect. In a variety of test environments, fewer than 10 percent of people get the right answer to this type of question.
I suspect that this cognitive failing is a natural result of our constant search for meaning in an environment where noise is everywhere and signal vanishingly difficult to detect. Randomness is difficult for us to deal with. We are meaning-makers at every level and in nearly every situation. Yet, as I have noted often and as my masthead proclaims, information is cheap while meaning is expensive and elusive. Therefore, we tend to short-circuit good process to get to the end result – typically and not so coincidentally the result we wanted all along.
As noted above, science progresses not via verification (which can only be inferred) but by falsification (which, if established and itself verified, provides relative certainty only as to what is not true). Thank you, Karl Popper. In the investment world, as in science generally, we need to build our investment processes from the ground up, with hypotheses offered only after a careful analysis of all relevant facts and tentatively held only to the extent the facts and data allow. Accordingly, we need always to be on the look-out for disconfirming evidence — proof negative — even though doing so is oh so counterintuitive pretty much all the time.
---------------------------
A great little article.
I picked K & 7 to attempt to disprove the statement!
I reckon the last sentence is on the money!
I don't know that we have any positive evidence upon which to make conclusive deductions re commerciality (according to technical experts in our ranks) and specifically we don't have any disconfirming
evidence at this stage re Cambay 77H. So far all is
un-negative (not positive! just not negative), until disconfirmed by evidence either way.
Also, I know when I'm out of my depth - but, I think it would be erroneous to attempt to apply probability and/or inference (or deduction) to any disconfirming evidence/falsification proofs, even if OEX had inadvertantly provided such data.
While disconfirmation can eliminate one possible outcome with certainty it does not infer anything beyond that. As OEX has provided no evidence that can even be used for anything, let alone disconfirmation, any attempt to move beyond that would result in speculations more dangerous than wild guesses.
With O&Gs we typically just don't know enough to be certain about anything - maybe even after we get an apparent conclusive statement re commerciality. I think we'll still be using the equivalent of weather forecasting terms e.g. a temperature of 30c with an 80% chance of 20 mm rain early afternoon resulting in 85% humidity and perceived temp of 39c - variables built on variables and non-linear algorithms.
Just trying to be helpful.
