Sunday, September 9, 2007

The Delphi Method

Here's a another method for trying to see into the future by the systematic pooling of insights and ideas from a group of experts. Developed by RAND in the 1950s. A good discussion is contained in this article on the future of netcrime. The idea, in essence, is to have structured questioning, anonymous responses (to let information and opinion flow freely), and a system of iteration and feedback. As Sheridan Morris of the UK Home Office puts it in the paper,

...a Delphi approach may also be combined with other futures techniques such as the use of scenarios. The Delphi technique may be found in areas where there is an absence of sufficient data and/or an incomplete theory on cause and effect in regard to the area under study. Sitting between knowledge and speculation, the outcome of the panel may be considered informed judgement.


This is an interesting idea...and not unlike some notions from physics. Even if complete and specific prediction is not possible, qualitative prediction may be possible.

of course, one might compare this with the various futures markets that pool information to make predictions, apparently with considerable accuracy.

Saturday, September 8, 2007

The dismal science

You must have a chapter on the appalling science of economics, and how it has failed forever to make decent predictions. A good starting place is this post from Leiter Reports. He's a lawyer or philosopher and has a lot to say, with many links, to deep criticisms of the field, its scientific status, and its ability or inability to make predictions. A couple selections:

As fate would have it, an economist has been posting on the topic du jour--the scientific status of economics: see Tyler Cowen here and here. Professor Cowen's perspective on this question (rather typical of economists, I fear) is well-expressed by a colleague of mine:

"I guess the reason that I think economics is a science is that empirical testing is a huge part of economics. I.e., if economics were only about the mathematical models, without falsifiable claims, I would agree it's not scientific. But economics makes falsifiable claims all the time and tests them frequently. And some are confirmed, repeatedly, and they become accepted wisdom. Others are falsified, and they fall by the wayside. Isn't that what science is all about?"

This isn't, however, what "science is all about" on any plausible account. Pest control, for example, would be a science on this account (exterminators operate with evidentiary hypotheses ["ah-ah, tell-tale rat droppings!"], which they test ["since it's rats, we'll lay this kind of poison"], and which are sometimes falsified ["well, I'll be damned, it turns out it's not rats, but field mice"]), which doesn't seem the right conclusion. And lots of paradigmatic scientific propositions ("there are black holes," "there are quantum singularities") wouldn't be "scientific", because they aren't falsifiable (I owe the examples to Larry Laudan). Some philosophers of science go further, and argue that no claims are falsifiable (on evidentiary or logical grounds) because of the underdetermination of theory by evidence (the "Duhem-Quine" thesis as it is known) (Laudan has interesting arguments against this point--a nice presentation is in the so-titled chapter on undeterdetermination in his Science and Relativism [Chicago, 1990], which is still the best introduction to the subject I've read.)


and

Often, modern economic analyses of law turn out to be remarkably indifferent to the empirical facts. Even predictions generated from the Coase Theorem--the cornerstone of modern law-and-economics--have been empirically falsified! (See Robert Ellickson, Order Without Law: How Neighbors Settle Disputes (Cambridge, Mass: Harvard University Press, 1991)). The Coase Theorem says that "if there are no transaction costs to impede bargaining, legal rights will be allocated efficiently through private exchanges, regardless of the underlying rule of law." Robert Cooter, "Against Legal Centrism," California Law Review 81 (1993), pp. 417-429, at 419 n. 6. As Cooter explains: “Liability rules can be structured in different ways. With cattle, for example, the law can make owners of cows responsible for fencing them in ("closed range") or non-owners responsible for fencing them out ("open range"). Coase reasoned that owners will fence cows in or non-owners will fence them out, whichever is cheaper, regardless of the law. . ..Ranchers need only bargain together. . ..The law simply tilts bargaining against the party who is legally liable. Thus the law affects who pays for the fence ("distribution prediction"), but not its extent or location ("efficency prediction"). Id. at 419. What Ellickson found is that neither prediction was borne out among real ranchers that Ellickson studied in Northern California. Cooter notes: "Not only does the Coase Theorem fare poorly in empirical tests, but theorists who have tried to prove it mathematically usually conclude that it is either false or a tautology." Id. at 422. None of this, remarkably, has undermined its importance for the "science" of economics!

Philosophers, too, have recently launched a devastating attack on the scientific and cognitive credentials of economics, starting from the observation that, "[E]conomic theory [is] one of the more dismal empirical failures in the history of science" (John Dupre). This is widely conceded about the laughably unsuccessful predictions of macroeconomics, but it is only somewhat less true of microeconomics, which "has made no advances in the management of economic processes since its current formalism was first elaborated in the nineteenth century" (Alexander Rosenberg, "If Economics Isn't Science, What Is It?" reprinted in D. Hausman (ed.), The Philosophy of Economics: An Anthology, 2nd ed. (Cambridge: Cambridge University Press, 1994), p. 377. Cited hereafter in the text by page number only as Rosenberg II. See also, Wassily Leontief, "Input-Output Economics," Scientific American 185 (1951), pp. 15-21.) This last point bears elaboration.