How Much Leeway Should Bureaucrats Have to Explore?
Can Government Experiment As Does Science?
| A new book by Jim Manzi argues that more randomized control trials of government programs could measurably improve public policy outcomes. This 2012 article is from The Atlantic, May 14. |
by Conor Friedersdorf
To understand the argument that entrepreneur Jim Manzi makes in Uncontrolled: The Surprising Payoff of Trial-and-Error For Business, Politics, and Society, it's useful to forget everything you believe about freedom. It is completely irrelevant to a less familiar, more utilitarian argument about why this country's citizens ought to be afforded freedom.This argument holds that ignorance is inevitable in a world as complicated as ours -- that it's impossible for anyone to be certain of the best way forward at a given moment. But a free society permits individuals to try different ways of living. Successes are imitated. Customs develop around what works. The things that don't are abandoned or made taboo. And so long as there's always an ability to depart from custom or challenge taboos, important innovations keep happening. In this telling, the freedom to engage in trial-and-error is indispensable to progress.
These insights are core to the classical liberal tradition and its skepticism about the ability of policymakers to innovate via central planning. Uncontrolled’s author makes a compelling argument for an ideologically neutral method for improving policy, one that left and right might both embrace, even as it challenges both sides to rethink some of their reflexes.
The first part of Uncontrolled is an enjoyable summary of how philosophers, scientists, medical researchers, and corporations all discovered -- at different times and places -- that conducting lots of rigorous experiments is the most reliable way to figure out what works and what doesn't work. As Manzi takes pains to demonstrate, the gold standard in experimentation is the double-blind, randomized field trial with a control group. Fortunately, technology has made conducting these sorts of experiments more cost effective than ever.
That's partly due to their limits. "The reason we have increasing trouble building compact and comprehensive predictive theories as we go from physics to biology to social science is the increasing complexity of the phenomena under investigation," Manzi writes, which also makes it "far harder to generalize the results of experiments."
We can run a clinical trial in Norfolk, Virginia, and conclude with tolerable reliability that "Vaccine X prevents disease Y." We can't conclude that if literacy program X works in Norfolk, then it will work everywhere. The real predictive rule is usually closer to something like "Literacy program X is effective for children in urban areas, and who have the following range of incomes and prior test scores, when the following alternatives are not available in the school district, and the teachers have the following qualifications, and overall economic conditions in the district are within the following range." And by the way, even this predictive rule stops working ten years from now, when different background conditions obtain in the society.
But that's no reason to give up. Thus the book's suggestion: conduct lots of rigorous experiments to test the effectiveness of every new policy that's implemented. Make that rigor a reflexive part of government culture.
“For the hypothetical literacy program described above, an experiment to test the program is not really a test of the program; it is a test of how well the program applies to a specific situation.
A brute-force approach to this problem would be to run not one experiment to evaluate whether this program works, but to run hundreds or thousands of experiments to evaluate the conditions under which it works. If it could be tested in a very large number of school districts, we might very well discover some useful approximation to the highly conditional rule that predicts its success. This is the opposite of elegant theory-building... But it might provide practically useful information.
Confident that its now plausible to do lots of large-scale, cost-effective policy experiments, Manzi wants the federal government to create an agency charged with running and refereeing them. And he wants states to enjoy greater freedom to experiment, if they agree to rigorously test the efficacy of everything they do, making the results transparent so best practices can spread.
His claim is that this can modestly improve policy at the margins.
There's a lot more to this book, which has been justly praised in every review I've read, partly because its the rare political book that goes out of its way to raise the most powerful objections to its arguments and to point out the limits of the reform program that it recommends. In a country where the left is sometimes reflexively confident in the ability of social scientists to model effective policy, and the right sometimes reacts to policy failures with reflexive do-nothing nihilism, a non-ideological method of distinguishing between good and bad policy is appealing.
Does anyone in power have sufficient incentive to implement such a system?
JJS: The above idea seems worth a shot, at least while government delivers so many programs. But the above idea might not even be necessary if government were not so involved in our lives but instead simply paid everyone a dividend from surplus public revenue, from all the recovered rents, which are all the spending society does for using or owning nature, for land, nature, EM spectrum, ecosystem, etc. Then students of society might finally find real natural laws.
---------------------
Editor Jeffery J. Smith runs the Forum on Geonomics and helped prepare a course for the UN on geonomics. To take the “Land Rights” course, click here .
|
Also see: An international panel declares ...
High Reward From Targeting So-called High Risk
A Bulletin on Wasteful Government Spending |
| Email this article | Sign up for free Progress Report updates via email |
What are your views? Share your opinions with The Progress Report:
| Page One | Page Two | Archive |
| Discussion Room | Letters | What's Geoism? |
