Testing natural experiments is a fine idea that all social sciences embraced decades ago. It is bizarre and inaccurate for Chetty to claim that it is novel and that it produces "science." It can help and it can hurt depending on how well it is conducted. It is one important methodology, not the holy grail of scientific methodology for economics or criminology.
I will discuss several examples of why Chetty's "scientific experiments" have repeatedly produced what economists assured us were "compelling answers to specific policy questions" that were disastrously wrong. It turns out that the scientific method that economists purport to embrace is frequently the thinnest of fancy veneers hiding a core composed of the cheapest pressboard. Economists who study fields beset by financial frauds, for example, are overwhelmingly betrayed by ideology and conflicts of interest.
Economists do not study fraud. They have a primitive tribal taboo against using the word. This, of course, is because economics is assuredly not "firmly grounded in fact." Ignoring fraud is a pure ideological construct that requires economists to ignore fraud, particularly private-sector "control fraud." Economists do not study the criminology literature on elite white-collar crimes. Economists do not study and do not understand sophisticated financial fraud schemes.
I will briefly mention five examples during the savings and loan debacle. Richard Pratt, an academic economist who was Chairman of the Federal Home Loan Bank Board, exploited a series of natural experiments to study which state-chartered S&Ls reported the highest income. The answer was Texas. Pratt, the architect of the deregulation bill that became law as the Garn-St Germain Act of 1982, used Texas' deregulation law as the model for the Garn-St Germain Act.
Alan Greenspan, having studied the natural experiment provided by S&Ls following different investment strategies praised Charles Keating's Lincoln Savings as the exemplar that should be the model for the industry because Lincoln Savings reported record profits. Greenspan assured the federal regulators that Lincoln Savings "posed no foreseeable risk of loss."
Daniel Fischel, exploiting a similar natural experiment, praised Lincoln Savings as the Nation's best S&L. He also praised CenTrust as a superb S&L. In each case the basis for his conclusion was the extreme income reported by the S&L.
George Benston used a natural experiment by studying state-chartered S&Ls that had made large amounts of "direct investments" that the federal regulators were proposing to restrict. He found that such S&Ls reported substantially higher income than other S&Ls.
James Barth studied a natural experiment provided by failed S&Ls and concluded that their owners must have been honestly gambling for resurrection because as the S&Ls approached failure they increasingly invested in high risk assets. Barth was the regulatory agency's head economist.
The common denominators in these five examples are that the economists implicitly assumed accounting control fraud out of existence and as a result their conclusions were false. The "recipe" for accounting control fraud by a lender has four ingredients.
- Grow extremely rapidly by
- Making enormous numbers of bad quality loans at a premium yield while
- Employing extreme leverage and
- Providing only trivial allowances for loan and lease losses (ALLL)
The recipe produces three "sure things." The lender is guaranteed to report record income in the near term, the senior officers will be made immediately wealthy by modern executive compensation, and the lender will eventually suffer severe losses. Texas led the Nation in accounting control fraud because it deregulated and desupervised, so its S&Ls reported the highest (fictional) profits and suffered the worst losses. Pratt's use of Texas as the model for deregulation was the worst possible choice and caused catastrophic harm.
Greenspan's assurance that Lincoln Savings posed no foreseeable risk of loss proved incorrect -- it caused the largest loss to a federal deposit insurance fund in history. It was impossible for Greenspan to make a larger error when writing about a single institution.
Fischel praised the worst, most fraudulent S&L in the Nation as the best S&L. He made a 3,000 position error in an industry with 3,000 positions. It is impossible to get it more wrong. CenTrust was also a massive accounting control fraud that caused roughly $1 billion in losses.
Benston praised roughly 33 S&Ls that made large amounts of direct investments -- they all failed. They were overwhelmingly accounting control frauds. It is impossible in a sample size of 33 to bat worse than 0 for 33. Greenspan, Fischel, and Benston shared another characteristic that helps explain the astonishing extent of their errors -- Lincoln Savings paid for their research.
Barth did agree that control fraud existed, but he missed the fact that firms following the fraud recipe will invest heavily in high risk assets but they will do so in a manner that demonstrates that they are engaged in accounting control fraud rather than honest gambles. George Akerlof and Paul Romer's 1993 article ("Looting: The Economic Underworld of Bankruptcy for Profit") explained this point in detail.
"[M]any economists still [do] not understand that a combination of circumstances in the 1980s made it very easy to loot a [bank] with little risk of prosecution. Once this is clear, it becomes obvious that high-risk strategies that would pay off only in some states of the world were only for the timid. Why abuse the system to pursue a gamble that might pay off when you can exploit a sure thing with little risk of prosecution?"
[S]omeone who is gambling that his thrift might actually make a profit would never operate the way many thrifts did, with total disregard for even the most basic principles of lending: maintaining reasonable documentation about loans, protecting against external fraud and abuse, verifying information on loan applications, even bothering to have borrowers fill out loan applications" (Akerlof & Romer 1993: 4-5).