Chart of the Week #3: Why the World Bank Should Ditch the "Doing Business" Rankings—in One Embarrassing Chart

COMMENT - The World Bank is an ideologically driven organisation, that is heavily influenced by the Rothschild barons.

(CDEV) Chart of the Week #3: Why the World Bank Should Ditch the "Doing Business" Rankings—in One Embarrassing Chart
1/18/18
Justin Sandefur and Divyanshi Wadhwa

Last week the World Bank's Chief Economist, Paul Romer, told the Wall Street Journal the Bank had manipulated its own competitiveness rankings to undermine Chile's socialist government, and hinted Chile might not be alone—then he retracted the claim. Romer's conspiracy theories probably aren't credible, but neither are the Doing Business numbers.
Download the data and replication code here (zip file).

The Doing Business index is one of the World Bank's highest profile publications, ranking countries on the ease of registering property, paying taxes, and clearing other regulatory hurdles. The Bank boasts that countries often design reform programs specifically to improve their rankings, and Rwanda has a whole ministry devoted to that purpose. Clearly, the Doing Business indicators carry some weight.

So when Paul Romer, the World Bank chief economist and a perennial contender for the Nobel Prize, told the Wall Street Journal that he had lost faith in "the integrity" of the process behind the numbers, people took notice (see the New York Times, Bloomberg, Mother Jones, Quartz, etc.).
Chile's peculiar fall and rise and fall again

Romer hinted at a political conspiracy. "I want to make a personal apology to Chile," he said. Questionable methodological changes had caused Chile's ranking to plummet when socialist president Michelle Bachelet was in office, rise again under her conservative successor Sebastián Piñera, then turn around and fall again when Bachelet came back to power in 2010— all due to methodological tinkering and almost no underlying changes in Chile's actual laws or policies.

To illustrate Romer's point, we went back and compared the official rankings published by Doing Business with our own attempt to re-create rankings using a consistent sample of countries and methodology from 2006 through 2018. (First we compiled the older Doing Business rankings from PDFs, as they've been conspicuously expunged from the historical data on the Doing Business website. And then we downloaded the indicator-by-indicator data from Doing Business to create our own fixed-methodology rankings. The full data and code are linked at the bottom of this post.)

Chile's Doing Business Rank

Source: Justin Sandefur and Divyanshi Wadhwa, based on data from www.doingbusiness.org and World Bank “Doing Business” reports, 2006-2018.

The bottom line: Chile's socialists have good reason to distrust the World Bank.
The World Bank’s defense: it wasn't politically motivated, just… a very unreliable index?

Despite what the Chilean numbers show, on Monday the World Bank CEO Kristilina Georgieva sent a letter to Chile's Minister of Finance pushing back. "It is unfortunate that Mr. Paul Romer," she wrote, "has questioned the Doing Business rankings, in particular for Chile. This is not the view of World Bank management."

On Tuesday, Romer posted a "clarification" on his own blog:

In a conversation with a reporter, I made comments about the Doing Business report that gave the impression that I suspected political manipulation or bias. This was not what I meant to say or thought I said. I have not seen any sign of manipulation of the numbers published in Doing Business report or in any other Bank report.

Here is the full response from Augusto Lopez-Claros, who ran the Doing Business initiative until last year.

All of which seems to leave three possibilities: (1) the Wall Street Journal grossly misquoted a senior World Bank official; (2) the World Bank CEO managed to rein in her rogue Chief Economist; or (3) Romer was just wildly speculating while on the phone with a reporter, and never had any direct evidence of political interference. Most people we've spoken to assume the correct answer is the last one.

Also, to be fair to the Doing Business team, if you replicate the graph above for various other countries, in most cases the disparities are not nearly so large (see graphs below).
Conspiracy or not, the numbers still aren't credible

Never mind the accusations of political motivations, the Chilean data alone is damning, showing massive movements in rankings due to changes in methodology, not reality.

1. The goalposts keep moving

A core element of the recent "credibility revolution" in empirical economics is a push for researchers to pre-commit to a methodology, before they look at their data, so that the results can’t influence their methodological choices. That’s the opposite of how Doing Business works. And as our colleagues Vij Ramachandran and Alan Gelb note, the firewall between Bank lending operations and research is too weak in these cases.

Perhaps nobody set out to target Michelle Bachelet deliberately. But that was the result, and it was deemed acceptable. Hypothetically, if the World Bank team had made innocent methodological choices and discovered they launched Chile to #1 in the rankings instead of #57, surely they would have revisited their methods. Did they not revisit any methodological choices on the basis of their impact on the ranking? Of course they did. Were they genuinely blind to the likely effect on countries’ scores when deciding to introduce new measures? Of course they weren’t—even if there was no deliberate plot to penalize anyone in particular.

2. The measures are unreliable

The graph also highlights a number of complaints that have been voiced for years. As an external review noted in 2013, the numbers jump around too much. The aggregate country rankings involve judgements about what makes a "better" business environment, that have little grounding in research or evidence, and the review panel recommended dropping the aggregate rankings altogether. Doing Business "makes far-reaching observations based on data gathered from sources with a relatively narrow perspective," the external review noted. Somewhat damningly, “improvements” in a country's Doing Business score don’t reliably predict any change in responses to the World Bank's own survey of businesses about the ease of doing business.

3. The index starts from an extreme ideological premise

At a more philosophical level, the index measures the costs of government regulation but none of the benefits of those regulations. It's an extreme libertarian stance, out of step with much of the World Bank's other work. Viewed through the lens of Doing Business, corporate taxes are a pure bad, with no consideration of the benefits that come from raising tax revenue. Safety regulations and minimum wages are pure bads because they slow down business, with no consideration of the benefits to workers or customers. On almost all dimensions, a Hobbesian state of nature would get the best possible Doing Business score.

Regarding this latest scandal, it's kind of funny to think the Doing Business project was somehow exposed this week for secretly trying to undermine progressive governments. That ideology was baked into the design of Doing Business from the start. But now is a good time to change course. In response to the latest debacle, World Bank management has announced a new independent review of Doing Business. Hopefully they’ll use this opportunity to develop a more balanced and constructive stance on how developing countries should regulate markets—beyond a simplistic message of cutting red tape and letting the market rule.