Official China statistics have a reputation of being patchy and even oblique. Thus, new ways of assessing the economy are needed.

In previous years as provincial-level GDP data rolled in, China’s top leaders and the provinces’ respective officials could no doubt take a certain satisfaction from the bountiful increases in wealth displayed therein. But were they to spend a bit more time with the data, such pleasure would likely lead to a state of confusion, if not consternation—the sums, quite literally, don’t add up.

Taken together, the total of provincial-level GDP has outstripped the national figure year after year, a problem that has only gotten worse over time. What was once a discrepancy of a mere RMB 1.97 trillion in 2009 had increased to RMB 4.8 trillion by 2014, as shown by National Bureau of Statistics (NBS) figures.

Far from an isolated incident, official Chinese statistics are home to a range of quirks, which, in combination with various gaps in data provided by the NBS, has led economists, analysts and businesses to cast around for other measures of the world’s second-largest economy. Never was that truer than last year, when the skepticism surrounding Chinese statistics seemingly reached its peak.

Consequently there has been a rise in the number of indicators that purport to give a true reflection of the situation, and which encompass everything from boutique services, such as the China-focused data analytics firm China Beige Book, through to sales of KFC and movie tickets. Moreover, signs often taken to show a weakening economy, such as falls in the stock market and a weakening exchange rate, are less useful in China—the former bears little relation to economic fundamentals, while the latter is subject to heavy intervention from China’s central bank.

But for an economy as complicated as China’s, not to mention one growing as fast, obtaining clear and accurate information may be easier said than done. “Because of the size of China, the ability to be absolutely accurate becomes harder, particularly because it’s a developing, fast-growing economy,” says Matthew Crabbe, author of Myth-Busting China’s Numbers and Research Director, Asia-Pacific for Mintel. “Any deviation from reality is going to be a magnified by an order of scale.”

Casting Doubt

Naturally one of the most contentious statistics is one of the most prominent: China’s GDP. And scrutiny has only intensified as growth rates have fallen amidst the country’s economic transition, with many wondering whether the official figures convey the full extent of the slowdown—a fact that has led some organizations to issue their own growth rates, some of which are several percentage points below the official figure.

More specific doubts stem from the speed at which the GDP figure is produced, with the initial figure for 2015 being revealed less than a month after the start of the New Year. Hong Kong, by comparison, releases its figure in late February despite having an economy a fraction of the size.

Adding to these concerns is the oft-repeated anecdote that in 2007 China’s current premier and then-Party Secretary of Liaoning province, Li Keqiang, dismissed GDP figures as “manmade”, according to a US diplomatic cable revealed by WikiLeaks. Such revelations have done nothing to temper beliefs that the headline figure is unreliable, or perhaps even fabricated.

Moreover, in a period where even shortfalls in growth amounting to a tenth of a percentage point are enough to alarm investors and, perhaps more importantly, determine whether government targets are achieved, highly technical debates about the NBS’s methodology for calculating GDP have assumed greater importance.

Recent focus has been on the GDP deflator used by the NBS, which adjusts the growth rate for changes in prices, after Capital Economics suggested that this was exaggerating nominal GDP value by claiming China was in deflationary territory.

Calculations published by Bloomberg Briefs in September showed that using an adjusted deflator would lead historically to more volatile GDP growth with higher highs and lower lows and, at times, differences of up to two percentage points in either direction from the official figure.

While that debate is by no means settled, there is a more substantive, and perhaps more prosaic, reason for concerns over the accuracy of the NBS’s GDP data—an alarming lack of resources. “They have a very, very teeny national accounts division in the NBS—the number of real professionals on national income accounting is extremely limited,” says Nicholas Lardy, the Anthony M. Solomon Senior Fellow at the Peterson Institute for International Economics.

Having a national accounts division a fraction of the size of equivalents in other countries would be a big enough problem in its own right, but it is exacerbated by the inherent scope of the bureau’s duties.

That lack of sophistication naturally means that the specificity of data can be misleading. “You’re getting general trends rather than genuinely fine-tuned data,” says Fraser Howie, co-author of Red Capitalism.

As is often the case, failures in China’s GDP figure may well then constitute more cock-up than conspiracy. “I think [the GDP figure] is the best they can make it,” says Lardy. “No one’s really been able to demonstrate that they’re falsifying the data or cooking the data in my opinion.”

Indeed, Crabbe notes that the way China’s statistics are assessed don’t always fully take into account its developing world status: “Many of the assumptions they make about China’s statistics are based on assumptions made from a developed-world point of view.” Moreover, Crabbe adds, no national statistics from any country should be taken simply at face value. “There’s an inaccuracy in any national statistics… it’s always a best estimate,” he says.

All the same, Howie notes that while there is a lack of evidence for outright falsification, in general, numbers in China are often political due to government targets, which in turn creates incentives for officials to try to be seen matching or, better still, outperforming.

“I don’t think China actually falsifies its data—I think there’s certainly an incentive to over report it or take the benefit of the doubt and I certainly think lower down at the provincial or county level there could certainly be a lot of falsification,” he says. Indeed, in December Xinhua reported that economic data was being inflated by some local governments in the northeast, the area arguably worst hit by China’s slowdown, in some instances by as much as 127%. Howie adds, “They are trying their best, but when a process is politically driven you’re going to get bad answers.”

Certain numbers, such as highway freight traffic, have been known to be prone to statistical oddities, and the fact that these numbers are harder to verify may make it tempting to massage them for political purposes. Even if that is hard to prove, it and other major numbers can raise more questions than they answer.

One example is China’s unemployment statistic, which has remained remarkably flat in the face of events such as the Asian Financial Crisis, China’s accession to the WTO and the Great Recession. Looking more closely at the data, however, and there is the strange phenomenon of frequent shifts of one-hundredth of a percent that do in fact roughly track growth in the economy.

The idea that the government hasn’t being completely transparent with regard to unemployment data was given credence in January 2015 after Ma Jiantang, then head of the NBS, said in response to a question from the Financial Times that internal non-published unemployment data put the figure at 5.1% for 2014, compared to 4.09% for the official figure for urban unemployment.

But the clearer charge against the quality of the number is what it doesn’t tell observers: the unemployment rate for the huge migrant workforce, the very people who have powered China’s economic boom. And even amongst urban workers it only counts those who bother to apply for the country’s paltry unemployment benefits.

That information may be unavailable simply because the NBS feel they don’t have reliable data. That scenario would fit with a pattern Lardy identifies of the bureau doing just that: “When they don’t think the data are so good they don’t release them… it takes them a long, long time to have confidence that the numbers that they have are worthy of being put out.”

But whatever the reasons behind official headline figures’ vagaries, the question remains: if they can’t give a full picture of the economy, what can?

Manufacturing Data

Li Keqiang’s comments in 2007 not only fueled skepticism around the GDP, they also bequeathed the world a new makeshift index for tracking China’s economy. Having supposedly dismissed the GDP figure, Li noted that he instead looked at electricity output, freight volumes and loan growth to get a true gauge of the economy. Subsequently these have been bundled together to create what is known, unsurprisingly, as the ‘Li Keqiang index’—and the story it appears to tell about the Chinese economy in recent years has not been a good one, with the index frequently cited in support of the argument that growth is being overstated.

But if the Li Keqiang index might once have been a reasonable measure of the economy, it is increasingly less so as China transitions towards growth driven more by consumption and services. Neither of these relies on electricity and freight for growth, and services had already become the main driver of growth in the economy as of 2013. That transition has also had a knock-on effect on the relevance of other closely watched indicators such as growth of exports.

Moreover, reliance on heavy industry isn’t evenly spread around the country, with regions such as the north-east more seriously affected by the current downturn than more developed metropolises such as Shanghai. While a declining Li Keqiang index might suggest something of note about the former, it has a lot less to say about the latter. Nor should it, given one key detail that is often ignored when economists apply the index to the country as a whole.

“[Li] was actually talking in a relatively narrow framework, he was talking just about Liaoning province,” says Howie.

Taking a similar approach to the Li Keqiang index are proxies for the national economy from Bloomberg, Capital Economics, Nomura, Lombard Street Research and Oxford Economics. While offering a more rounded view of the economy than the Li Keqiang index, they too are typically hamstrung by a reliance on data concerning the so-called ‘old’ economy such as construction, electricity output and seaport cargo.

Similar concerns apply to China’s much-watched manufacturing purchasing managers indices (PMI)—both the official gauge and alternatives from companies such as Markit—which spent most of 2015 in contractionary territory.

“I don’t think the PMI data is worth very much at all,” says Lardy, noting that official output data has also recorded persistent declines, albeit not as severely as the PMI data would suggest it has. “It’s trying to be forward looking, but how successful it is I think is an open question.”

In addition to the manufacturing PMI, the NBS and Markit both produce another gauge for services (or “non-manufacturing” as it is called by the NBS), both of which produce an altogether more positive picture when taken together with their respective counterparts. But not everyone is convinced of their value generally.

“A lot of economists tend to downplay PMI because they’re a pretty blunt measure of things,” says Howie. “I think in China as well it’s not clear that with the PMIs you’ve actually got a long enough historical data set that you can actually really tell that much about the underlying economy.”

Questions of Service

If consumption and services are increasingly important to the economy, then information on these areas is, too. However official data for both leaves much to be desired. Breakdowns for the composition of value-added in the tertiary sector, which covers services, are dominated by a blanket category simply labeled ‘others’—which in 2013 accounted for as much as 38.5% of the sector’s total.

“The real problem is we don’t have any real good, high-frequency data on the service sector,” says Lardy. “It’s half of GDP now and we only get value-added data on a quarterly basis and there’s very little disaggregation when we do get the value-added… there’s no underlying data to back it up or granular data like there is in the manufacturing sector.”

Meanwhile official figures on retail sales are becoming increasingly useless as indicators of total consumption, which is itself more and more dominated by services. And to add still further uncertainty, official retail sales data also covers purchases by the government and public agencies.

With NBS data so wanting, economic sleuths must look elsewhere. “[The data] is coming out of industry associations and other government agencies, so you have to look at things like transportation, movie ticket sales, passenger traffic on the trains, civil aviation,” says Lardy. Crabbe also adds tourist numbers, consumer
confidence indices, online retail sales and growth in consumer incomes as pertinent data points.

“I’ve just done some research on this which basically indicates consumer services is probably as big as the retail market, if not bigger,” says Crabbe, noting that the work included looking at everything from hairdressing to funeral services to elderly care. “A better understanding of the size of the consumer services market will give us a better overall idea of the true size of the consumer economy, and actually I think we would find that the economy is very robust because of that.”

But Howie sounds a note of caution regarding some of this data and notes the very real risk of over reporting. Last year the domestic film Monster Hunt was reported to have become China’s highest grossing film ever after it took in RMB 2.43 billion at the box office—RMB 160,000 more than the previous record-holder, Furious 7. However, after internet users highlighted unusual screening times for the film during its final 15 days in theaters, the distributor later acknowledged giving away RMB 40 million worth of tickets to bolster its box office total.

“You just know that’s happening in a lot of cases because that’s how China works, that’s the incentive structure,” says Howie.

That being so, Howie advocates not simply relying on statistics but instead matching this with how things seem on the ground. To this end he cites surveys of the kind conducted by J Capital Research’s Anne Stevenson-Yang as providing helpful information. “I still feel the anecdotal is very important, and this especially if you’re talking about the consumption or service sectors,” he says.

Howie points to the key example of overstaffing: there is no index measuring this phenomenon, but is easily observable in everyday life and all the same tells you something about the economy and productivity. “It’s taking some of these numbers with a pinch of salt—does that reconcile with what I understand it to be?” he says. Conversely, the fact that Chinese cities are now awash with couriers delivering packages supports reports of prodigious growth in numbers relating to e-commerce.

Drilling Down

While information on the health of China’s national economy remains vital, there is nonetheless the possibility that for certain observers it is beside the point.

“Many of the indicators people rely on, such as GDP, aren’t actually that useful when you’re looking at an economy like China,” says Crabbe. “The key thing here is that really what companies need to know about is the sector that they’re really involved in, not the overall national economy size and so on.”

An important part of that, says Crabbe, is to realize that China isn’t just one homogenous market. “As a company looking to invest in or actively take part in the China market, you have to look at it regionally and you have to understand regional markets, different consumer trends within those regions and the more detailed micro-demographic or regional trends within the market specifically that you’re looking at.”

GDP remains a convenient and appealing concept for any country, and China is no exception—and indeed, the longstanding obsession isn’t much helped by a strong, target-based political culture. Still, the best analysis will continue to come from those who take a broad and even-handed view of the economy, although there is also an onus on the NBS to improve its methodology and transparency as, rightly or wrongly, so many decisions and perceptions are based on their numbers.

“When I met with Liu He [Vice-Director of National Development and Reform Commission] last year, I said one of the most important things you should do to improve policy making is to have better GDP data,” says Lardy. “You need better data in order to judge exactly where the economy is so policy making can be improved—the NBS doesn’t have enough resources.”

In its place, observers will need to give consideration to the full gamut of information coming out of China—everything from e-commerce data to tourist numbers to manufacturing output—together with their own street-level observations to get a reasonable picture of China’s economy, however imperfect it may nonetheless remain. Meanwhile, businesses will be best served by moving to a more nuanced picture of China’s composition and, consequently, focusing on their specific market while also keeping abreast of the national situation.

“China is one of those markets where you really have to do your homework to get it right, and I think relying too much on easily accessed data is erroneous,” says Crabbe. “It’s time for people to stop fixating on the national macro-economic data and move into more granular, regional, micro-market data to base their business decisions upon.”