The Japanese Economic Review - Note on the Policy of the JER Editorial Board on the Recently Found Problems with the Monthly Labor Survey of Japan

(1) Overview of the issue

The Monthly Labor Survey forms a foundation for the Income Series side of the Japanese National Accounts. It is an establishment-based survey on employment, labor earnings, and work hours.  The Survey has the following official sampling policies. 1) For establishments with the number of employees ranging from 30 to 499, the survey conducts random sampling with uniform sampling rates across all prefectures. 2) The survey samples all establishments with more than 500 employees.

It is now known that the Monthly Labor Survey went through the following important changes without announcements.

1. At least since 1996, the actual number of establishments sampled was about one-tenth smaller than what was announced.
2. From 2004 to 2017, in Tokyo prefecture, in some industries, the data on establishments  with  more than 500 employees only included randomly sampled subset of the establishments, which amounts to about one-third of the total sample. Such a sampling scheme was inconsistent with  the official rule, which stipulates that the entire population of  the establishments  with  more  than 500 employees must be surveyed. Moreover, no sample reweighting was applied to the randomly sampled data. Consequently, without appropriate reweighting, the collected subset of the establishments with more than 500 employees would under-represent the population. As larger establishments and establishments in Tokyo tend to pay higher wages, such under- representation would likely result in a downward bias in the aggregate regular earnings statistics. Since necessary establishment-level data for reweighting for the years 2004 to 2011 are still missing, it has been impossible to reconstruct the appropriately weighted aggregate statistics for those years.
3. From 2009 to 2017, the sampling rate of establishments in some industries in Tokyo prefecture whose number of employees range from 30 to 499 was higher than  the other prefectures, but  the official data description states no such exceptions. Those industries were the ones whose numbers of establishments were low in Tokyo prefecture.

From 2012, the original data on sampled establishments can be retrieved and proper reweighting is underway as of 2019. The Monthly Labor Survey has not been able to correct issues 2) and 3) due to the loss of the original data.

The above overview is based on the report on the facts and assessments on the inappropriate treatment of the Monthly Labor Survey, issued in January 2019, and its supplementary report, issued in January 2019.

(2) The policy of the JER editorial board.

The JER editorial board met on May 29, 2020, and agreed on the following policy.

The JER editorial board welcomes researchers to work on finding ways to correct the problem.

We encourage any authors who submit the manuscript that uses the Monthly Labor Survey to 1) include alternative results that use similar datasets. 2) make reasonable adjustments to the data. 3) use various methods mentioned above for robustness checks. Furthermore, we ask authors who use the Monthly Labor Survey to check the government websites and use the official corrected series whenever they become available. This time, irregularities in the Monthly Labor Survey were detected and corrective actions are being taken. But that does not guarantee reliability of the MLS in the future  or  other  datasets. We reaffirm our strong belief that active empirical research using macro and micro datasets is their best quality control. The editorial board would like to consider positively any empirical studies that take a careful look at widely used data and discuss its properties, including potential irregularities, as the main results or part of them.

As we stated above, the Monthly Labor Survey is a source of labor data at the macro level, and forms a foundation for the Income Series side of the Japanese National Accounts. As a consequence, it could influence statistics such as saving rates and labor shares. Results of statistical analyses that rely on those series could be affected by the data problem. At the editorial board, we are discussing our policies regarding the papers that have been potentially affected. Those include both the articles that we have already published in the JER and the papers that will be submitted to us in the future. We shall notify our decisions as needs arise.

(3) Follow-up notes

Since the above decision was made at  the editorial board, there have been two notable developments  on the Japanese government web site in the second half of the year 2020.

(A) The Ministry of Health, Labor and Welfare published "Estimates for the Purpose of Time Series Comparison" for the Monthly Labor Survey, which represents their best efforts to correct the statistics.

(B) Based on the above estimates, the Economic and Social Research Institute of the Cabinet Office published a new National Accounts statistics with Benchmark Year Revision of 2015, which reflects those corrections.