全球智库信息集成服务系统

Economists have long recognized the importance of high-quality information to the labor market. Without knowledge of opportunities available or skills in demand, searching for a job can become costly, not to mention miserable. By contrast, widely available and high-quality labor market information can make the process of finding new opportunities easier and more efficient.¹ Accessing and utilizing such relevant and granular information, however, has long proved challenging. Going at least as far back as the 1930s, the public and private sectors have experimented with innovative ways to reduce search costs and informational frictions.² Ideas have included greater use of public and private intermediaries to gather and disseminate information on the labor market, better end-user services that match workers with employers, and, more recently, efforts to enhance and/or supplement publicly available information with richer private sector data.³ Few, however, have considered more fundamental refoms aimed at how real-time labor market data is published and shared in the first place.

Over the past two decades, online job postings have emerged as a principal mechanism for signaling real-time information about the labor market. Their movement to the web has reduced the costs of advertising openings, enabled more functionality for employers to convey information, and made it easier to aggregate postings into central repositories.⁴ Together, these advances have transitioned online job postings from mere advertisements to increasingly informative signals of real-time labor market demand. Yet, more can be done for online job postings to reach their full potential. First, while they have become a commodity of sorts for online job search tools, online postings have also become concentrated in a small number of third-party applications that constrain easy data sharing and reuse. Second, postings frequently leave out pertinent information such as wage and skill requirements. And third, postings are seldom published in consistent, machine-readable formats that would make it easier for algorithms and third-party applications to make better use of them. These constraints hinder not just the quality of available data arising from online job postings, but also competition and innovation among the job search-and-match services that rely on such data.

The labor market would benefit from a collaborative data governance strategy that mitigates the influence of data monopolies—in which a few companies have unparalleled access to job postings—and that enables a more seamless exchange of high-quality real-time data. In other sectors of equivalent national importance, a powerful policy instrument has been the encouragement and implementation of open data standards—consistent machine-readable formats for how information is published online.⁵ From health care to energy to education—all industries where data is a critical asset—the use of open standards has made information easier to access, query, and share. This, in turn, has enabled more competitive markets for third-party applications and services.⁶

A similar opportunity now exists for the labor market. The broad adoption by employers of an open data standard would make online job postings better sensors of real-time labor market demand. Specifically, it would ensure that postings are more widely disseminated; it would nudge job descriptions to be more consistent, detailed, and precise articulations of employers’ needs; and it would make algorithmically curating specific information from a posting—such as skill keywords or evolving technology requirements—substantially easier.

From job seekers and businesses to education providers, policymakers, and innovators, this would benefit nearly all labor market stakeholders. For one, the resulting data could serve as a critical building block for a more robust and responsive public support system for job changes and job training. It could also be a boon to researchers studying evolving trends in skills and labor markets.⁷ It may even accelerate advances in machine learning and artificial intelligence being applied for better job matching, improved recommendation services, and more sophisticated local labor market insights.⁸

This report first reviews the limitations of online job postings as real-time sensors of labor market demand, describes the benefits of open data standards, and explains why the public sector is well-suited to facilitate their adoption. Second, it examines two pilot efforts that are currently testing potential policy approaches. Finally, it provides a set of policy recommendations through which federal and state governments can facilitate the development of an open data standard and encourage its adoption.

Job postings are a crucial but not well-understood aspect of the job search process.⁹ Signaling job titles, job locations, and required skills and credentials, online job postings are powerful indicators of labor market demand. Over the past several decades, through technical innovations, postings have become cheaper to advertise, revise, and amass. This has subsequently increased their value as a resource for online job search tools. Yet, for several reasons, job postings have yet to reach their full potential as a definitive primary source of real-time labor market information.

Restrictions on reuse

First, there are problems with access. The past two decades have been marked by an emerging market for third-party job search tools.¹⁰ These applications have improved the job search experience in important ways.¹¹ Employers upload ads for job openings to these platforms knowing that they will reach a robust network of job seekers. Platforms may also harvest postings by regularly scraping the web. In building out these job listing repositories, these entities cultivate some of the richest labor market information available. However, third parties can also apply frictions on the reuse of those aggregated postings, largely on account of valuable intellectual property used in the curation, normalization, and organization of that data. These frictions, however, can be applied to the pre-processed data—the initial aggregation of raw job postings. Through technical mechanisms, legal restrictions, or expensive fees, these frictions can end up stifling the broad dissemination of online job postings.¹²

Labor market data are best utilized when widely available. Proprietary restrictions can make accessing such real-time data either difficult or prohibitively expensive. This, in turn, hinders utility. As Woods and O’Leary write, “while a proprietary firm rightly would desire to maximize profits from selling [labor market information], such an approach would not likely optimize use of the information to improve efficiencies in the overall labour market.”¹³ Moreover, Woods and O’Leary caution that proprietary data may poorly cover important parts of the labor market—“particular niches” but not “a nationwide system.”¹⁴ There is some evidence of this. A comparison of online job postings by the private job board CareerBuilder and the monthly U.S. Bureau of Labor Statistics’s Job Openings and Labor Turnover Survey found that the CareerBuilder data overrepresented fields such as finance and insurance and underrepresented state and local government, food services, accommodation, and construction.¹⁵ This may be attributable to the distribution of jobs that can be found online, with some sectors still unlikely to use the web as a primary method for recruitment. However, the potential of a gap provides another reason for a more open data architecture.

Finally, restrictions on reuse can stymie the development of new products, services, or analysis, while serving as a competitive moat for existing platforms and aggregators. Greater burdens on accessing critical job posting data can pose a barrier to innovators and entrepreneurs who might wish to enter the market for providing real-time labor market services and applications.¹⁶ This entry barrier is further compounded by the fact that third-party platforms have grown fairly concentrated. A 2014 report published in the American Economic Review found that “about 60 percent of all ads appear on five job boards,” while a forthcoming paper in the Journal of Public Economics reports that a single private sector job board included more than a third of all U.S. job postings.¹⁷ Improving the overall utility of job postings as signals of labor market demand will mean ensuring that postings are more widely available and interoperable across third-party applications.

Insufficient and inconsistent quality

Next, online job postings could be more informative signals of labor market demand. Evidence points to a generally low level of quality marked by the frequent absences of critical information, inconsistent descriptions, as well as a tendency for postings to be outdated or misleading. Economist Vera Brencic, for example, has looked at job listings from several countries, finding that employers differ widely in the amount of information they include, such as required skills and wages.¹⁸ A separate analysis of a large U.S. private job board found that only a fifth of the online job postings included projected wages.¹⁹ A 2015 survey of several hundred U.K. postings found that just 12 percent met the minimum job quality standard used by the nation’s Advertising Standards Authority.²⁰

Although it may be the case that employers are simply choosing not to include this information, the variability for what is included and how such information is defined may be in part a function of the way employers post ads for vacancies.²¹ Most employers, be they large or small, post job listings on their own, with little sense of, or no standardized template for, what constitutes a high-quality listing. This decentralized process avoids the costly latency that would come with someone dictating job descriptions from on high, but nonetheless contributes to the inconsistency—not to mention the insufficiency—of the information found in job postings. Adding to this challenge, as economist Peter Diamond has noted, “a central element in the labor market is that workers and jobs vary greatly.”²²” labor market information due to the “intrinsic heterogeneity of workers and jobs.” Autor, “Wiring the Labor Market.”] This heterogeneity among skills, industries, and credentials makes preserving a consistent level of quality across all job postings naturally more difficult.²³ Some have even argued that skills in demand may be evolving at an accelerated rate due to changes in technology, a trend that could make the task of capturing nuance more formidable as well as more important.²⁴

If employers and job seekers continue to rely on online job postings as sensors of real-time labor market demand, the consistency and detail of job postings must be improved. Indeed, job seekers should be well-served by such an improvement. New research, for example, shows that relative to other factors, the words contained in job titles play a powerful function in guiding job applicant behavior.²⁵ Similar benefits may accrue if employers were to adopt more appropriately standardized and detailed skill and competency requirements.

Inadequate access to standardized, structured data

The final limitation on online job postings is that a narrow number consistently adhere to a standardized, open, and machine-readable format or structure. This makes postings less useful as signals of labor market demand.

When job seekers read a posting online, they easily discern various elements of the ad, such as skills, wages, and/or benefits. However, when a machine queries the web for postings, without the benefit of a standardized data structure or markup that provides context for what is included in the text—metadata—this task inevitably becomes more difficult.²⁶ To roughly analogize, imagine reading a tax return without a format clearly delineating different types of income, exemptions, credits, and deductions. It would be nearly incomprehensible. Similarly, without a consistant format or structure, algorithmically making sense of postings becomes harder. Although dominant third-party applications have in some cases begun to use techniques such as natural language processing and machine learning to parse unstructured data, these advances are limited to a small number of firms.

Separately, even when online job postings are structured, different postings frequently have different markups. Unless third-party applications are familiar with the specific markup, they are hard-pressed to use those postings.²⁷ This constrains the interoperability of a job posting once it is published and ultimately leads to siloed data or reduced job board quality.²⁸ If online job postings are to fully realize their potential as machine-readable and interoperable sensors of labor market demand, broader technical reforms are needed for how postings are published, accessed, and used.

One potential approach to improve online job posting quality and availability would be for labor market stakeholders to reach consensus on an open data standard for online job postings. This would not be a standard for the specific job titles or tasks that are unique to each employer, but rather for how such categories of information are published on the web.

What is an open data standard?

From screws and fire hydrants to telephone networks and internet protocols, standards ensure compatibility between products and services. As studies of standardization efforts have shown, such compatibility can result in “considerable gains to consumer welfare.”²⁹ By harmonizing the interactions within a system, standards create opportunities for innovation and entrepreneurship as well as greater consumer choice.³⁰ This is especially true of the exchange of real-time information across the internet. As data have grown more valuable and third-party applications that make use of data feeds have proliferated, there is a growing demand for data to be structured and seamlessly exchanged in a neutral way across platforms and applications.³¹

This has led to the creation of voluntary, industry-led data standards for online content. A data standard “describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource.”³² Inconsistent publishing formats are akin to mismatched fire hoses and hydrants; the incongruence stymies the flow of the information. By comparison, a consistent data standard makes it easier for third-party applications to find, parse, and utilize content regardless of where it is published. As a 2016 Obama administration report noted, data standards can “reduce the cost and difficulty of making new data sets useful.”³³

One such effort is Schema.org, an initiative among leading search engines to standardize the back-end framework for specific types of web and app content.³⁴ Categories include blogs, movies, and restaurants.³⁵ For each category, Schema.org has established generic but standardized technical structures—schema—underlying the text of a webpage.³⁶ By adding context, or structure, to the text of a webpage, the schema makes it easier for such content to be identified and used.³⁷

Applied to online job postings, a data standard would enable stakeholders—from innovators to developers of state-based workforce development systems—to more seamlessly take advantage of online job postings as real-time signals of demand.³⁸ First, it could serve as a template for employers, providing suggestive parameters for what to include in postings. This could nudge employers toward more consistent and precise job posting descriptions, while preserving their autonomy to advertise as they deem fit. Second, if broadly adopted by employers, a data standard could make online postings more widely available, resulting in more robust competition and variation among third-party applications that put job postings to use. This could also support researchers and policymakers, allowing them to be less reliant on vendors or aggregators for raw labor market data or to more efficiently use such firms for value-added insights or services.

The public sector’s role in standard setting and labor market information

The public sector, with its experience facilitating standards development and its long-held role providing timely labor market information, is well suited to organize a consensus-seeking effort to design and adopt a job posting data standard.

First, going back to the early 20th century and the development of the National Bureau of Standards—now known as the National Institute for Standards and Technology (NIST)—the U.S. government has played a role in facilitating sectorwide norms and guidelines.³⁹ The Obama administration was particularly active in convening outside stakeholders to design and adopt standards aligned with key policy priorities. Examples of these efforts include NIST and the U.S. Department of Energy working with industry to make energy consumption data available to consumers, as well as a NIST-facilitated effort encouraging the adoption of voluntary industry-led standards and best practices for cybersecurity.⁴⁰ The authority for these efforts was articulated in a 2012 White House memorandum outlining the involvement of the federal government in encouraging and facilitating standards for “national priorities.”⁴¹

In addition to facilitating standards, since as far back as the passage of the Wagner-Peyser Act of 1933, the public sector has also been a critical player in the provision of real-time labor market information.⁴² Over the years, government has frequently harnessed new technologies to enhance the quality and dissemination of these data. During the 1970s, for example, state employment services began to collect and share labor market information with other states through such innovations as the Employment Security Automated Reporting System.⁴³ This ended up being the antecedent of the more formulized Interstate Job Bank: “a national database containing state job openings from employers who requested nationwide listings,” and, later, America’s Job Bank, a web-based interface for job and resume posting.⁴⁴

While America’s Job Bank is no longer in operation and, in many cases, public actors from state governments to community college systems are purchasing granular labor market insights from third-party vendors, there remains a general recognition that “providing information to the public is one of the primary rather than ancillary functions of the public workforce investment system.”⁴⁵ Specifically, the secretary of the U.S. Department of Labor (DOL) is required to “oversee the development, maintenance, and continuous improvement of a nationwide workforce and labor market information system,” including ensuring the “dissemination of such data, information, and analysis in a user-friendly manner and voluntary technical standards for dissemination mechanisms.”⁴⁶ Additionally, states are required to administer labor exchange systems that, among other things, “assist job seekers in finding employment … assist employers in filling jobs; [and] facilitate the match between job seekers and employers.”⁴⁷

The combination of these policy legacies, unique capabilities, and existing authorities makes the federal government a natural fit to bring together requisite stakeholders and accelerate the design of an open data standard for online job vacancies.   

Two case studies are already showing progress toward the development, adoption, and use of an open data standard for online job postings. These pilot projects show how a data standard for job postings can be implemented as well as how the resulting data can be harnessed for new applications and analysis.

The United Kingdom: Adopting a schema

Spurred by a 2015 survey finding that just 12 percent of U.K.-based job postings included sufficient information to meet country-certified quality standards, the British government began to search for ways to enforce a baseline level of quality for job posting content.⁴⁸ Research by the country’s Government Digital Service resulted in an initiative to have civil service job listings adopt a data standard.⁴⁹

The idea for an open data standard was inspired by an initiative championed by the Obama administration back in 2011 called the Veterans Job Bank.⁵⁰ The job bank was a repository of job postings from employers making an explicit commitment to prioritize the hiring of veterans. To create a relevant and up-to-date list of postings, the Executive Office of the President partnered with Schema.org to design and adopt an open data standard.⁵¹ Built with industry participation, the resulting JobPosting schema (see Table 1) enabled employers to mark up, or tag, job postings where the employer had made a commitment to hiring veterans, thus making it easier to algorithmically find, aggregate, and publish these postings in real time and at low cost.⁵²

In researching this effort, the United Kingdom saw the potential for a broader goal: using the schema as an overall template for what should be included in job listings.⁵³ The government realized that such an intervention would make data on job postings easier for third-party websites and applications to algorithmically parse, query, and reuse.⁵⁴ Excited about the potential, the Government Digital Service began a thorough review process to determine whether or not this information technology standard should be implemented. The review included a preliminary exam by a panel of open standards experts followed by a subsequent review by the Open Standards Board, a government-appointed coalition.⁵⁵

As of 2017, the United Kingdom has been implementing the schema’s adoption across all central government openings.⁵⁶ Agencies not under the direct jurisdiction of the central government are also being encouraged to adopt the schema. The U.K. Government Digital Service is assisting with the adoption by providing implementation guidelines, informal trainings, and a communication channel.⁵⁷ Meanwhile, Britain’s Department for Work and Pensions has prototyped a series of different citizen-facing services that could take advantage of the schema.⁵⁸ The government’s expectation is that over time, the private sector will also take steps to adopt the open data standard.

Virginia: Publishing real-time data on labor market demand

The second case study comes from Virginia. The state is showing what is possible if a data standard were to be broadly adopted. In 2014, the governor of Virginia signed Executive Order 23: “Establishing the New Virginia Economy Workforce Initiative.”