Introduction

Questions about statistics can strike fear even in the most experienced librarian. By the time patrons contact the library with a question, they’ve already plumbed the depths of the internet and have found the statistics that are easy to discover, so librarians are usually asked more challenging questions. The availability of statistics is constantly changing, but knowing the most common government statistical sources provides a foundation for helping patrons to find data and statistics.

It’s essential to have a firm understanding of the differences between the terms data and statistics. Data refers to the raw data that is collected through three primary means: enumeration, surveys, and administrative records. Enumerations are usually accomplished through censuses, which count all of the target population, be it people, houses, businesses, or animals. Microdata is the actual raw data, or a sample thereof, with personally identifiable information removed. The term PUMS usually stands for Public Use Microdata Sample, which allows researchers to perform their own analyses rather than rely on the published aggregate statistics. While some microdata is available for free, other data files are available for a (sometimes hefty) fee. Surveys collect data on a sample of the population and present the data as estimates. The sample needs to be representative and large enough to be valid. Because they don’t count the entire population, surveys have a margin of error that increases in inverse proportion to the sample size. Administrative records such as tax returns, death certificates, or school enrollments are another source of government data.

Statistics are derived from the raw data. In the old days, statistics were compiled in either tabular or textual form and published in printed reports. With the common availability of computers, most patrons nowadays want downloadable data that they can manipulate. Ideally, someone else has converted historic statistics into a spreadsheet or database. Sadly, the availability of downloadable statistics often does not match patron expectations.

When statistics are available electronically, they may simply be in an Excel spreadsheet. For example, the Hawaiʻi Data Book allows users to download statistics in PDF or Excel format. Increasingly, agencies have developed query interfaces so that researchers can conduct their own queries by selecting the rows and columns they need. For example, the Centers for Disease Control has a data visualization tool that allows users to select which data points they want to display in a chart or graph.

Reference Strategies

As with any reference question, a librarian needs to nail down exactly what the patron wants. Some questions to ask include:

• What is the geographic area in question? The entire U.S., a state, county, or smaller area?
• Is the patron looking for annual, quarterly, monthly, weekly, or daily data?
• Is the most recent information needed, or is the patron seeking historical data?
• How granular does the information need to be? Is the patron looking for aggregate data or microdata?

Librarians need to manage patron expectations regarding statistics, just as we would for any other type of information. Patrons often want data at a frequency or a level of granularity that is not available. A patron may seek monthly data, but it may only have been published in annual releases. Or, they need county data, but can only find it for the state as a whole. It may be necessary to contact an agency to verify the availability of its data products.

Once the specific parameters are clear, determine who collects the desired information. Assessing whether the statistics are collected by a government agency, a nonprofit organization, or a private company can be tricky and depends in part on your knowledge of what kind of information is collected by each type of entity. A little preliminary searching in Google or USA.gov might help. You can also consult Statistical Programs of the United States Government produced by the Office of Management and Budget to gain an overview of the statistics maintained by government agencies. Older editions are in HathiTrust.

Let’s analyze a question that was posted on the Govdoc-l list. A patron wanted to find statistics on crimes committed by immigrants in two counties in Texas. He also wanted to know the interval between the date of immigration and the date of the crime. Where could we look for this information? The demographic data might be on the arrest record, for starters. Viewing the criminal history report form for Texas, it is apparent that the form does not include a space to record an arrestee’s immigration status. Where else might we find the immigration status of a person convicted of a crime? It might be mentioned in court records or newspaper accounts, but it might not be available for each arrest. Determining the period of time between when a crime was committed and when the person immigrated could also require a lot of legwork. Immigration and Customs Enforcement collects statistics on immigrants who have been convicted of crimes, but they don’t release the data by state, much less by county.^[1] As you can see, this is a question that could, at best, be answered using administrative records or possibly a survey of convicts, but it’s unlikely that the patron could find this precise data in a published form. Sometimes the desired data is available on an agency’s computer, but it may not be publicly posted. In such cases, a patron might need to make a request through the open records process prescribed by the state or federal government, which will be covered in Chapter 17.

General Sources

Statistical Compendia

One of the strategies we can use when trying to figure out where to find statistics is to consult statistical compendia. The most well-known such work is Statistical Abstract of the United States (StatAb), which was formerly produced by the Census Bureau and is now a printed reference book and a searchable database available from ProQuest covering 2013 to the present. ProQuest also offers separate databases with data from historical statistical abstracts covering 1878-2012. Now that most government-produced statistics are published online, it is not as critical to be able to consult the StatAb for current statistics, but when you are looking for historical statistics, it can lead you to the appropriate agency.

Most federal agencies produce some kind of data book. For example the IRS Data Book has tables with numbers of returns, tax collections, and other data by state. States also publish data books like the Hawaiʻi Data Book that can help you to determine whether the statistics being sought are available from the state or the federal government.

Government agencies have also published guides to their statistical works or programs, such as the EPA’s A Guide to Selected National Environmental Statistics in the U.S. Government and Data on Health and Well-Being of American Indians, Alaska Natives, and Other Native Americans: Data Catalog by the U.S. Department of Health and Human Services.

Statistics and Data Portals

Portals are becoming increasingly important because of the proliferation of datasets and databases. Data.gov provides access to public databases and data sets, but it also lists some non-public data. One helpful feature of Data.gov is that it provides metadata about the datasets as well as contact information for the person responsible. The relatively new portal ResearchDataGov is a website that allows users to search for and request government-collected microdata.

Most federal agencies have a data and statistics portal, like the Centers for Disease Control’s Data & Statistics page. Some agencies have 20 or more years of statistics on their websites, but others provide limited online access to historical statistics. A data catalog is a type of data portal that allows a researcher to search for and download datasets. An example is the Department of Labor’s data catalog, which provides access to various enforcement datasets from Department of Labor subagencies.

Another important category of data is geospatial data, which is discussed in chapters 14-15. Most states have a portal that provides access to a number of downloadable files that can be used in a geographic information system (GIS). Some also have online mapping capabilities.

Cooperation in Statistics

The federal government provides aid to states to enable them to collect data. For instance, the USDA offers funding to states to help them meet reporting requirements of the Food and Nutrition Act. State and local statistics may also be collected in cooperation with a federal agency:

• State Data Centers—Census of Population and Housing; Economic Census
• State departments of agriculture—Census of Agriculture, agricultural production, prices
• State departments of health—health surveys, vital statistics

Similarly, national governments report statistics to intergovernmental organizations. Some examples:

• International Labour Organisation’s Global Estimates of Child Labour, 2012-2016 was compiled from 105 national reports.
• The World Health Organization compiles Global Information System on Alcohol and Health (GISAH) based on the WHO Global Survey on Alcohol and Health.

University Research Centers

Many university-operated research centers collect, analyze, and publish data and data tools. Here are some university research center projects that provide access to data sets and aggregate data from a variety of government sources.

• StatsAmerica is a service of the Indiana Business Research Center at Indiana University. It is primarily focused on economic and labor statistics.
• ICPSR is a consortium of universities that provides data curation and management for data from government agencies, private organizations, and individual researchers. Some of the data is free, but membership is required to access some data products.
• Minnesota Population Center (MPC), as the name implies, provides access to data on population, among other things, through its Integrated Public Use Microdata Series. MPC has done some processing of the data to make it more usable. It’s another way to access census data, health surveys, and education data.

Universities also receive federal funds for statistical programs such as the UH Mānoa Center on the Family, which compiles demographic and social statistics about Hawaiʻi families. Universities also maintain lists of scholars who can be tapped for their expertise.

Industry Associations, Think Tanks, and Non-Profit Organizations

Some government statistics may be gathered or republished by industry associations, research organizations, or non-profit organizations. Industry associations typically provide statistics to members, but some charge a fee for access or restrict access to members. An example is the School Nutrition Association, which publishes data for its members about school breakfast and lunch prices and program participation, among other things.^[2] The Pew Research Center, a non-profit research organization, uses government statistics to perform its own analyses of phenomena such as wage stagnation.^[3] Rand Corporation is a well-known think tank that publishes research in a number of subject areas, including education and public policy. Its report on student outcomes in Louisiana was based in part on U.S. Department of Education data.^[4]

Testimony submitted during congressional hearings by representatives of universities, research organizations, and think tanks are another source of statistics. For instance, the hearing Comprehensive Immigration Reform: Government Perspectives on Immigration Statistics, held before the House Subcommittee on Immigration, Citizenship, Refugees, Border Security, and International Law, includes testimony containing statistics from the think tanks Heritage Foundation and the Center for Immigration Studies.^[5]

 

Commercial Databases

ProQuest Statistical Insight (SI) is the granddaddy of sources for government statistics. SI started out as three printed indexes accompanied by microfiche sets: American Statistics Index (ASI), Statistical Reference Index (SRI), and Index to International Statistics (IIS). ASI is compiled by contacting all of the federal statistical agencies and acquiring copies of their statistical publications and data. The list of sources includes hundreds of federal government agencies, both current and past.^[6] The original publisher, Congressional Information Service, created abstracts of the publications and issued microfiche reproductions of them. Libraries could choose to purchase the index/abstract volumes alone or as a package with the microfiche. ASI provides indexing and abstracting from 1973 forward and full text from 2004 onward. Now published by ProQuest as an online database, SI also indexes individual statistical tables within larger publications, so it provides a level of granularity that can be helpful when one is trying to find very precise data. SRI is an indexing and abstracting source for non-federal statistical publications, with coverage beginning in 1980 and full text coverage from 2007. Its sources include professional associations, non-profit organizations, research institutes, state governments, and commercial sources. IIS covers intergovernmental organizations. Statistics of intergovernmental organizations and foreign countries will be covered in chapters 12 and 13.

Even if a library does not subscribe to the full text modules of SI, it is frequently possible to use SI’s indexing to find the publications online. Let’s say we are looking for statistics on homeless veterans and we want to compare Hawaiʻi to another state. Using SI, we can search for homeless veterans, then narrow our results to sources that have a breakdown for states. The table State of Homelessness looks promising. If we visit the home page of the National Alliance to End Homelessness, which is the source of the table, we can download the source publication and find the table.

These commercial publishers offer databases that include current and historical census data and other government statistics:

• Data-Planet, published by SAGE, is a collection of datasets, many of which are from U.S. government sources. It also contains data from commercial sources and intergovernmental organizations such as Organisation for Economic Cooperation and Development and Food and Agriculture Organization of the United Nations.
• Geolytics sells a variety of analytical tools for current and historical census data going back to 1960.
• Social Explorer employs an intuitive interface to explore current and historical census data as well as elections and law enforcement data.
• SimplyAnalytics (formerly SimplyMap) includes demographic data from the American Community Survey along with other government and consumer data, and there is an optional historical census module.
• Gale’s DemographicsNow includes demographic, business, and residential data. 
• PolicyMap is primarily a mapping tool using demographic, health, and economic data. 
• Data USA pulls together federal data with non-government sources and offers mapping and analytical tools.

Interpreting Data

Several types of documentation may be helpful when you are assisting a patron who is trying to understand statistics or data. Manuals such as Uniform Crime Reporting Handbook: How to Prepare Uniform Crime Reports prescribe how data is to be collected or reported. Often, there is a code book that explains the coding system for the survey. For example, the Behavioral Risk Factor Surveillance System (BRFSS) has a codebook that explains variable names and locations. Technical documentation includes definitions of terminology, as shown in the Table Definitions, Sources and Explanatory Notes for petroleum statistics. Occasionally, it is necessary to contact the issuing agency to seek additional information about a database or statistical publication.

Statistical Resources by Topic

Business and Economic Data

The U.S. government compiles and publishes a wide variety of statistics and data related to trade, economic activity, and commercial enterprises, not only for the United States as a whole, but also for states, counties, and even foreign countries.

Some of the most important statistics issued by the U.S. government are economic indicators. The Council of Economic Advisors publishes the annual Economic Report of the President and the monthly Economic Indicators, which includes such indicators as gross domestic product (GDP), labor force statistics, interest rates, federal receipts and expenditures, and consumer prices. The data is also available in the Federal Reserve Bank of St. Louis’ FRED database, which has coverage back to 1948. In addition to national statistics, FRED also includes some international and regional data series.^[7]

Frequently, we receive questions about imports or exports. Important databases for this data are USA Trade and UN Comtrade. In USA Trade you can search by commodity, state, or port, and use either the North American Industry Classification System (NAICS) code or the Harmonized Tariff Schedule code. So, for example, you can create a report showing the total value of seafood exports from Hawaiʻi to the world or to particular countries. USA Trade requires users to register for a free account. The freely available United Nations Commodity Trade Statistics Database (UN Comtrade) “contains detailed goods imports and exports statistics reported by statistical authorities of close to 200 countries or areas. It concerns annual trade data from 1962 to the most recent year. UN Comtrade is considered the most comprehensive trade database available with more than 3 billion records.”^[8]

The International Trade Commission’s Dataweb provides access to U.S. import and export statistics, U.S. tariffs, U.S. future tariffs and U.S. tariff preference information from 1989 to the present. Access is free, but it requires the user to register for an account. ITC also publishes the Harmonized Tariff Schedule, a vast list of tariffs based on a classification scheme. Using HTS, we find that the code for macadamia nuts is 2008.19.90.10. Using that code number, we can search Dataweb for export data.

Companies, especially small businesses, often need information about conducting business in foreign countries. The International Trade Administration helps U.S. companies with exporting. Its country commercial guides offer information about the business and political climate, trade regulations and restrictions, and information for business travelers, among other things. The Office of the U.S. Trade Representative produces reports about the status and effects of various trade agreements to which the U.S. is a party.^[9]

The Bureau of Labor Statistics (BLS) collects data that feeds into economic forecasts. Its press releases are eagerly awaited by analysts who are looking at trends in prices, employment and unemployment, or inflation. Major economic indicators, including the Consumer Price Index (used to measure inflation), Producer Price Index, Employment Situation, and many others are issued periodically. BLS also publishes area wage surveys to measure wages in various industries in different metropolitan areas and states. BLS’s Consumer Expenditure Surveys collect data on consumer characteristics and expenditures. A variety of data tools are available to access the data.

Patrons sometimes need 30 years of economic data to examine trends or compare to other indicators over time. ICPSR and other data sources contain downloadable data sets. If you’re looking for economic publications, FRASER has digitized long runs of historical and current publications such as Bureau of Labor Statistics bulletins, which cover a wide variety of topics.

You might think that consular reports would involve statistics about visas issued, diplomatic relations, or similar matters. In fact, employees of consulates throughout the world have promoted U.S. commerce by collecting data on local industries and markets for decades. Consular officials produced reports on industries and markets in foreign countries in order to promote U.S. business interests. These reports provide very detailed historical information about industries and commerce such as leather tanneries, blacksmiths, bicycles in China, or interisland travel in the Pacific. Figure 1 shows an example of a report prepared by the Bureau of Foreign and Domestic Commerce from information collected by consular officials.

 

Publicly traded companies file 10K (annual) and 10Q (quarterly) reports of financial data as required by the Securities and Exchange Commission (SEC). These reports, and many others, can be found in SEC’s EDGAR database, which covers the mid-1990s to the present.^[10]

The Bureau of Economic Analysis (BEA) is another federal agency that collects and publishes economic data including GDP for the U.S. and states, income, consumer spending, personal savings, healthcare, and purchasing power. Most data tables can be downloaded in Excel format.

The Small Business Administration (SBA) exists to administer programs to support small businesses. It offers counseling, research assistance, loan programs, disaster assistance, and support for businesses that want to bid on federal contracts. Among its programs and services are the following:

• Small Business Development Centers serve local small businesses and provide market research assistance and information about loans, grants, disaster assistance, and other programs. The Hawaiʻi SBDC includes the Hawai‘i Business Research Library, which provides research assistance for business development.
• HUBZone is a certification program for small businesses in underutilized business zones that allows qualified businesses to bid on federal contracts.
• SBA also provides export assistance and helps veterans, women, and minorities develop their small businesses and acquire certification to do business with the government.
• Statistics such as the dollar amount of disaster loans can be found on the website of each headquarters office or in the Agency Financial Report.

Census Bureau surveys related to business are discussed in Chapter 7.

Transportation

Roads, airports, bridges, ports, and other transportation facilities and infrastructure are especially confusing for patrons because they may not know which entity controls them. Generally, the federal government controls interstate travel and produces statistics on the highway system, airports and air travel, rail transport, and shipping. The U.S. Department of Transportation’s Bureau of Transportation Statistics is a gateway to national statistics. Accident data and reports of accident investigations are available from the National Highway Traffic Safety Administration, the Federal Aviation Administration, and the Pipeline and Hazardous Materials Safety Administration.

States or cities maintain statistics about state- or municipality-controlled roads and modes such as ferry systems, as exemplified by the statistics page for Washington State Ferries. States are required to report some statistics to federal agencies. For instance, the Federal Highway Administration publishes statistics about the condition of bridges in each state based on data reported by the states according to National Bridge Inspection Standards.^[11]

The Census Bureau also collects transportation-related statistics in its surveys, including commute times and local employment dynamics, which explores where workers live and where they travel to work. The U.S. Coast Guard, which does not have a centralized statistical branch, maintains statistics about shipping in coastal and inland waterways, maritime law enforcement, numbers of boats, marine pollution, and boating accidents, among other things.

Education

The National Center for Education Statistics produces some data for states and school districts, accessible through its tool Search for Public School Districts. It also publishes data about the number of schools, employment in education, school finance, and other characteristics. Some of this data is reported by states, which compile data on educational achievement, enrollment, employment, and other indicators. Similarly, school districts publish statistics about student demographics, achievement, and other characteristics.

One of the most commonly sought statistics is the number or percentage of students eligible for free or reduced-priced lunches, which is considered a key indicator of poverty, and the number of students who actually get free lunches. These statistics are usually maintained by school districts, and you can find state-level data through the U.S. Department of Agriculture, which administers the school lunch program.

The National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation tracks trends in STEM education and publishes data on STEM education, advanced degrees by institution, and employment statistics in science and engineering fields. It conducts several surveys of universities and students and makes PUMS available through its website.^[12]

Commonly sought historical statistics include educational attainment and literacy, which can usually be found in state board of education reports or in StatAb. Educational attainment is also recorded in Census data.

Agriculture, Forestry, Fisheries

The National Agricultural Statistical Service (NASS) collects and publishes statistics on state agricultural production, farm animals, acreage in production, and farm employment. Much of the data is derived from the Census of Agriculture, a survey-based data collection that is specific to each state. For example, the Hawaiʻi state statistics don’t include berries because the most common types of berries, such as blueberries and raspberries, are either not grown in the state or the production is minimal. Be aware that, like the Census of Population and Housing and the Economic Census, the Census of Agriculture is subject to data suppression to protect privacy. In addition, the data is only available at the state or county level.

The National Agricultural Library (NAL) hosts the Ag Data Commons, a repository for USDA-funded research data. Researchers can find data by topic or through full-text searching.

The Economic Research Service (ERS) publishes a wealth of data about agricultural commodities, aquaculture, exports and imports, rural development, food security, and other topics in agricultural economics.

Energy and the Environment

The Energy Information Administration publishes statistics on energy generation, use, and prices for the nation and states. State energy offices like the Utah Office of Energy Development compile similar statistics related to production, consumption, and energy development.

Water can be a confusing topic when one is searching for statistics. Water resources and consumption are monitored by the US Geological Survey (USGS), which produces reports for all of the states. The USGS also publishes data on flooding, volcanoes, and earthquakes. Water quality data is reported by local water systems to the Environmental Protection Agency (EPA) and is published in the Safe Drinking Water Information System. EPA also issues the Toxics Release Inventory (TRI), a report of toxic chemicals released by industrial facilities. EPA provides several data tools to examine TRI data.

The National Centers for Environmental Information (NCEI) collects temperature, pressure, rainfall, and other climate data. Over 100 years of data is available online for states and local areas. Note that access to some reports is free only if you are connected through a .edu domain. NOAA’s Climate.gov includes climate-related datasets and maps.

The U.S. Fish and Wildlife Service (USFWS) administers federal wildlife refuges and the Endangered Species Act in collaboration with state agencies. It publishes statistics about species health and recovery, hunting, importation of animals, fishing, and other topics. When it comes to marine species, the National Marine Fisheries Service publishes statistics on fisheries, marine mammals, and other ocean life. State departments of fish and wildlife typically produce statistics on game, endangered species, and invasive species like the Little Fire Ant or zebra mussels.

Criminal Justice

The FBI produces a number of statistical reports about crimes. The Uniform Crime Reports (UCR) Program consists of four data collections: The National Incident-Based Reporting System (NIBRS), the Summary Reporting System (SRS), the Law Enforcement Officers Killed and Assaulted (LEOKA) Program, and the Hate Crime Statistics Program. The data is available through the Crime Data Explorer (CDE). The FBI’s main statistical publication, Crime in the United States, is compiled from statistics provided by states and covers broad categories of violent and property crimes. The FBI also compiles reports on specific types of crimes such as active shooter incidents and financial crimes. The Bureau of Justice Statistics is the gateway to statistics about criminal characteristics, prisoners, types of crimes, law enforcement, and other topics. Statistics on crime, criminals, and prison populations at the level of detail desired by patrons are often more difficult to find than most other types of statistics. Bear in mind that there is a federal criminal justice system with federal prisons, and parallel state systems. The federal Bureau of Prisons maintains statistics on its prison population, while states maintain statistics on state prisons.

Many states such as Texas and California have state police agencies that issue statistical reports. For instance, the Texas Department of Public Safety issues reports on crime that feature data supplied to the Department of Justice to be included in Uniform Crime Reports. Municipal police departments have varying amounts of information on the web, and some police departments have fallen behind on reporting to the Department of Justice.

Health

The primary resource for federal health statistics is the National Center for Health Statistics. It provides access to vital statistics for the U.S. and states along with data from national health surveys. In addition, it produces specialized reports such as Viral Hepatitis. Health, United States is the annual statistical compendium of health statistics. Morbidity and Mortality Weekly Report contains up-to-the-minute reports about public health issues and disease outbreaks. The Substance Abuse and Mental Health Services Administration publishes statistics about the prevalence of substance abuse and mental health disorders as well as the utilization of behavioral health services.

The Census Bureau collects data on persons with disabilities through the American Community Survey and the Survey of Income and Program Participation. The Current Population Survey collects information about fertility. These three surveys also collect data about health insurance coverage. 

Military and Veterans

Although the Department of Defense has a data website, https://data.defense.gov, it points to datasets posted on Data.gov. Detailed statistics on military personnel and veterans can be difficult to find. Patrons want both current and historical information about numbers of service members, casualties, ethnicities, and service in particular conflicts or locations. This information might not be readily available online and often requires some sleuthing. The Department of Defense used to publish Atlas/Data Abstract for the United States and Selected Areas, but it is no longer being issued. Military One Source offers interactive dashboards and infographics detailing demographic data about active duty and reserve servicemembers back to 2017. Congressional hearings, especially on defense appropriations, can be fruitful sources of statistics about military topics. Detailed information about troop strength and military facilities may be restricted.

The National Center for Veterans Analysis and Statistics produces reports about the veteran population in the U.S. and states, pensions, healthcare utilization, and other information. The Census also collects data about veterans.

 

 

It happens fairly often that we cannot find the data requested by the patron in any published sources. In these cases, the patron might have to contact the agency directly to request the data, possibly through a FOIA request. Another approach is to find an expert who may possess unpublished data. Journal articles and dissertations can identify these individuals. Finally, our colleagues have often developed guides to statistics in their areas of interest. For example, a librarian at the Naval Postgraduate School created an extensive set of guides that includes military statistics.

Historical Statistics

In addition to StatAb, there are several other sources for historical statistics. Historical Statistics of the United States is a compilation of statistics about the United States that covers the period 1610-1970. During the 19th and early 20th centuries, the annual reports of federal agencies were published as congressional documents. These annual reports are packed with statistics about everything from legumes to lighthouses. Congressional publications are an often-overlooked source of statistics. In the course of budget allocations and program review, Congress is often provided with statistics by federal agencies and programs. Special reports submitted to Congress may also contain statistics. For example, a 1921 report on Japanese immigration includes statistics about Japanese and Chinese immigration, land ownership by immigrants, and related topics.^[13] Now that many of these documents are available in HathiTrust and other repositories, it is relatively easy to locate historical statistics.

Insider’s Library

Andriot, J. L., Andriot, J., and Andriot, L. (1998). Guide to U.S. government statistics. Documents Index.

John Andriot was an indefatigable chronicler of U.S. government publications. This guide, updated by his surviving family members, was an attempt to list all of the statistical works of the federal government.

Schmeckebier, L. F. (1925). The statistical work of the national government. Johns Hopkins Press.

This important resource for historical statistics explains the major data-gathering programs and statistical series of the agencies that existed at the time of writing.