• Jeff tries to explain how government works to readers of Science.

By the Numbers: NSF's Science Indicators Offers Answers, of a Sort

6 February 2014 4:30 pm
Leading indicator? New statistical report from the National Science Foundation includes data on how often science and engineering is taught. Here, students in Oregon learn about bridge engineering.

Oregon Department of Transportation

Leading indicator? New statistical report from the National Science Foundation includes data on how often science and engineering is taught. Here, students in Oregon learn about bridge engineering.

True or false?

  • More than one-third of U.S. elementary students learn no science in a typical week.

  • The U.S. science and engineering workforce grew by 10% between 2008 and 2010.

  • The number of authors of scientific papers from U.S. universities increased by two-thirds between 2010 and 2012.

These three questions are based on information in the latest edition of Science and Engineering Indicators, a biennial collection released today by the National Science Foundation (NSF). And the answers—1) True; 2) True, but with a big caveat; and 3) It seems unlikely—point to both the strengths and weaknesses of the massive report.<--break->

The 2014 tome describes in great detail the state of U.S. science and its relative ranking among global competitors. It draws from surveys conducted by NSF or work that it has funded, as well as data from other sources that NSF’s statistical shop, the National Center for Science and Engineering Statistics (NCSES), has vetted.

Once again, the report points to the continued scientific progress by China and other Asian nations as measured by  any number of indices, from high-tech manufacturing to research spending to training the next generation of scientists and engineers. “The longstanding dominance of the United States continues to erode,” says Dan Arvizu, chair of the National Science Board, the presidentially appointed oversight body that issues the report. “Other countries have been building up their research capacity very rapidly, and this report shows how fast this new world is arriving.”

With 600 pages of text and graphics and a 939-page appendix of tables, Indicators provides lots of yardsticks by which to measure those changes. At the same time, readers may want to exercise some caution to avoid leaving the statistical buffet with indigestion—or with egg on their faces.

Science teachers

The answer to the first question comes from the 2012 National Survey of Science and Mathematics Education. The survey of 7752 teachers at 1504 elementary and secondary schools around the country—the fifth since 1977—is designed to assess the background and experience of teachers, what goes on the classroom, and the resources available to them.

The answers show that science is a “forgotten step-child” to math and reading in the lower grades, according to the team at Horizon Research Inc. in North Carolina that conducted the NSF-funded study. Several metrics suggest that elementary science instruction is on shaky ground: Those who teach it are more likely to be novices; science receives only 40% the time given to math and less than 30% spent on reading; and teachers feel only half as well prepared to teach science as math. Engineering is almost nonexistent, the study notes, with only 4% of elementary school teachers feeling very confident about tackling the subject.

Scientific workforce

One apparent bright spot in the 2014 Indicators is the recent rapid growth in what NSF calls the scientific and engineering (S&E) workforce. The report says the total was 5.40 million in 2010, up from the 4.88 million reported for 2008 in the 2012 edition of Indicators. But the real increase is probably closer to 3% or 4% rather than 10%, says Beethika Khan of NCSES and lead author on the workforce chapter.

That’s because the 2008 number was in all likelihood an undercount, Khan explains. It was based on people who self-identified as members of the scientific workforce in the 2000 decennial census and who NSF then tracked for the rest of the decade through two surveys, one of recent college graduates and one of doctoral recipients. But those surveys don’t capture those who didn’t earn a U.S. degree, that is, many foreign-born scientists and engineers. Nor do they include people working in the field without any science and engineering degree.

In contrast, the 2010 data draw upon the American Community Survey (ACS), the successor to the long form of the census. That survey is done annually, and offers a much more accurate snapshot of the workforce at any given time. NSF used ACS for the first time to help calculate the size of the scientific workforce in 2010. “Our use of the ACS as a frame has improved our results,” she says.

(This year’s report also takes a first-ever look at those in the S&E workforce without a bachelor’s degree. It finds that blacks and Hispanics are overrepresented, and Asians underrepresented, compared with the demographics of those with S&E degrees working in the field. The non-degree holders are also much more likely to be working in computing and information technology—69% of all non-degree holders work in that field, compared with 44% of all college-educated S&E workers.)

Author, author

The final question pertains to cross-sector collaboration, which the report calls “one of the most striking changes in the U.S. S&E landscape in recent years.” But NSF’s attempt to highlight how scientists are “cross[ing] boundaries to enter previously unfamiliar territory” appears to have taken a wrong turn at some point.

The questionable figure appears in the report’s overview. It purports to show a steep rise in the number of academic authors on scientific papers and the number of authors per paper.

Each name is counted as a separate author, so the absolute count is obviously inflated. But the real problem is that both numbers, which have grown at a steady rate since 1988, suddenly spike in 2011 and 2012.

What could have caused U.S. academics to suddenly become so much more prolific—and collaborative? The statistical answer, in a word, is CERN. More specifically, the Large Hadron Collider (LHC) at Europe’s high-energy physics laboratory near Geneva, Switzerland.

“One sees more huge recent collaborations in physics,” says NCSES’ Robert Bell, who oversaw the 2014 Indicators. An unofficial analysis by NSF staffers, he says, found one paper with more than 3000 authors in 2010, 45 such papers in 2011, and 88 in 2012. Before 2010, he notes, there were none.

The pattern of greater collaboration has been evident for the past 2 decades, he notes. In 2012, only 55% of papers in all fields had four or fewer authors. That compares with 83% in 1988, 74% in 1996, and 65% as recently as 2004. The LHC has distorted the picture, however, by making it seem as though the entire scientific community has suddenly embraced massive co-authorship.