Null Results, No Headlines


| 12/9/2014 4:04:00 PM


Many scientific studies have been so embraced by the media, they’ve become facts we often regard with, “Well, duh.” Consider bilingualism: It’s the full-proof edge! Hire a Spanish-speaking nanny! Pop in Rosetta Stone and double your salary!

Despite it being an invaluable skill, it’d be wise to hold off on the “duh.” A new metastudy suggests that there’s a publication bias to what scientific journals choose to headline—ones that eventually make their way through the mainstream media and into the public’s bank of “common sense” facts. That’s because there’s more meat to a headline that claims there’s a link between two things (*blank* causes cancer, for example) than a study that finds the same research inconclusive or that denies the connection entirely.

One research team from Edinburgh University looked at 104 papers on bilingualism that were presented at academic conferences (typical venues for researchers to receive edits before attempting to get published in peer-reviewed journals). Despite there being an even split between research that either supported or questioned the advantages to bilingualism, only 36 percent of papers that challenged the popular notion got published, compared to 63 percent that supported it.

But publications are not the only ones to blame; another study looked at how often researchers reported their null results. One Stanford team analyzed projects run through the National Science Foundation’s Timeshare Experiments in the Social Sciences (TESS), which keeps records of all experiments it’s conducted. The team could then figure out what has yielded positive or negative results and if those findings have been published, written but never published, or never written at all. While they found that publication bias is, indeed, real, they found that researchers aren’t submitting their null results, either. Of the 47 negative outcomes in the TESS data, only 38 percent were even written. Of the 170 positive results, roughly 93 percent were reported. 



Neil Malhotra, co-author of a new study on publication bias and a professor at Stanford’s Graduate School of Business, said it’s important for researchers to at least write down their null results so that other scientists can be aware of them. But it’ll take institutional changes, he said, such as new journals dedicated to negative findings so scientists feel encouraged to report these results.



Pay Now Save $5!

Utne Summer 2016Want to gain a fresh perspective? Read stories that matter? Feel optimistic about the future? It's all here! Utne Reader offers provocative writing from diverse perspectives, insightful analysis of art and media, down-to-earth news and in-depth coverage of eye-opening issues that affect your life.

Save Even More Money By Paying NOW!

Pay now with a credit card and take advantage of our earth-friendly automatic renewal savings plan. You save an additional $5 and get 4 issues of Utne Reader for only $40.00 (USA only).

Or Bill Me Later and pay just $45 for 4 issues of Utne Reader!




Facebook Instagram Twitter