Restoring The Scientific Method to All Its Glory
Take that you doubters of scientific vigor! Yesterday, the session, "Denial, Détente, and Decisions: Fisheries Science at the Crossroads," provided a shining example of the power of the peer review process to root out inaccuracies and refine results, and bring us one step closer to unraveling the mysteries of the universe. Amidst the growing din of skepticism and doubt surrounding the validity of the scientific process, fueled most recently by the slip-ups by the IPCC (which really should have been caught earlier), this story emerges a reminder that the scientific endeavor remains alive and well and worthy of our confidence; it just requires our patience.
Dr. John Lynham, at the University of Hawai'i, presented a fantastic talk about the relationship between catch share management systems in fisheries, and declining fish stocks. This research, conducted with colleagues and published last year, provided the first evidence of a positive effect of catch shares to halt or sometimes reverse declines in fish populations. And while the authors were careful to emphasize that the relationship they found was correlative, and not necessarily cause and effect, their publication caused quite a stir, as catch shares have been fairly controversial in the U.S.A.
Not surprisingly, they received a whole host of criticism. And, as Dr. Lynham carefully pointed out, some of it was extremely valuable, especially the one criticism that turned out to be right.
Dr. Lynham presented the new findings, based on the refined analysis motivated by the thoughts and insights of this critic. The result was that catch shares still do have a positive relationship with preventing fish stock declines, but not quite as strong an effect as the previous results had implied. They weren't wrong, but the new analysis has helped increase accuracy and account for another factor that contributes to the observed relationship.
This is how science is done. It is a constant conversation, an iterative process that allows for one idea to build and shape the next through refinement of the last.
For many outside the walls of academia, the scientific process remains largely a figment of imagination: a fuzzy blurring of Hollywood CSI-style lab detectives mixed with the mad chemist filling bubbling test tubes or the eccentric natural historian filling bug jars of pickled insects. Many haven't heard of "peer-review," a pillar of modern science, and for those who have, my guess is that they don't understand the full scope of the process. Thus, the (flawed) argument that errors in published papers are evidence of a system gone awry; the assumption is that the review process is supposed to catch all errors before publication.
But peer-review is an on-going process whereby the entire scientific community is able to read, review, and then respond to published work. The majority of the time, the first filter of publication catches most of the errors or inaccuracies. But, sometimes, especially with complex systems, the second filter, comprised of the entire scientific community, will unearth a missing piece of the puzzle. More often than not, this piece does not completely negate the findings (though sometimes, it can); instead, it refines the outcomes.
Like the natural world it seeks to understand, science evolves. Making this process known would help alleviate concern about the mistakes within or revisions made to published works and would go a long way towards restoring confidence in the scientific endeavor.
Photo credit: wikimedia
home of the 2010 Science / AAAS blogging contest.
The contest is now closed, but you can see details and participate next year!
For coverage of the meeting by Science staff, see our AAAS 2010 news page.
