Taxpayer-funded research has revolutionized every aspect of our lives. Although the US has fallen to 10th place among nations of the world in terms of percent of gross domestic product spent on research and development (American Academy of Arts and Sciences, 2014), we nonetheless support a biomedical research enterprise that is unparalleled. This model of public funding for works that support public good is very much in line with Adam Smith’s teachings in An Inquiry into the Nature and Causes of the Wealth of Nations, published in 1776. Although typically invoked for his revolutionary idea that a free market regulates itself, Smith also recognized that governments must serve the needs of the population as a whole, because no individual or group has incentive to distribute resources that do not benefit them directly and immediately. Thus, Smith argued that governments should protect worker wages, regulate banks, issue patents, set educational standards, and control disease, among other things. The economic argument for public funding of biomedical research makes it one of the few nonpartisan issues currently supported by conservatives and liberals alike.
As financiers of biomedical research, the public has a right to demand accountability for their investment. There can be no accountability without access, and the National Institutes of Health (NIH) now requires that all research articles arising from NIH-funded work be made available to the public no more than one year after publication (NIH Public Access Policy). Yet the public cannot evaluate most scientific publications. Indeed, even expert reviewers can differ broadly in their judgment of a given article. Because scientific understanding evolves over time, our evaluation of previously published work can change dramatically. The complexity of human physiology and the need to reinterpret earlier studies based on new ones make keeping track of any given field an intensive process. The lay public, who have neither the time nor the training to evaluate the primary literature, may thus come to doubt the capacity of scientists to establish facts.
Our federal government “of the people, by the people, for the people” (Abraham Lincoln, Gettysburg Address, 1863) allocates public resources through a process that relies on people to direct their elected legislatures to follow public will. People, and therefore Congress, want to know that the money spent on biomedical research is being well spent. Increasing reports on the retraction of scientific papers, as a result of fraud or careless work, have fueled public concern, and the apparent capriciousness of science has created doubt in the minds of many about whether scientists are competent and trustworthy. Physicians who take large sums of money from the manufacturers whose products they prescribe or implant, pharmaceutical companies that hide unfavorable safety and efficacy reports, and government regulators who allow contaminated medications to be distributed across the country have all damaged the reputation of the biomedical research enterprise in its entirety. The public outcry for scientific accountability is understandable and legitimate.
This is the context for the current Principles and Guidelines for Reporting Preclinical Research. The principles are that studies should be well designed, properly analyzed, and reported with sufficient detail that they can be reproduced (see Landis et al., 2012; Collins and Tabak, 2014). Although very simple, these three principles are fundamental to research that will stand the test of time and regain public trust. As such, The Journal of General Physiology, along with the other journals of The Rockefeller University Press, is proud to endorse them. Our long-standing commitment can be seen by examining our publication policies, including the absence of arbitrary limits to description of methods. We take our responsibility as a community of editors, authors, reviewers, and readers of the scientific literature very seriously, and we will continue to experiment with new ways to earn and enhance the public trust.
It would be all too easy to anoint ourselves a success as a result of signing on to the Principles and Guidelines for Reporting Preclinical Research. Having pledged our allegiance to be better conservators of the literature, we could then issue new author and reviewer checklists, mandate statistical guidelines, and bask in the glow of our own integrity. Yet, as Voltaire wrote, with great power comes great responsibility (Œuvres de Voltaire, Volume 48, 1832). We hold the health and well-being of current and future generations in our hands; surely we must do more than sign a document, declare the problem solved, and move on. There are real, pervasive problems that undermine the integrity of the scientific literature. We believe it is our duty to name them and address them.
In some ways, our current methods of evaluating scientific merit have created incentives to play fast and loose to build a publishable story. This can be best illustrated by examining scientific success in terms of natural selection. Each generation of scientists trains the next, allowing each of us to reproduce by training our scientific offspring. The more successful our offspring are in establishing themselves as successful scientists, the more likely we are to have our scientific style and views expanded in the population. Reproduction is expensive in the laboratory as in life, and it requires significant financial resources. We must procure grant money to keep our laboratories running and train more people. We will be judged worthy of more grant money only if we show that we used previous grant money productively by publishing many papers. Papers will also help our scientific children establish their own laboratories and begin the reproductive process anew. Success is always about power and money. Papers are currently an absolute prerequisite to the accumulation of both power and money in the academic world. The incentives to publish are thus enormous, much greater than the incentives to be careful.
Rigorous work that validates a hypothesis, even if the hypothesis was supported by other evidence in the literature, truly advances a field because definitive results provide a strong foundation upon which the next, higher level questions can be explored. A well-designed set of experiments, carefully and thoroughly executed, analyzed in the most appropriate way, and explained in sufficient detail so that it can be repeated is essential to scientific progress. Such rigorous work should be our ideal, as it would create a literature of high integrity that stands up over time and provides real value to the public. However, the current emphasis on “high profile” work that leads to a “conceptual advance” ignores the importance of rigor and threatens to undermine our values, as well as our value to the public.
As the number of methods used in any one paper grows, careful reviewing becomes more challenging. If it takes a group of several laboratories to combine expertise to put together a story, how could any two or three scientists appropriately review the technical aspects of all those techniques? How do we ensure that a work is rigorous and deep rather than broad and superficial? The real answer is that it isn’t possible to properly review multi-method papers that may span studies of single molecules through organismal behavior. It would take an army of reviewers to match the army of authors. But the choice between good work and good review represents a false dichotomy. If we stopped focusing on papers that change the world and started to value work that is rigorous and establishes truths, we would have a more robust review process and would restore the integrity of the scientific literature.
The current focus on reproducibility is, to some extent, a distraction from the impact factor wars in which decisions on hiring, promotion, and funding are made on the basis of publication in high profile journals, rather than on the quality of the work. The reproducibility problem is just a symptom of the larger problem that we no longer take responsibility for reading papers to evaluate their content. The San Francisco Declaration of Research Assessment (DORA; http://am.ascb.org/dora/) directly addressed this problem and suggested several actions we can all take to reclaim our scientific judgment and, perhaps one day, the public trust. Although DORA was signed by many, including myself, it has yet to break through the barrier of fear as required for any grassroots movement to succeed. Supporting nonprofit journals run by scientists, such as JGP, is a powerful antidote to the focus on quantity over quality and prestige over insight.
In conclusion, although the Principles and Guidelines for Reporting Preclinical Research represent a reasonable collective statement of values, they must be part of a larger cultural shift to advance meaningful change. All of us involved in peer review of manuscripts and research grants and in hiring and promotion decisions will have to more truthfully represent the scientific enterprise to our sponsors, the taxpaying public, as the slow, methodical work it is. This representation is fully compatible with the immense creativity, brilliance, and ambition that has made biomedical research the source of the technological, political, and social innovations that drive the engine of our economy and improve the health of so many. Accountability means, to paraphrase Isaac Newton, acknowledging that each of us stands on the shoulders of those who came before. All breakthroughs are built upon a foundation of rigorous, careful work. We must change the incentives to reward building with strong, sound bricks, or the enterprise will fall like a house of cards.