Scientific publishing faces challenges and changes induced by the same forces which have reshaped the news industry and are reshaping the traditional publishing industry. You could say that the Internet changes every information-centric industry, but some industries are slower to change than others.
The National Institutes of Health has a “request for information” (RFI) out for comments on “interim research products” including things like preprints. Comments are due Nov. 29, and I encourage anyone with interests in preprints and the future of disseminating scientific information to submit a response.
Here, I want to briefly give a few of my thoughts relating to the future of scientific publication, then include some of the content I sent to the NIH (which I slapped together rather quickly).
Changes in scientific publishing/credit:
In my view, there are several major things which need to change about scientific publication, and I believe the Internet is helping to make these changes happen — but some of these changes have details that need working out, or threaten the existing order, so they have been slow in coming. Here are some of the changes I see coming or hope are coming:
- Publications need to be judged individually based on their impact, rather than based on the impact of the journal they are published in.
- Papers should be available and cite-able once finished, even prior to peer review; peer review serves to help improve articles and provide a type of recommendation, but it’s not magical
- The distinction between different research products (research papers, review papers, papers or brief reports on null results or failed experiments, tutorials, data sets/databases, software tools, etc.) needs to blur somewhat; all should be versioned (with all prior versions accessible), updateable, citeable, peer reviewed when appropriate, and valuable in terms of academic or other credit.
The first of these is where I hope things are headed in terms of credit, and I’ve already been acting like this for a while myself. I tend not to worry that much about the “quality” of the journals I publish in, with the thinking that if the work is important enough, it will be recognized as important. Sure, papers I published in J. Med. Chem. get cited more, but it seems likely to me that that effect will diminish over time. In the old world, we read only certain journals because this was where all of the “important” science would appear and searching for relevant work required flipping through the pages of journals rather than a quick Internet search. In the new world, we can easily monitor far more journals (such as via RSS feeds) plus we can easily find out about other important work via Twitter, various customized alerts, exchanges with colleagues, monitoring work of key players in the field, etc. And work is far more discoverable. It seems to me that it will only get easier to find the papers we need to find, regardless of where they were published.
It’s also clear that work ought to be available (and cite-able) far earlier, once finished and submitted if not before. One argument for this is that review at the highest impact journals can be an onerous and slow process. However, I think the larger argument is “why not”? Once it’s done, if it’s posted in a permanent, citeable way as a preprint, it establishes priority for the work and can begin drawing interest (and people can begin to build on it or verify it) advancing the field. So, why wait? But complaints about the timescales of publishing are relevant as well. Far too often I’ve sent a paper to a high profile journal only to have it sent back for minor stylistic revisions; I make the edits, re-submit, and then have it rejected prior to review (after a week or two of delay!) because it will not be “of general interest to readers of the journal” or similar reasons. I send it somewhere else and it proceeds through the review process without difficulty and picks up 30-100 citations in the next several years, from readers who certainly read the first journal. It seems far preferable to just get it out in the open where people can begin to use and read it, and let the vagaries of the review process happen later.
I also believe we need to reduce the distinction between a formal “publication” and these other means of disseminating research products. As my group moves towards making more of what we are doing available on GitHub as we do it, our code development and some of our results become available to the public sooner. When exactly does this constitute a “research product”? In some sense, it is a research product as soon as we have it up online, but especially so if we make it versioned and cite-able, perhaps using a tool like Zenodo to automatically give each version a new DOI. This also helps with the issue of null results or “failed experiments” — typically, these tend to never be published, despite the fact that we know that null results are quite valuable. Well, allowing reports of these results to be posted even without formal publication reduces the barrier to getting this important information out.
I have a lot more thoughts on these issues, but lack the time to write more on it at present, so I’ll just jump to the somewhat stream-of-consciousness response I sent the NIH:
1. Types of interim research products your or your organization create/and or host:
We are a computational group, and typically make use of several established types of interim research products:
* Preprints (publications made available in pre-publication form)
* Databases consisting of calculated and/or experimental values, updated as new calculations and/or experiments are done
* Test sets that are maintained/curated
* Open source software tools, even those under development and not yet published on (note that this means in some cases, people can be fairly clear on what the contents of our publications will be long before we are producing publications)
We also experiment with less-established types of interim research products:
* Online tutorials, test sets, and materials at Alchemistry.org and other wiki-like sites, as well as at GitHub
* “Perpetual reviews”, update-able reviews that we make available as preprints but then continue updating to stay current
* In some cases, we post grant proposals online when they detail a project which will impact our community, when posting these will not cost us a competitive advantage and may help the field make progress; this has some overlap with the RFI’s “preregistration of protocols” point.