Is Academic Research a Dead Man Walking?
On March 4, 2015 at 1pm ET (timezone converter), #hcsmca will be co-hosted by OpenLab (@), a design and innovation shop dedicated to finding creative solutions that transform the way health care is delivered and experienced, and the Teaching and Learning Institute at Holland Bloorview Kids Rehabilitation Hospital (@). Moderators Shoshana Hahn-Goldberg (@), Post-Doctoral Fellow at OpenLab, Dr. Howard Abrams, Director at OpenLab and Dr. Kathryn Parker, Director Academic Affairs and Simulation Lead at Holland Bloorview wrote the following blog to set the stage for the discussion about innovative scholarship and alternatives to traditional peer-reviewed publications we hope to inspire on this week’s #hcsmca chat.
By Shoshana Hahn-Goldberg, Dr. Howard Abrams, and Dr. Kathryn Parker
An #hcsmca discussion about innovative scholarship and alternatives to traditional peer-reviewed publications
There is a revolution occurring in healthcare. This is partly being driven by need (demographics of an aging, increasingly educated population) and opportunity (distributed health information technology and very rapid innovation cycles).
Is traditional academic research and its currency of peer reviewed publications still the right model to test and disseminate innovative models of care? Especially in complex interventions where the scientific method of holding all things constant except the intervention is unrealistic in the messy real world, negates the active learning and innovation cycle inherent in the innovation process, and risks concluding that no effect means “not necessary” rather than “necessary but not sufficient”.
So there are two separate but related issues that arise from this;
- Is there another model of “proof of concept” that we should recognize or develop in order to assess the validity of the rapid innovation in health care that is necessary, and is in fact occurring among free-living human beings?
- Is there an alternate form of peer review that is better suited to the new requirement for rapid assessment and dissemination of new models of care, or has this academic model already died and just doesn’t know it yet?
Traditionally, research papers undergo peer review before publication, which generally takes about eighteen months to go from submission to publication1, and this estimate does not include the time it takes to prepare a manuscript or that a manuscript may need to be submitted to several journals before it is accepted. The result is that research disseminated through peer-reviewed journals is often several years old. One could argue that the peer-review process is slow, stifles innovation, and lacks transparency2. One could also argue that the peer-review path is not the best way to get the information to all the desired audiences and there are certain types of research where peer review is not necessary. In these cases, there are potentially many other ways to disseminate the research, such as through various forms of media.
There are two main roles of scholarly journals and peer review:
- An archive of validated research in a field – Most people would agree that getting it right, through the care and effort of peer review, outweighs the need for rapid publication when viewing these journals as our archive of knowledge1.
- Communicating research among others in the field – Speed and interactivity are much more important for this role. Peer review may be less important, as experts in a field can make their own decisions about validity1.
Two trends, open access and social media, have the potential to change the peer review process2. Some new models of peer review are developing, such as comment crowdsourcing, where anonymous peer review is replaced with public reviews that can include the reviewer’s reputation (as determined by peers) to weight the review score. Open publishing platforms such as PeerJ and the Journal of Participatory Medicine employ concepts such as invited moderation, post-publication comment, post-publication measures of quality and impact, and community-based review2.
Another publishing route is through preprint sites based on the arXiv model. Information is disseminated rapidly, but papers submitted are not subject to traditional peer review. Mendeley, with over 34 million research papers, also incorporates an academic social network. PaperCritic, which works in tandem with Mendeley, allows scientists to review each other’s work2.
These innovative systems have potential; however, although many have already been around for several years, they are not as widely used as we would expect. With the innovation cycle increasingly rapid, the traditional approach of academic research and publication may not meet the needs of current challenges. Will there always be a place for it?
In a world where increasingly more people are actively participating to improve their health, often collectively, open access methods will be required to disseminate research results.
- T1: What role(s) do social networks play in peer-review, when more people now participate in improving their health?
- T2: What can replace traditional peer-review and provide credit when dependent on academic structures for support?
- T3: When is peer-review not desirable? For which audiences? For which projects? What are better options?
- T4: How does knowledge created through non-traditional research get shared? What is “peer review” within this context?
- T5: What are risks to abandoning peer review? Should e-patients be made aware of peer review versus social media review?
Solomon DJ (2007). The role of peer review for scholarly journals in the information age. The journal of electronic publishing 10(1).
Pickard KT (2012). The impact of open access and social media on scientific research. The journal of participatory medicine 4.