Is Academic Research a Dead Man Walking?
On March 4, 2015 at 1pm ET (timezone converter), #hcsmca will be co-hosted by OpenLab (@UHNOpenLab), a design and innovation shop dedicated to finding creative solutions that transform the way health care is delivered and experienced, and the Teaching and Learning Institute at Holland Bloorview Kids Rehabilitation Hospital (@BloorviewPR). Moderators Shoshana Hahn-Goldberg (@HahnGoldberg), Post-Doctoral Fellow at OpenLab, Dr. Howard Abrams, Director at OpenLab and Dr. Kathryn Parker, Director Academic Affairs and Simulation Lead at Holland Bloorview wrote the following blog to set the stage for the discussion about innovative scholarship and alternatives to traditional peer-reviewed publications we hope to inspire on this week’s #hcsmca chat.
By Shoshana Hahn-Goldberg, Dr. Howard Abrams, and Dr. Kathryn Parker
Shoshana Hahn-Goldberg
Howard Abrams
Kathryn Parker
An #hcsmca discussion about innovative scholarship and alternatives to traditional peer-reviewed publications
There is a revolution occurring in healthcare. This is partly being driven by need (demographics of an aging, increasingly educated population) and opportunity (distributed health information technology and very rapid innovation cycles).
Is traditional academic research and its currency of peer reviewed publications still the right model to test and disseminate innovative models of care? Especially in complex interventions where the scientific method of holding all things constant except the intervention is unrealistic in the messy real world, negates the active learning and innovation cycle inherent in the innovation process, and risks concluding that no effect means “not necessary” rather than “necessary but not sufficient”.
So there are two separate but related issues that arise from this;
- Is there another model of “proof of concept” that we should recognize or develop in order to assess the validity of the rapid innovation in health care that is necessary, and is in fact occurring among free-living human beings?
- Is there an alternate form of peer review that is better suited to the new requirement for rapid assessment and dissemination of new models of care, or has this academic model already died and just doesn’t know it yet?
Traditionally, research papers undergo peer review before publication, which generally takes about eighteen months to go from submission to publication1, and this estimate does not include the time it takes to prepare a manuscript or that a manuscript may need to be submitted to several journals before it is accepted. The result is that research disseminated through peer-reviewed journals is often several years old. One could argue that the peer-review process is slow, stifles innovation, and lacks transparency2. One could also argue that the peer-review path is not the best way to get the information to all the desired audiences and there are certain types of research where peer review is not necessary. In these cases, there are potentially many other ways to disseminate the research, such as through various forms of media.
There are two main roles of scholarly journals and peer review:
- An archive of validated research in a field – Most people would agree that getting it right, through the care and effort of peer review, outweighs the need for rapid publication when viewing these journals as our archive of knowledge1.
- Communicating research among others in the field – Speed and interactivity are much more important for this role. Peer review may be less important, as experts in a field can make their own decisions about validity1.
Two trends, open access and social media, have the potential to change the peer review process2. Some new models of peer review are developing, such as comment crowdsourcing, where anonymous peer review is replaced with public reviews that can include the reviewer’s reputation (as determined by peers) to weight the review score. Open publishing platforms such as PeerJ and the Journal of Participatory Medicine employ concepts such as invited moderation, post-publication comment, post-publication measures of quality and impact, and community-based review2.
Another publishing route is through preprint sites based on the arXiv model. Information is disseminated rapidly, but papers submitted are not subject to traditional peer review. Mendeley, with over 34 million research papers, also incorporates an academic social network. PaperCritic, which works in tandem with Mendeley, allows scientists to review each other’s work2.
These innovative systems have potential; however, although many have already been around for several years, they are not as widely used as we would expect. With the innovation cycle increasingly rapid, the traditional approach of academic research and publication may not meet the needs of current challenges. Will there always be a place for it?
Discussion Topics:
In a world where increasingly more people are actively participating to improve their health, often collectively, open access methods will be required to disseminate research results.
- T1: What role(s) do social networks play in peer-review, when more people now participate in improving their health?
- T2: What can replace traditional peer-review and provide credit when dependent on academic structures for support?
- T3: When is peer-review not desirable? For which audiences? For which projects? What are better options?
- T4: How does knowledge created through non-traditional research get shared? What is “peer review” within this context?
- T5: What are risks to abandoning peer review? Should e-patients be made aware of peer review versus social media review?
References:
-
Solomon DJ (2007). The role of peer review for scholarly journals in the information age. The journal of electronic publishing 10(1).
-
Pickard KT (2012). The impact of open access and social media on scientific research. The journal of participatory medicine 4.
Colleen Young 
Many patients and e-patients are not savvy about the peer-review process and how it differs from social media reviews. While that’s a curating issue best tackled by researchers who can make sense of the clinical research, I think the biggest impediment for lay people is the paywall to access such studies. With the advent of open source data repositories, I think the gated peer review system will someday become obsolete.
There are private researchers who are setting up their own systems and leaving the peer-reviewed world in the dust. Take for example Craig Venter (from the Celera Genomics and Human Genome fame) who has now launched Human Longetivity, Inc., seeking to create the world’s most comprehensive database on human genotypes and phenotypes to address the diseases linked to aging-related human biological decline. This is the sort of disruptive movement that doesn’t slowly contribute to the scientific literature. They are building their own sandbox and accelerating the learning cycle. Happily, I welcome it.
LikeLiked by 2 people
Hi Carmen,
Great to see you here on the #hcsmca blog again. As usual, you contribute valuable observations and resources.
The paywall access to studies is frustrating at best. I would welcome the opportunity to say adieu to the gated peer review system. I also herald scientific recognition and compensation models that encourage collaboration over working in more insular settings that protect ownership.
I hope you’ll be able to join the #hcsmca conversation on Wed. Mar 4 at 1pm EST.
LikeLiked by 1 person
Great comments Carmen. We are caught in a transitional phase where the traditional hierarchical scientific model still controls significant resources of access to research funds and the “credible space” to disseminate results via publications. Distributed information and access by the public is disrupting this but we don’t yet know what is worth preserving, and what should be “democratized”. On the care provision side, some of us are calling this democratization the “Second Stream” of Health Care, potentiated by the proliferation of mHealth. Perhaps there is also a “Second Stream” of health research that can respond more quickly, uses a different standard of “proof”, and is better suited to the complexity of the evolving health care landscape. It is both an exciting and frustrating time.
LikeLiked by 1 person
Great topic – wish I could join in the discussion! Another question: if the traditional model is broken (which I accept), what will create space/time for researchers (including patients) to do the “big thinking” that is essential to system transformation? Virginia Woolf needed a room of her own — so do academics/researchers. The current grant system doesn’t serve this purpose well. But we should think about what could do it better.
Will look forward to the transcript!
LikeLiked by 1 person
I believe that health care and research institutions need to better understand the different roles that traditional academic research, and “Second Stream” research can play. I think there is still a place for the careful, incremental science model. But this approach will not solve large system re-design issues. So where is the “room of our own” to do this? Good question. In a very forward looking system, this wouldn’t be an “either or” question, but a “yes and” solution. Currently, Second Stream is being driven by commercial interests that see the opportunity for Big Data, and some public groups that compete for funds through grants, industry, and donors. There are a very few large institutions that have “Innovation” groups funded internally, but it is not clear yet whether their mandate is to do the “same old” better, or really do it differently.
There is a lot of talk about “system thinking’ and “system change” among grant agencies in Canada, but with few exceptions they seem to be using the same academic vetting model. Lots of system “talk”, not much system “do”.
LikeLiked by 1 person
I believe that it is a matter of “both, and…” and not “either, or”.. The peer review process as currently defined is but one way to define it but there are others…I think of some amazing thinking that is coming from educational researchers such as Marcus Law and Maria Mylopoulos. They argue that the concept of scholarship needs to be broadened beyond the traditional research and knowledge translation frameworks. That scholarship or knowledge building is an iterative, dynamic and collaborative process embedded in practice. The concept of peer review in this way of thinking is that one’s peers have an opportunity to comment on the innovation/or idea from a perspective of evolving it and applying it to different contexts. Again, we should not through traditional peer review processes out but acknowledge that the concept of peer review needs to be expanded.
LikeLiked by 1 person
agree wholeheartedly. The challenge is to be reviewed by peers in sync with very rapid innovation cycles. I’m not sure we know what the limits will be on this, since the new models of peer review are still evolving. The other challenge is how to unlock some of the resources tied up in the traditional academic grant system to be used for innovative review and necessary change. Is using “traditional” and “innovation” in the same sentence a form of oxymoron?
LikeLiked by 1 person