Publons, Social Media & Science: an Interview with Andrew Preston
As part of the interviews I conducted for my European Science Editing Social Media article (see previous blog post), I spoke to Andrew Preston over email, but we concluded the conversation too late to make it into the final version of the article. Andrew is a chemist from Victoria University of Wellington, New Zealand, and the founder of Publons – an online platform at the forefront of one of the more interesting avenues of open/social science in acknowledging the contributions of reviewers in the research publishing process.
Publons allows reviewers to keep a record of the reviews they perform for journals, either as a simple reference or with the full text of the review. It is also a platform for posting open, post-publication reviews. There are points and rating systems to award reviewers for their activities and the community evaluation of their contributions. It is an enormously interesting project, and a much needed one at a time when many academics are calling for the value of reviewers to be recognised. I spoke to Andrew about the role social media is playing in science from his perspective and how it has influenced and affected his work with Publons. I post the interview in its entirety here, as Andrew offers some interesting comments, and resources to follow up on. ----------------------------------------------------------------------------------------------------------------
Duncan Nicholas: As the founder of Publons, I am sure you must believe that there is a great deal of value in capturing the activity of papers online, and utilising the collaborative social aspects of the web.
Andrew Preston: Absolutely. The vast majority of all scientific discussion is never indexed or brought online. Our mission is really to save this discourse from waste and make it available online, starting with traditional peer review.
I should point out that one of our founding beliefs is that you will only see high quality online activity in large quantities if we can put the correct incentives in place to make this happen. In other words, we need to recognise researchers for their online contributions. This is much more important in research than other areas because it's the research outputs you can measure and point to that literally determine your career progression.
DN: How do you feel any form of Social Media has effected different parts of the academic publishing you are involved in?
AP: I'd say social media has been a net positive for the community but I get the impression that "conventional" forms of social media (e.g., Twitter) are only used by a minority of relatively vocal researchers. Usage is growing but still less than 10% of all articles ever get a mention on Twitter. I get the feeling that we'll need to invent new tools, social conventions, and incentives, before we see the broad range of contribution that we see in traditional publishing (where just about everybody takes part).
DN: Do you feel it plays any part in where authors choose to publish their work?
AP: For a few, but not most. From what I've observed, most authors choose to publish in the traditional journals of their field or emerging journals with growing Impact Factors. There are of course problems with this, but that's a different topic for another time.
DN: Have any aspects of the peer review process migrated to social media in any way?
AP: Absolutely. You often see short post-publication comments on Twitter, for example. However, as I mention above, these comments only tend to cover a small fraction of published articles.
DN: Have you ever seen incidences of commentary through social media influencing change in an article?
AP: Yes. For example, I believe that a number of article retractions have come from anonymous comments on PubPeer. This is probably good for science and there is a place for it but I do think we need to ensure focus on developing a culture of improving research with positive contributions rather than witch hunts.
DN: Have there been any significant reviews or discussions on Publons that you would like to highlight?
AP: I'd actually like to highlight the first publication, which was our manifesto of sorts. It highlights the different kinds of reviews and discussion we hope to encourage and recognise on Publons.
I was also a big fan of the reviews of Krzystof Gorgolewski's paper. He posted this paper as a pre-print to the bioRxiv and then solicited reviews from the community. Based on their suggestions he and his co-authors produced a new version of the manuscript and submitted it for publication in Frontiers in Neuroinformatics. Pretty cool!
DN: Do you think any part of the readership of academic research has changed as a result of its presence on social media?
AP: There are some forms of activity that have picked up to varying degrees with the advent of social media. One of its killer apps has been to provide access to paywalled articles. This seems to have driven the growth of things ranging from the #icanhazpdf meme all the way to networks like Academia.edu and ResearchGate.net.
That said, I suspect that most social media developments haven't really created much of a new audience for science. Most on-the-ground science is so focused on the details that only those who are heavily involved in the research already can fully engage with it. Pushing the boundaries of human knowledge is (almost by definition) a niche topic.
DN: Do you think social media is a successful influence in raising the profile or awareness of research? If so, who do you feel this awareness raising has been amongst? Can you make a distinction between peers and colleagues, and/or a wider non-scientific audience?
AP: I think that the primary effect of social media, blogs, and the like has been to raise awareness of the fact that it is possible to publish content openly online. DN: Has it affected the discourse around particular papers, or of science in general? And has it played an integral part in the new Open Access movement – or vice versa?
AP: This, I suspect, has indeed been a big driver of uptake in OA. On the flipside, I think the fact that peer review is critical to trust in published research has been largely missed.
DN: Do you feel social media has effected, or facilitated any forms of research itself, such as peer-communication, collaboration, or data gathering?
AP: Absolutely. My favourite example is probably the Polymath Project, which is a collaborative effort between mathematicians to discuss and solve open problems online. It started as a blog post by Tim Gowers and has generated some new and interesting results.
That said, it's early days yet. There is so much offline discussion that would be even more valuable if made available online. That's a big part of what we're trying to achieve.
DN: What do you feel the differences are, between viewing social media metrics as a mark of scientific quality or as a measure of value? And, how does this compare to using them as a measure of social interest?
AP: The big problem with using social media as metrics for a mark of scientific quality or impact is that 90% of papers don't (yet) get mentioned on social media. There is a pretty strong power law in play, and those papers that get a lot of attention tend to get it for the wrong reasons (something to do with sex in the title, a retraction, etc.). This is essentially the problem that peer review solves, since it has almost universal coverage.
I wrote some detailed thoughts in my review of the HEFCE Metric Tide report
DN: Have you ever discovered anything through social media you may otherwise have been unaware of?
AP: Not yet, but fingers crossed!