Anthea Sutton and Andrew Booth publish paper in JEAHIL

Before ScHARR Towers shuts up shop for the Christmas break, we bring you news of our latest IR publication. Anthea Sutton and Andrew Booth have published an article in the Journal of the European Association for Health Information and Libraries (JEAHIL) on the topic of leadership in the library and information profession. This contributes to a special themed issue on "The librarian of the future" and along with our article has some interesting features on hot topics such as open access and social media. Well worth a read!

Apps to help you be a better mobile academic

A few months ago I recorded the first part of a collection of very short videos to help encourage the growing number of tablet owners to be more productive with their devices in an academic setting.
Since then I have recorded a few more videos that can help academics and students learn how to use these apps to do anything from reference to stay up to date with their favourite journals.
The collection of videos can be viewed on our ScHARRvids YouTube page, below is just a an example of what you can do with your tablet device in higher education.

RefMe App 



Papership App



AudioBoom App

Write Club - The first rule of Write Club is that you do not talk at Write Club

One of the biggest issues my colleagues in ScHARR and Information Resources face these days is that of trying to put concentrated time into writing. For anyone working in a shared office or providing a service, as we do in Information Resources will know, is that all too often interruptions can impact on your flow of work. There are various articles and books published on the issue of modern technology affecting our attention spans. Some argue that it can take about 15 minutes to refocus the brain to one task properly once you switch to another, such as going from writing to reading emails. For the modern student it must be worse with Snap Chats, Facebook updates and text messages all vying for attention when you are trying to complete an important essay. At ScHARR we have tried to address the issue of researchers trying and often failing to finish various writing projects, usually those kind of projects that do not have a strict deadline.

I attended a University of Sheffield hosted writing retreat which opened my eyes to the idea of silent, writing focused events. An event where staff and students are not allowed to access any form of communication, email, Social Media or their phones. Everyone begins by discussing in small groups what they hope to get out of the retreat and sets out a few clear objectives. We then wrote for a solid hour followed by a short break to hunt down coffee, tea and sweets. Another one hour session followed and we finished by discussing what we think we'd achieved and how well we had done. My group, all from ScHARR agreed that it had been a really worthwhile exercise, the silence had given it a feeling of sitting an undergraduate exam. It took me back to the late1990s and sitting in a wood-clad Firth Hall, head down writing an essay about the role of jazz in modern music, fearful of sneezing or needing the loo (OK, not that bad). The subtle peer pressure of writing and not peeking a look at your email was effective enough to make this a worthwhile event. 

In the end I managed to write two thousand words in two hours which just goes to show the kind of output you can have if you have got a topic or objective to write about and the right environment to facilitate that. The format is so simple and effective and has brilliant results. Obviously it is not for everyone, there are those lucky individuals who can stay incredibly focused these days, or do not have the distractions others have. As a result we are running a pilot over our writing week in January by hunkering down in the Library at ScHARR for six, two hour sessions where academics can work with colleagues in total silence and hopefully make some progress on their writing.
I've aptly called it Write Club - based on the cult film, Fight Club. This does not mean academics will strip off and fight in pairs in the underground car park, although this may already happen. Instead they will work on the simple rules of Write Club - that being.
The first rule of Write Club is that you do not talk at Write Club
The second rule of Write Club is that you do not talk at Write Club

Dates for the first Write Clubs have been distributed to ScHARR staff

Happy Holidays from the ScHARR Library team!

We've been trimming the tree and hanging the lights in the ScHARR Library! To all our readers, users, fans and friends, a very Merry Christmas, Happy Hanukkah, and a Joyous Kwanzaa!

We're Hiring!

Do you fancy working in the above building at one of the UK's leading health research centres as part of an established team of information professionals? We are looking for a skilled information professional to cover a 12 month contract, details are below.

Information Specialist
University of Sheffield - School of Health and Related Research (ScHARR)

Location:	Sheffield
Salary:	£24,775 to £36,661 Please see advert
Hours:	Full Time
Contract Type:	Contract / Temporary

 

Placed on:	8th December 2014
Closes:	5th January 2015
Job Ref:	UOS009835

Contract Type: Fixed-term for 12 months

Faculty: Faculty of Medicine, Dentistry and Health

Salary: 

Grade 6

£24,775-£28,695 per annum

Grade 7

£28,972 - £36,661 per annum.

Within the School of Health and Related Research (ScHARR) Information Resources Group (IRG), you will participate in the provision of a specialist health research information service to ScHARR staff and students and to NHS staff undertaking research. You will undertake literature searches to support ScHARR research and consultancy projects, and contribute to the IRG’s information skills teaching and training activities. You will take a proactive role in identifying and responding to the information needs of ScHARR Information Service users.

To apply go here:

http://bit.ly/1wvXseg

So you're an Information Specialist - What does that mean?

Me playing with the world's biggest tablet ;-)

So...what exactly do you do?

Library and information professionals are used to being asked questions, it comes with the territory. Yet one of the hardest questions I get asked on a personal and more often professional level as part of my role as an information specialist at The School of Health and Related Research is that of; ‘So what do you do?’

I’m sure I’m not alone in having problems trying to describe what I do, these days people’s job roles are either very diverse, specialised or complex. And like lots of my peers we get to meet lots of people at networking events and that invariably leads to ice-breaking questions such as that one.

Life was so much easier in the past

When I left school in Derbyshire in the 1980s I had the options of going to work down the mines, in a factory or a shop, in roles I could have easily explained in a couple of short lines. Thankfully subliminal teenage foresight stopped me from going into the doomed mining industry but I did start in a warehouse that fitted out and repaired fire engines and ambulances. Not realising it back then, this was an obvious toe-dip into library-associated work as I worked my hardest ensuring everything was catalogued; and could be sourced easily from behind the counter on request. Fast-forward nearly thirty years and things have changed somewhat. These days it’s far from easy to capture what I do, as I lecture, train, write, present, network, research and most importantly support others.

What does Chandler Bing do?

Anyone familiar with the series Friends and the character Chandler Bing will be familiar with the long running joke that no one really knew what he did. At times I feel a bit like Chandler and the thought of trying to explain my job to someone does fill me with dread. Take for example the night out I had whilst in London at Internet Librarian International where the problem again came to light. I had known the two friends I was at the bar with for over 20 years, one of whom is a master chef, the other a manager for a bike store. We got onto the topic of health research and information and I started a long monologue about how health research is carried out and how information is assessed for quality. I remember the odd look they gave me and could see their brains working overtime; “how does he know this?”. At that point I said: “you do know what I do for a living and where I work?” Obviously not, they had an idea of what I did, that it involved the Internet, social media, bits of teaching and probably a few other things.

With networking being an important aspect of our professional lives I am used to responding to the "what do you do " question in more formal situations, but even then it can feel like I don’t really know the answer myself. People can easily make assumptions: ‘ah, you’re a librarian - you shelve books’ or ‘you work in a library, so therefore say ‘shush’ a lot’. Obviously librarians reading this right now will say that is far from the case. For a start, there are different types of librarians, subject, liaison, cataloguers, systems ones to name but a few. Yet many jobs, even though not explicit, do give people an idea of what kind of environment they work in and potentially what they might do.

So again, what is it you do?

So I’m at an event or conference and that question pops up; ‘what do you do?’ My often stunted (it’s stunted as I subconsciously think; ‘oh oh, here we go again’), reply; ‘An information specialist’. I realise that the best option is just to have a pre-written short paragraph script I can call upon, but it is always too late.

The replies to that vary, from either, ‘what is that?’ to:

‘Is that like being an IT technician?’

‘Is that like being a learning technologist?’

‘Is that like being a librarian?’

‘Is that like doing social media stuff?’

‘Do you work for the Government?’

‘Ah, excuse me, I’ve just seen a friend I need to speak to’

My answers have been from the very short; ‘If I wasn’t working for a university, I’d be at GCHQ reading your emails’; to very long ones where I can see them eventually looking for the nearest exit or friend. The reality is, that for a large part of the time I do not know what I do, let’s be clear here, I know what I have to do and know how to do it - hopefully very well. Yet I do not know how to explain my job, if I was to apply for a new job (note to boss here, am not presently doing this) I would obviously use the additional information to capture the many things mentioned above as best I can. Yet, it would still leave many thinking, so what does he do again? Often this comes back to the previous notion that many people in our aligned professions are doing very diverse and unique jobs now. You could get a dozen library and information professionals in a room to talk about their profession, yet most would be coming from different perspectives, experiences and roles.

It is what it is

At my institution I’m in a fairly unique role and have been for about eight years, one where I try and look for opportunities to engage with new technologies and help others do so. So yes, to some extent I’m a learning technologist, although I lean more towards research support. I also try and keep an eye on issues relating to these technologies, security being one of them, as many are third party, so yes, there’s a bit of IT technician about me. I encourage people to use, understand and exploit technology more. I try to help people use everything from cameras to computers, from browsers to searching better, so the IT side does play a big part. I engage with social media and altmetrics quite a lot and like everything else, keep a keen eye on the ethics and privacy issues. I am interested in information and how it is used, afterall. I have a big interest in the areas of Altmetrics and social media and therefore using it a lot often means people come to me asking how to engage with social media, so yes I do that a bit.

I also have some involvement with libraries and librarians as they are a group aligned to my role, my research interests and my education. My team has a small academic library and I have had involvement with how that runs, I speak at library conferences, so again I can see why they say that I’m a librarian. What I’m trying to say is that like so many people in my profession, often our real value comes from the sum total of our parts. What underpins my role, like so many professionals I come into contact with is that I help people, or try to help them. Like any good library or information professional I may not always know the answer but can certainly find out or find someone who does know.

To conclude I have to say that to me, my job is rarely complicated, it is not hard and whilst I do not know how to hack computers or write php I am adept at using technology. It is however a very diverse role, at times demanding, very changeable and requires me to make quick decisions and think for myself whilst always looking for the right opportunities. That pretty much describes many jobs, so I am not sure why it can be so hard to describe mine. Perhaps the real reason is that the role is not only diverse, but many things to many people. Even if a lot of people see you as one thing and that one thing is useful for them then you still have a role to play in supporting them. It has been a common problem for libraries and their survival, often people see them as just a place where books are kept. Not that they are places where activities are held, experts are in residence or knowledge and technologies can be accessed. That said, for the individual in the library and information setting it remains ever-important to work hard at what you do best and exploit that area of expertise. And if anything, there will always be people who can’t really understand what you do, but will know where to find you when they need help.

Frankly, its Christmas!

There are many signs that Christmas is on its way: the lights in town, the presents in every shop window, the sound of tiny feet pattering downstairs to open their advent calendars. But perhaps the most universally recognised sign that Christmas is upon us the moment when our Frank dons his full festive ensemble:

If you're at a loose end and need a hug from Sheffield's best-dressed skeleton, come on down to the ScHARR Library, 1st floor, ScHARR Towers. He'd be delighted to see you.

Posted by Claire

Peer review is fraught with problems, and we need a fix

This piece was originally written by myself for The Conversation and published in November looking at the growing interest in post publication peer review. It is republished under a Creative Commons Licence.
It was written in collaboration with Akshat Rathi from The Conversation

Image used under a Creative Commons CC BY 2.0 licence © Sebastien Wiertz  http://bit.ly/12litMf

Peer review is fraught with problems, and we need a fix
By Andy Tattersall, University of Sheffield

Dirty Harry once said, “Opinions are like assholes; everybody has one”. Now that the internet has made it easier than ever to share an unsolicited opinion, traditional methods of academic review are beginning to show their age.
We can now leave a public comment on just about anything – including the news, politics, YouTube videos, this article and even the meal we just ate. These comments can sometimes help consumers make more informed choices. In return, companies gain feedback on their products.
The idea was widely championed by Amazon, who have profited enormously from a mechanism which not only shows opinions on a particular product, but also lists items which other users ultimately bought. Comments and star-ratings should not always be taken at face value: Baywatch actor David Hasslehoff’s CD “Looking for the Best” currently enjoys 1,027 five-star reviews, but it is hard to believe that the majority of these reviews are sincere. Take for instance this comment from user Sasha Kendricks: “If I could keep time in a bottle, I would use it only to listen to this glistening, steaming pile of wondrous music.”
Anonymous online review can have a real and sometimes destructive effect on lives in the real world: a handful of bad Yelp reviews often spell doom for a restaurant or small business. Actively contesting negative or inaccurate reviews can lead to harmful publicity for a business, leaving no way out for business owners.

Academic peer review

Anonymous, independent review has been a core part of the academic research process for years. Prior to publication in any reputable journal, papers are anonymously assessed by the author’s peers for originality, correct methodology, and suitability for the journal in question. Peer review is a gatekeeper system that aims to ensure that high-quality papers are published in an appropriate specialist journal. Unlike film and music reviews, academic peer review is supposed to be as objective as possible. While the clarity of writing and communication is an important factor, the novelty, consistency and correctness of the content are paramount, and a paper should not be rejected on the grounds that it is boring to read.
Once published, the quality of any particular piece of research is often measured by citations, that is, the number of times that a paper is formally mentioned in a later piece of published research. In theory, this aims to highlight how important, useful or interesting a previous piece of work is. More citations are usually better for the author, although that is not always the case.
Take, for instance, Andrew Wakefield’s controversial paper on the association between the MMR jab and autism, published in leading medical journal The Lancet. This paper has received nearly two thousand citations – most authors would be thrilled to receive a hundred. However, the quality of Wakefield’s research is not at all reflected by this large number. Many of these citations are a product of the storm of controversy surrounding the work, and are contained within papers which are critical of the methods used. Wakefield’s research has now been robustly discredited, and the paper was retracted by the Lancet in 2010. Nevertheless, this extreme case highlights serious problems with judging a paper or an academic by number of citations.
More sophisticated metrics exist. The h-index, first proposed by physicist Jorge Hirsch, tries to account for both the quality and quantity of a scholar’s output in a single number: a researcher who has published n papers, each of which has been cited n times, has an h-index of n. In order to achieve a high h-index, one cannot merely publish a large number of uninteresting papers, or a single extremely significant masterpiece.
The h-index is by no means perfect. For example, it does not capture the work of brilliant fledgling academics with a small number of papers. Recent research has examined a variety of alternative measures of scholarly output, “altmetrics”, which use a much wider set of data including article views, downloads, and social media engagement.
Some critics argue that metrics based on tweets and likes might emphasise populist, attention-seeking articles over drier, more rigorous work. Despite this controversy, altmetrics offer real advantages for academics. They are typically much more fine-grained, providing a rich profile of the demographic who cite a particular piece of work. This system of open online feedback for academic papers is still in its infancy.
Nature journals recently started to provide authors with feedback on page-views and social media engagement, and sites such as Scirate allow Reddit-style voting on pre-print articles. However, traditional peer-reviewed journals and associated metrics such as impact factor, which broadly characterises the prestige associated with a particular journal, retain the hard-earned trust of funding organisations, and their power is likely to persist for some time.

Post-publication review

Post-publication review is a model with some potential. The idea is to get academics to review a paper after it has been published. This will remove the bottleneck that journals currently put up because editors are involved and peer review has to be done prior to publication.
But there are limitations. Academics are never short of opinions in their areas of expertise – it goes with the territory. Yet passing comment publicly on other people’s research can be risky, and negative feedback could provoke a retaliation.
Post-publication review also has the potential for bias via preconceived judgements. One researcher may leave harsh comments on another’s research based on the fact they do not like that person: rivalry in academia is not uncommon. Trolling on the web has become a serious problem in recent times, and it is not just the domain of the uneducated, bitter and twisted section, but is also enjoyed by members of society who are supposedly balanced, measured and intelligent.
One post-publication review platform, PubPeer, allows anonymous commenting – which, as seen with sites that allow for anonymous posts – could open the door for more trolling and abusive behaviour. It could offer reviewers an extra level of protection from what they say. One researcher recently filed a lawsuit over anonymous comments on PubPeer which they claim caused them to lose their job, after accusations of misconduct in their research. In a similar case, an academic claimed to have lost project funding after a reviewer complained about a blog post they had written about their project.
Post-publication comment can also be susceptible to manipulation and bias if not properly moderated. Even then, it is not easy to detect how honest and sincere someone is being over the Web. Recent stories featuring TripAdivisor and the independent health feedback website Patient Opinion show how rating and review systems can come into question. Nevertheless, research can possibly learn something from the likes of Amazon in how a long tail of research discoverability can be created. Comments and reviews may not always truly highlight how good a piece of research is, but they can help create a post-publication dialogue, a connectivity, globally about that topic of research, that in time sparks new ideas and publications.
Many now believe that long-standing metrics of academic research – peer review, citation-counting, impact factor – are reaching breaking point. But we are not yet in a position to place complete trust in the alternatives – altmetrics, open science, and post-publication review. But what is clear is that in order to measure the value of new measures of value, we need to try them out at scale.

Andy Tattersall does not work for, consult to, own shares in or receive funding from any company or organisation that would benefit from this article, and has no relevant affiliations.
This article was originally published on The Conversation. Read the original article.

Information Resources blog reaches more than 100,000 views

The Information Resources Blog, as managed by the Information Resources (IR) team here at ScHARR, has reached an impressive 100,000 views. In fact, because I’m so late in sending out this blog post, we have now reached an even more impressive 107,000 views. Not bad hey?

The blog was first set-up in 2007 and is aimed at fellow colleagues in the worlds of health information and higher education, with many of the posts being relevant to the work of academics more generally. There is often a focus on new technologies and social media platforms, as well as recent and upcoming work undertaken by IR.

Amongst the 107,000, we have received the most ‘page views’ from the following countries:

• United States
• United Kingdom
• France
• Russia
• Germany
• Ukraine
• Canada

The top five most highly read blog posts since it all began in 2007 are:

1.      From October 2009: 

Recommended Website of the Month - Mendeley

Total views = 4145

2.      From November 2007: 

New! Netting the Evidence

Total views = 1136

3.      From April 2013:  

The Story Behind the Elsevier Purchase of Mendeley...

Total views = 811

4.      From September 2012: 

Mendelely Advisors Appreciation Day

Total views = 760

5.      From September 2010: 

CASP Checklists have moved

Total views = 613

In addition to the blog, we have a dedicated Information Resources Twitter account (@scharrlib), which focuses on the latest developments in health and information science. If you're yet to set yourself up with a Twitter account but think you might be interested in the information we post, you can visit the following page and sign-up for our free daily newspaper via paper.li. Or you could simply visit the page to view the latest content: http://paper.li/scharrlib/1338371438. Not convinced about the benefits of Twitter? Why not read the following blog post by Andy Tattersall about 7 new ways to work in 2013, where you will find a useful overview of the benefits of Twitter in academia.

We also have a YouTube channel called scharrvids, where you will find a whole host of tutorials on subjects such as online reference management tools and useful searching techniques. There is an option to subscribe available from the main page, so you need never miss out on a new video!

We're thrilled that the blog has continued to be so popular over the years, especially since so many new platforms and tools have become available in the years since it was first launched. A big thank you to everyone who has supported the blog over the years, by contributing to it, reading it, commenting on it and sharing the content with colleagues.

Conference Review: SEDA 19th Annual Conference, 13-14th November 2014

SEDA conference: wired for sound from Claire Beecroft

Last week I attended the 19th Annual SEDA Conference. SEDA is the Staff and Educational Development Association, and the attendees included staff working directly in educational development, learning technologists, enthusiastic teachers and other support and academic related staff.

The theme of this year's conference was "Opportunities and challenges for academic development in a post-digital age". From my discussions with other delegates I sense that this was perhaps the first time that this conference had pitched itself fairly and squarely at issues relating to educational technology.

The opening speaker Grainne Conole did a brilliant job of mapping the history of the development of educational technology and particularly innovations such as MOOCs and Open Educational Resources. It was reassuring to be in the presence of both a speaker and an audience who feel positively about these topics and have a more optimistic mindset towards them.

There was a fantastic choice of parallel sessions, and the first I chose to attend was by Kathryn James. The focus was on a topic related to her PhD work, "Rhetoric and reality: The drive of learning technology and its implications for academic development". Catherine talked through the tensions and pressure points that surround academic use of learning technologies, and allowed us to chat on our tables- I was sat with a really nice group we had a really interesting discussion about the tools we use and the challenges we face in engaging academic staff with them- 'a trouble shared..' and all that!

The next session I chose was by Rebecca Dearden and was entitled "Working in a "third space" to create an institutional framework to underpin use of audio and video".  Rebecca talked about a project she had led which looked at the legal and privacy issues related to lecture-capture at Leeds University. She gave a really interesting presentation that showed how serendipity, willingness to work in all kinds of physical and virtual spaces, and teamwork can really pay off and deliver results in moving forward institutional policy.

After lunch, my third parallel session was entitled "A "menu" of teaching approaches to transform engagement with technology-enhanced learning", by Stuart Hepplestone and Ian Glover of Sheffield Hallam University. They discussed how they had developed a menu tool for helping academic staff to identify appropriate technologies to embed in their learning in teaching and access case studies demonstrating how these technologies can be effectively used. The session also demonstrated a kind of "diagnostic" tool that allows academic staff to identify where within their existing learning and teaching activity the opportunities to use technology might lie.

Finally, after a networking break, it was time for my workshop on using audio to deliver feedback. I was worried that the topic might not appeal, but I needn't have done, as around 20 people attended my session. I demonstrated how I have used audio and what I have found to be the benefits and advantages for both myself and my students. We also looked at the many free and low-cost tools available to deliver audio feedback, and the risks and challenges of maintaining security and privacy for students when sharing and storing audio feedback.

Many of the attendees had attempted audio feedback themselves and those that had were really enthusiastic. Those that hadn't seem to leave full of enthusiasm for the subject, and three people approached me afterwards to say they would definitely try audio feedback in the future. My slideshow presentation is above.

Overall I really enjoyed the conference, and ended up wishing I had booked for both days. I will next year!

Posted by Claire

Spot On Conference - Altmetrics and other Fads (or how I learned how to stop worrying about trains and love Skype)

Last week I was lucky enough to be invited to deliver a talk at the Nature Publishing hosted Spot On Conference at The Wellcome Trust in London to talk about my experience of using technologies to help academics communicate and build their scholarly profiles. I was delivering a short presentation as part of the parallel session ‘Measuring social impact - the tools available and whose responsibility is it?’.

I was looking forward to speaking as it was an opportunity to meet in person my journalist collaborator from my recent The Conversation piece on post-publication peer review, Akshat Rathi, as well as Charlie Rapple from Kudos and Jean Lui from Altmetric.com.

Sadly it was not to be, as I left my home in Chesterfield at 7.30am, little did I know that I would be back in my conservatory three and a half hours later. In the end it turns out to be a good experiment into how technology can resolve such problems as a serious signals failure at Derby Train Station. As my train was cancelled at this important network junction and with it every other train heading north and south I made the executive decision to head back home and hopefully make my allotted slot at 11am.

Firstly Twitter came to my aid as I Tweeted that I was unlikely to make it to the conference, to which I got an immediate reply from the organisers. I was able to request a direct message on Twitter from my contact at Spot On requesting their mobile number, which I got within minutes and from there I was able to put them in the picture as to my predicament. My only option was to head north-west to Matlock via a train that stopped at a multitude of small Derbyshire villages. I rang my wife asking that she would pick me up from Matlock and take me home- it all felt very Jack Bauer from the series 24 (well almost). My decision to return home was justified when I spoke later to a colleague who had pushed onto London via Euston and arrived at 12, meaning I would have missed my presentation slot. Years of train commuting has taught me when to stick or twist.

I got back for just after 11 and hooked up with Spot On via Google Hangout and was given time at the end to present. This brought about another opportunity as my slides were being delivered on Google Drive - I had long ditched Powerpoint and USBs for presentations about four years ago and it paid off. Given my talk was about technology and aiding researchers to go about their work more productively it allowed me to update the slides on the fly. That morning I had received an email from an editor of the peer-reviewed journal asking me to write a paper for a special issue based on a blog post I had written. So this was last minute proof on how the system can be flipped via the use of social technologies. It also highlighted how presentations can be amended to reflect last minute change without the good old USB stick. It reminded me also that sometimes wifi can throw in the occasional spanner to these techniques, but it should not always stop you. I remember presenting at CILIP back in 2012 using my Chromebook to present something in Google Drive when they had a powercut and the wifi went down. This could have spelt the end for my presentation, but I remembered seeing someone set up a wifi hotspot using their phone and hey presto! the show was back on the road.

So back to Spot On, another glitch appeared when I was told via Hangout Chat that the presenter’s laptop would not be able to host it on the screen, so we instantly switched to Skype. I was then able to appear on the screen and deliver my slides remotely. As for the presentation it was titled ‘Altmetrics and Other Fads -  Helping Researchers Through the Social, Technology and Innovation Maze’. Despite the rather flippant title it was not intended as such but highlights the constant change of technology and how many in academia are wary to engage with those at the cutting edge. This is understandable as technologies come and go and all too often they don’t always do what you want. The idea behind the presentation was that researchers and students need guidance in discovering, learning and using many new technologies. If teachers have a pedagogy as their reason to engage with technology, what do researchers have?

It was a shame that I was unable to attend Spot On as it looked a very engaging two day conference, you can view the Twitter feed here for more information https://twitter.com/hashtag/solo14

http://www.nature.com/spoton/

Here are my slides from the conference.

Altmetrics and other fads - Spot On 14 Conference from Andrew Tattersall

I only come here for the comments - Websites that allow researchers to review and comment on other’s work

Image © CC BY NASA Goddard Space Flight Centre http://bit.ly/1wPSD0m

The idea of public post-publication peer review of journal articles would have been considered heresy just a couple of years ago, but in recent years there has been a growth in post-publication reviewing options. The following blog post looks at a few of them and discusses the idea of leaving post-publication reviews and whether its all as bad as some in the academy community seem to think it is.

The journal publishing model has been quite consistent for many years now, authors submit their paper to a peer-reviewed journal. Reviewers read and analyse it and give feedback, and depending on how good or bad that feedback is the paper is eventually accepted or dismissed for a variety of reasons, which may lead the authors to attempt a resubmission to another journal. For the papers that do get published, that often ends the cycle of conversation between peers unless the research is discussed at an event, such as a conference or appears in the media, where comments can be easily left - which are sometimes scathing, unjustified or plain unhelpful. And there is the rub, when does post-publication peer review become post-publication comment and how different are they?

Unlike many other things that appear on the Web, such as music, film, art and review, where comments and critique is normal and helps others make better informed judgements on what to consume; academic articles only appear above the surface for review and critique when it is used as part of someone’s research or teaching. Often there is no critique, good or bad of the research post-publication, just an important line, conclusion or otherwise to help build the hypothesis of new research; agreeing with or against previous works. To say there was no post-publication review system previously would be untrue as the likes of BMJ editors and others have accepted e-letters and Rapid Responses about research published in their journals, and blogging and social media have more recently offered platforms for researchers to discuss other’s works.

From my experience many researchers feel uncomfortable on speaking about other people’s research, which is understandable. Nobody wants to hear ill of their own hard work, as this is what they are potentially opening themselves up to. Take for example the reviews on such as YouTube or the Daily Mail website, often comments can become personal, malicious and quite damning. The issue is that everyone has an opinion and that can be on everything from Syria to fracking, and the Web has facilitated that opinion culture to the point where ‘trolling’ is now an acknowledged and serious problem. Yet academic publishing is different, certainly academics are more than culpable for their barbed comments, but making unjustified ones online will help no one, especially in the advancement of knowledge via discourse.

The journal publishing model has been criticised for being out of touch with modern publishing, and rightly so; in that a piece of research which can take years to complete can then take nearly as long to get published. So by that time things may have moved on in that topic of research, new methods, technologies and ideas may have surfaced. Post-publication reviews can help highlight this, and also may make researchers aware of potential future collaborators or similar research being undertaken.

For post-publication review to really be productive it has to be open, unlike sites such as YouTube which has allowed aliases and therefore trolls to flourish. Obviously not every piece of research commands a post-publication review and given the figures which range from about 12% to 90% of papers not being cited, it is pretty likely that not all papers will get reviews or at least have the mechanisms to be reviewed. We also have to remember that while some areas of research are less reliant on the journal publishing model, this does not mean post-publication review is not for them, in disciplines such as the humanities it may have just as much use.

Academic debate using the many Web 2.0 and social media tools freely available has only been embraced by a small percentage of academics. Interesting papers are more likely just shared using such as Twitter, Google+ and LinkedIn than discussed, but considering that it is far easier and less time-consuming just to share content on the web than review it, it is understandable. Reviewing takes time and unlike reviewing a film, which is foremost a subjective piece of writing and focuses on whether you enjoyed it or not and whether it was well made, peer-review requires more considered thought.  Research is measured on whether it was well designed and conducted, not how well it was written (although that does come into the formal review process - but more about whether it is understandable, not just using long words to impress). That said I will cover JOVE below which helps aid that second part of the review process.

The debate on whether is the best way forward for post-publication review will continue and like other topics such as measurement of research, there appears to be no ‘silver bullet’. Instead there is a collection of sites and tools operating in silos, all offering to solve a problem, that being the lack of post publication discussion and assessment. Below are a list of some of the main tools and sites offering some kind of comment, discussion or review system- it is not exhaustive or comprehensive, but it will give you some idea as to what they are and do.

PLOS ONE

PLOS ONE’s refers to its mission statement as; “Accelerating the publication of peer-reviewed research”. First and foremost PLOS ONE is an open access collection of journals that unlike many traditional journals has sped up up the publication process and ensures authors retain copyright. Not a post-publication review site outright, it does allow users to comment on the published research, very much how newspapers allow visitors to comment their news articles. Commenting on research is in essence less formal than post-publication reviewing, the reader comments and has the remit to post something as indepth as they wish. They may want to write just a few lines about a part of the research, the methodology, results or conclusion or a longer more in-depth review about the whole paper. When commenting on papers in PLOS ONE you must be a registered user and identify any competing interests. The rules are quite simple and say that anyone commenting on someone else’s research must not post content as stated below:

1. Remarks that could be interpreted as allegations of misconduct
2. Unsupported assertions or statements
3. Inflammatory or insulting language

Anyone breaking these rules will be removed and their account disabled- obviously it does not stop them creating new accounts, but that will always be a problem for many interactive websites.

http://www.plosone.org/

PubMed Commons

PubMed is a huge publicly accessible search engine that accesses the Medline database of references and abstracts on life sciences and biomedical topics. It recently launched PubMed Commons to enable authors to share opinions and information about scientific publications in PubMed.

To be eligible to use PubMed Commons you have to be an author of a publication in the database, therefore preventing anyone from going in and leaving comments. The emails of eligible authors have been collected from the NIH and the Wellcome Trust and authors emails within PubMed and PubMed Central. You can also ask a colleague to invite you into the system.

© PubMed

The guidelines for PubMed Commons are more stringent than that of PLOS ONE and other such sites. Commenters must use their real name and again disclose any conflicts of interest. By contributing to Commons they grant other users a worldwide, royalty-free, non-exclusive, perpetual license under the Creative Commons Attribution 3.0 United States License. Again the usual rules of not posting inflammatory, offensive and spam comments apply. Full guidelines can be viewed here:

http://www.ncbi.nlm.nih.gov/pubmedcommons/help/guidelines/

http://www.ncbi.nlm.nih.gov/pubmedcommons/

Open Review

Open Review is a new feature within the popular academic social network and research sharing platform ResearchGate. Open Review allows users to publish an open and transparent review of any paper they have read, worked with, or cited. ResearchGate say it is: “Designed to approach the evaluation of research in a different way, Open Review encourages scientists and researchers to focus on one key question: Is this research reproducible?” Users can discuss articles they click on, with a slant more towards asking questions about the publication, than commenting or reviewing it.

http://www.researchgate.net/publicliterature.OpenReviewInfo.html

F1000 Research

© F1000

F1000 - standing for Faculty of 1000 - is made up of F1000 Prime, which is a personalised recommendation system for biomedical research articles from F1000. F1000 Research is an open science journal with post-publication peer reviewed research with underlying datasets. Finally there is F1000 Posters which is an open repository for conference posters and slide presentations.

F1000 Research has a system of open peer review which publishes referee responses and allows for replies by the authors and reader comments, so a bit of everything. In addition they offer incentives for reviewers which include a 50% discount on article processing charges for the 12 months following the submission of their referee report. They also offer a 6 month free personal subscription of their sister service F1000 Prime. Users of F1000 can track the conversation and even discuss the article at the bottom of the page, so the entire process, paper, review and discussion are taking place on one page. Even referee’s reports can be cited in F1000 Research and published under a CC BY license. A DOI (digital object identifier) is assigned to every referee report, so it can be cited independently from the article.

http://f1000research.com/

PubPeer

PubPeer refers to itself as the online journal club that allows users to search for papers via DOIs, PMIDs, arXiv IDs, keywords and authors amongst other options. PubPeer’s goal is to create an online community that uses the publication of scientific results as an opening for discussion among scientists. Researchers can comment on almost any scientific article published with a DOI or preprint in the arXiv. You can also browse the list of journals with comments, although presently it is rare to find a journal with more than a couple of comments. First and last authors of published articles are invited to post comments- I’m presuming the authors in between also get a say. Unlike some of the other tools mentioned, PubPeer allows for anonymous commenting, which could open the door for more trolling and abusive behaviour as reviewers feel an extra level of protection from what they say. -one researcher has filed a lawsuit over anonymous comments which they claim caused them to lose their job after accusations of misconduct in their research.

http://news.sciencemag.org/scientific-community/2014/10/researcher-files-lawsuit-over-anonymous-pubpeer-comments

https://pubpeer.com/

Publons

The primary aim of Publons is to help researchers get credit for peer review. Whilst writing for peer-reviewed journals have often been seen by many of handing their hard work over to someone else to benefit from financially, that being the publishers, at least there is the benefits of the author’s increased kudos, profile and knowledge-building and potential to gain promotion within their organisation. Peer reviewing can also have similar rewards with regards to the researcher’s CV and promotion prospects and that they get to see emerging research but the anonymous nature of much of it means less opportunity for profile building, yet it is no less part of the system that is the research publishing cycle. Publons sets to works with reviewers, publishers, universities, and funding agencies to turn peer review into a measurable research output. They collect peer review information from reviewers and from publishers, and use the data to create reviewer profiles with publisher-verified peer review contributions that researchers can add to their CV. Publons state that; “reviewers control how each review is displayed on their profile (blind, open, or published), and can add both pre-publication reviews they do for journals and post-publication reviews of any article.”

https://publons.com/

The Winnower

The Winnower is possibly the least academic looking post-publication platform of them all, but that should not put readers off; in fact it should have the opposite effect. An attractive site that sets its stall out on the homepage with the statement that; “The Winnower is founded on the principle that all ideas should be openly discussed, debated and archived. As with so many new academic tools and platforms it began life thanks to a PhD student, namely Joshua Nicholson from Virginia Tech. It provides an interesting new angle looking at research from both ends of the spectrum, that which has made a big impact and research that was retracted with its own ‘Grain’ and ‘Chaff’ page. The grain features publications with more than 1000 citations or a Altmetric score above 250. Whilst the chaff looks at papers that were pulled from publication and give authors an opportunity to talk about their research rather than just a ‘name and shame’ list. The Winnower is obviously still in its early stages due to the handful of reviews and publications, but not every post-publication review site can begin from the point of PubMed. It is one certainly worth keeping an eye on.

https://thewinnower.com/

JOVE

The Journal of Visualized Experiments has now been around for some time, almost a decade but it is only in the last couple of years it has really broken through and is now subscribed to by many university libraries. JOVE is a PubMed-indexed video journal with a mission to increase the productivity of scientific research. Although not at the forefront of JOVE’s priorities, they do allow for comments on the published research videos.

http://www.jove.com/

Peer J

Peer J is an open access peer-reviewed scientific journal that focused on publishing research in the biological and medical sciences. It received substantial backing of USD 950,000 from O'Reilly Media - which founder Tim O’Rielly is famous for popularising the term Web 2.0. It is part of the same publishing company that was co-founded by publisher Peter Binfield (formerly at PLOS ONE) and CEO Jason Hoyt (formerly at Mendeley), who obviously have a lot of experience in scholarly communications.

© CC BY Peer J

Peer J has a points system for authors and commentators as an incentive to publish and comment on research. Anyone who has ever argued that citations, H Indexes and such as Twitter and followers are just multi-levelled multiplayer games will get this. The points system are below:

• Be an academic editor on a PeerJ article = 100 pts
• Be an author on a published PeerJ article = 100 pts
• Make your manuscript reviews public on a PeerJ article = 35 pts
• Submit an "open review" as a reviewer on a PeerJ article = 35 pts
• Be an author on a PeerJ PrePrint = 35 pts
• Be an academic editor on a rejected PeerJ article without reviews = 35 pts
• Have an answer on a question accepted = 15 pts
• Have feedback deemed "very helpful" by an author of a PeerJ PrePrint = 15 pts

• Receive an up vote for an answer = 10 pts
• Receive an up vote for a question = 5 pts
• Receive an up vote for feedback on a PeerJ PrePrint = 5 pts
• Receive an up vote for reply to question or comment = 1 pt
• Have first feedback approved in moderation on a PeerJ PrePrint = 1 pt

There are tables of the top authors and reviewers which can be filtered by topic area, publication date and those who have asked the most questions and given the most answers. The questions and answers aspect is a different angle to the commenting process as it does potentially open up further dialogue between authors and commentators. At present though there does not seem to be much activity in this area.

Peer J state: “Our annotation system goes beyond just answering questions or finding answers. Everyone from authors, editors, reviewers, and visitors to PeerJ are contributing in some way. Often, these are "hidden" contributions to the body of science that can go unrecognized. The points that we are starting to show on profile pages are just a light way to surface this participation.”

As for this points ranking system, it will appeal to some researchers, those with a competitive edge, but on the flipside will feel uncomfortable to others who do not want to see their work captured in numbers, and that applies to any kind of metric not just Peer J. Netherless, it is an interesting take on the publishing model and one that will continue to create interest and debate.

https://peerj.com/

        

PaperCritic

One of the first proper research commenting tools, PaperCritic appears to have ceased business but is still worthy of a brief mention. Created using the Mendeley API, PaperCritic connected with a user’s Mendeley account and allowed them to comment on research hosted within Mendeley’s huge database of references. Their blog, Facebook and Twitter accounts all fell silent in 2012 leading me to believe that this was no longer running. The chances are that Mendeley will at some point create their own commenting and reviewing system, so still well worth the mention.

http://www.papercritic.com/

There seems to be some difference between the notion of reviewing, discussing and commenting, something Kent Anderson in The Scholarly Kitchen wrote about earlier this year. With Anderson summerising that; “Today’s commentators seem to have many axes to grind. Far too often, commentary forums degrade into polemical attacks with win or lose dynamics at their heart. The pursuit of knowledge and science isn’t the goal. Capitulation of one combatant to another is.” Anderson questioned the validity of comments being championed by publications and websites and that they could never be considered in the same light as peer-review.

There is a need for both as comments can be insightful and highlight or spot useful content for the original authors or other readers without going into indepth reviews. On the flipside they can be malicious, unfounded and just clog up the whole knowledge process if left un-moderated and anonymous. Peer-review may not be perfect, but as the social web becomes more useful as a platform for discussion and knowledge sharing, it makes sense that other options are explored, even if they are supplementary. This is growing case for Altmetrics, first seen as an alternative to the traditional measurement of citations and now argued as more of an alternative indicator, rather than measurement. The real problem is that like with many other technologies and platforms for communication we run the risk of not being able to see the wood for the trees. Post publication review platforms need to be explicit in their aims and explain that clearly to readers and reviewers. Like social media, it is unlikely that we will see every researcher using these unless they became standardised and part of the research cycle. It is an option, as with academic discussion lists, where the most insightful and on occasion barbed communications take place. Post-review commenting is happening right now and someone out there may have already commented on your research- whether you respond remains your choice.

Recommended reading:

Pre-publication posting and post-publication review will facilitate the correction of errors and will ultimately strengthen published submissions

The new dilemma of online peer review: too many places to post?

Stick to Your Ribs: The Problems With Calling Comments “Post-Publication Peer-Review”

Peer review in scientific publications - Science and Technology Committee  

How Few Papers Ever Get Cited? It’s Bad, But Not THAT Bad