How the REF is killing science

All Universities in the UK participate in the Research Excellence Framework (REF). In theory the exercise is designed to reward institutions that are research active by supplementing their income from the government coffers. Success in the REF is very important to Vice-Chancellors and many academics as it is the metric by which many outsiders judge their achievements. Outsiders include the general public, funders, parents and potential students. However the REF is a bit of a sick puppy.

University senior management teams are very keen to see their profile raised by obtaining a good REF score and, as with every other ranking system this leads to game playing. The overall ranking for a university depends upon the ranking of the academics who are submitted. The ranking of academics is determined by the quality and impacts of their outputs. Generally academics who are publishing in high impact factor journals (e.g. Nature, Science) and bringing in lots of money via NERC, ESPRC, Leverhulme, Royal Society or FP7 type funding schemes will score highly. The score of individuals is judged by selected peers who sit on various subject orientated panels. One problem here is that those who are tasked with judging the worth of a colleague, cannot be expected to read every output (e.g. paper) submitted. They are therefore likely to take shortcuts by using journal Impact Factors as some sort of indicator of the value of a particular piece of work. There are many, many problems with this approach. Apart from the fact that you are statistically illiterate if you depend on them, some topics (e.g. taxonomy) garner citations very slowly but continue to do so for many years while others (e.g. biomedical papers) will achieve high citation rates for a year or so before disappearing.  Even when a taxonomy paper is particularly notable and is highly cited, such as the recent catalogue of decapods published by Sammy De Grave of Oxford University Natural History Museum, if it is placed in a journal with a generally low impact factor, it will probably be deemed unworthy of REF attention.

 One side effect of this is that journals with relatively low citation rates, such as the excellent journals of Crustacean Biology, Marine Biology, Biological Bulletin or Marine Ecology Progress Series are becoming deemed less valuable places to publish work in. These society journals have been the mainstay of academia for hundreds of years – fora in which subject related discourses are bound together and in which sometimes vehement arguments advance the cause of science.  High impact magazines such as Nature and Science following populist science quite often get things wrong with significant consequences.  In the world of fisheries science these magazines love Doom and Gloom type articles predicting that we will all be eating jellyfish in ten years time.  They will publish a paper that will receive huge media attention worldwide but will often not publish critiques of that work from other academics.

Back to the REF . . . If too many academics are submitted it is likely that the overall average “grade” for a particular institution would go down (not everyone can be a high flier). Universities therefore are tempted to submit fewer, more highly ranking academics. Thus while their shop window may look good, the rot may be setting in behind the scenes. One particularly challenging aspect to the REF is the fact that universities are generally patchy in their achievements, few can excel in everything they do – but this is not reflected in the basic University Rankings.

Universities therefore are being forced into nurturing the few, potentially at the expense of the many and the REF is a driver for the concentration of power/resources in the hands of a few profs. Thus we are stifling innovation, encouraging sycophancy – in science a postdocs career may depend on the view of an established prof –  and perhaps encouraging nepotistic tendencies as those with the power seek to retain it and drive science in the direction that they are passionate about.

The solution would be to make Universities submit every one of their established academics to the REF.  No one could hide, mid-ranking academics (like yours truly) would be more valuable and institutions would be encouraged to nurture all of their research active staff rather than forced to concentrate on a few.  In a situation where only a few folk can be submitted, those that would like to be but are not Nature/Science types are embittered and made to feel 2nd class, those that don’t care can just hide in the woodwork.

Magnus Johnson is a lecturer in Environmental Marine Biology at the University of Hull.  His thoughts are his own.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s