The pros and cons of the research excellence framework are being hotly debated once again.

As Lord Stern¡¯s review of the REF considers the cost and bureaucracy involved in running the sector-wide research audit, the ¡°game-playing¡± tactics used by institutions and how reforms might improve the quality of UK research, universities have been quick to put forward their vision of how the next exercise might be run.

But how does the REF affect the day-to-day lives of academics? That less-analysed question has been examined by Len Ole Sch?fer,?a visiting doctoral scholar at the UCL Institute of Education, who interviewed 23 UK academics and about 10 ÖйúAƬ policy advisers within government and various sector bodies on how they perceive the REF and its impact on academic life.

Mr Sch?fer, whose study into the impact of performance assessment was funded by the German Ministry of Education and Science, explained that there was a clear difference between academics¡¯ and advisers¡¯ views of the REF.

ÖйúAƬ

For those charged with putting the REF into action, the exercise was seen largely as a positive motivation, which pushed academics into producing better research outputs, Mr Sch?fer told an IoE seminar on 12 May.

But the prevailing view among those academics in chemistry, history and sociology interviewed by Mr Sch?fer was quite different, he said.

ÖйúAƬ

¡°For most interviewees, it creates pressure and stress, producing a decrease in the quality of research,¡± he said, adding that those interviewed spoke about depression and burnout resulting from the pressures created by the REF.

Interviewees also explained how the REF ¡°inhibits long-term research, blue-skies or risky research due to the short-termism¡± encouraged by the need to produce the requisite four papers for inclusion in the audit cycle, he said, noting that some policy advisers shared the criticism.

Those academics at lower-ranked universities tended to be less motivated by the REF to produce more outputs than those at more prestigious universities, where inclusion for assessment was often felt to be a make-or-break career?moment, he added.

That issue of who is included in the REF was another major conversation point among the academics interviewed, with many believing that women are far less likely to be submitted for assessment, to the detriment of their careers.

That perception is arguably borne out by the figures, with 67 per cent of eligible men included in the 2014 REF compared with just 51 per cent of women, with the gender selection gap widest between the ages of 45 and 55, according to a ÖйúAƬ Funding Council for England published in August 2015.

Better, but still hard on women

Measures to mitigate the impact of maternity leave and childcare responsibilities ¨C women who take leave for maternity in the REF cycle may submit fewer outputs ¨C were noted and commended by interviewees, Mr Sch?fer said.

ÖйúAƬ

But a large proportion still felt that women¡¯s professional advancement ¡°suffered from career breaks for maternity and childcare¡±, and as a result they ¡°cannot improve their research performances as much as men can¡±, he said.

¡°Some interviewees felt women were pushed towards more administrative tasks and other duties that are not valued by the scientific community, meaning they do not have the same amount of time to produce high-quality research outputs,¡± Mr Sch?fer explained.

ÖйúAƬ

In the case of some academics, the REF was said to actually reduce their research output, he added.

The desire to work across disciplines was also reduced because of the distinct units of assessment used in the REF, which it was felt discouraged staff from straying beyond the boundaries of their expertise. Again, the policy advisers interviewed felt that there was no evidence for this claim, with interdisciplinary research performing just as well as those outputs limited to a single subject, Mr Sch?fer added.

Simon Marginson, director of the UCL IoE¡¯s Centre for Global ÖйúAƬ, which hosted Mr Sch?fer, a PhD student at Germany¡¯s University of Bamberg, said that the study opened up the question of how the UK research landscape might be more radically transformed beyond merely tinkering with aspects of the REF.

The REF was part of a system that rewarded those who ¡°published early and often¡±, which often led to a ¡°frenetic¡± production of papers that offered few or no new insights.

¡°I often receive for consideration what are really master¡¯s level essays that are little more than literature reviews that do not say anything new ¨C the sector is awash with this kind of material,¡± said Professor Marginson.

He suggested that more fundamental reforms to university research might see the creation of more postdoctoral research positions, which would in effect allow PhD graduates to do a ¡°second thesis¡± with the benefit of their research experience.

ÖйúAƬ

¡°The better research is done at the postdoc level as PhD students are just learning how to research,¡± he said.

jack.grove@tesglobal.com