V-c says QAA's methodology hampers the sector's efforts to defend itself. Rebecca Attwood reports

Weaknesses in the Quality Assurance Agency's methods have made it difficult for the sector to refute claims of declining standards, the watchdog has been told.

Speaking at the QAA's annual subscribers' meeting in Belfast last week, Philip Jones, vice-chancellor of Sheffield Hallam University, said that while there was "a wealth of evidence" to suggest that there were few real issues with quality and standards in UK ÖйúAƬ, it was difficult to use the QAA's work to defend the system.

The six-year audit cycle meant that problems could persist for some time before they were detected, audits acted on the basis of secondary evidence, such as a self-evaluation document, and the QAA's "academic infrastructure" - nationally agreed reference points used by universities to set, describe and assure the quality and standards - was difficult for outsiders to understand, he said.

ÖйúAƬ

Meanwhile, external examiners were "not seen as genuinely external, independent or impartial".

Professor Jones argued that there were two kinds of critics of standards in ÖйúAƬ.

ÖйúAƬ

One set identified problems but did not want to resolve them, preferring to hold them up as evidence that the system was rotten. Such critics would never be satisfied.

However, there were also questions being asked "by relatively serious people".

Some critiques were "far reaching and not easy to respond to", but there were also "weaknesses in the QAA methodology, which inhibit the defence", he said.

"QAA reports are relatively technical in nature. They do not seek to assess institutional standards, but the way in which academic standards and the quality of teaching are managed ... That means it is almost at one remove from the kinds of questions that are being asked in the public domain."

He added that audit "tests the claims made by an institution but does not compare them against a benchmark or against other institutions". It "doesn't actually look at people teaching and it doesn't read assessment scripts", he continued.

ÖйúAƬ

But Professor Jones said he was not convinced that significant changes to the QAA were the solution to the current "quality crisis".

Reviving a system under which all courses offered had to be approved externally, would be, logistically, a "mind-boggling" process, he said.

rebecca.attwood@tsleducation.com

CONTINUING ASSESSMENT

The prospect of auditors "living" with a university for a year was raised at the QAA's annual subscribers' meeting.

ÖйúAƬ

Stephen Jackson, director of the QAA's reviews group, said the process that was invoked when institutions apply for degree-awarding powers could be replicated by reviewers.

This involves the QAA "looking over the shoulder" of an institution for a 12-month period and monitoring its performance.

Dr Jackson said it would be "a rather different way of engaging with institutions from the one-off stage-managed event that audit has become."

Other approaches being considered include audits that focus on core areas but also examine issues relating to universities' individual missions, and "risk-based" reviews that would allow for a "more proportionate approach" to QAA engagement.

ÖйúAƬ