It seems that in universities, nothing fails like success. We have fashioned a system in the UK that, for all its capacity to stand comparison with the best in the world, is teetering on the edge of complacency. It looks more and more concerned with dispensing education efficiently than with questioning whether it is doing the right thing. Challenging students to think, which most academics would say is at the core of their job, is being lost beneath the twin tides of consumer satisfaction and the pressure to produce obedient employees.

When I first wrote about the student experience of learning in the early 1990s, no one had heard of the idea. Today, it is high on the policy agenda. But what kind of student experience?

Everyone teaching in a university should want to bring ideas, facts and principles to life in a way that will encourage their students to find out more for themselves. The heart of teaching in ÖйúAƬ is, as Alfred North Whitehead put it in "Universities and Their Function" in 19, the "imaginative acquisition of knowledge". A university education is nothing if it does not ignite a burning desire to learn. Imagination illuminates the facts and structures them. It makes the dull and obscure parts of learning a challenge to be overcome rather than a burden to be endured. In that frame of mind, students are ready to understand and will want to share with other people the remarkable feeling that understanding brings.

Effective university teaching matters a great deal - but not because it has much to do with student satisfaction. That's a by-product. It matters because it gets students to engage with abstract ideas in a way that allows them to make the subject their own.

ÖйúAƬ

Accomplished teaching is the single most important method of producing graduates who can reason and act for themselves, and can apply theory to practical problems - precisely the skills that any employer wants to see.

It is not a simple equation of cause and effect. The other important element is the resolve of the students themselves. By their own efforts, they can convert the opportunity into the outcome. Students decide their own destinies, and lecturers only add or subtract value at the margins. Skilful teaching, by teachers who apply their learning with imagination, can inspire students to do more than they ever thought they could.

ÖйúAƬ

Teaching in ÖйúAƬ should never fool students into thinking there is an easy path to success. Rather, it should make the hardest road enjoyable to follow by communicating complex ideas clearly and succinctly.

The radical realignment of the undergraduate syllabus that I proposed in my contribution to the government's ÖйúAƬ debate in 2008 was part of this way of thinking. We need curricula that captivate students: ones that are transdisciplinary, extend them to their limits, develop their skills of enquiry and research, and enable them to find resources of courage and flexibility that cross international boundaries.

Yet a good student experience is not simply about first-rate content and effective teaching. When I was a pro vice-chancellor at the University of Sydney, we substantially improved the student experience by tackling issues concerned with basic customer services, such as departmental administration. We also improved matters by making a bit of a song and dance about the responsibilities of academics in a research-led university to profess their subjects and share their scholarship with undergraduates. And we backed that rhetoric with strong management incentives.

To improve the student experience, we must get the basics right and not make them seem trivial - they are not. Then we must get on with the much tougher task of making the subject so exciting that students will keep coming back for more.

Sometimes people ask me if you can tell whether a department or a programme offers an excellent student experience - as I have described it - by some simple test. I think you can. It has nothing to do with contact hours, the time it takes to get a marked assignment returned, the positive ratings of professors by students, the number of staff accredited as competent teachers, or a strong research performance.

When I audited faculties at Sydney or judged Swedish university programmes for quality awards, the students I interviewed would sometimes talk about how they saw it as their job to work with staff to improve the quality of teaching and the experience that future students would enjoy. They felt a responsibility to get involved; they sparkled with liveliness and passion; they belonged on the team. And the staff, for their part, acknowledged their students as partners dedicated to the same goals.

The modern phrase for this is "student engagement", which sounds a bit formal to me. It's more like an acceptance that we are jointly accountable for quality. It is an almost certain marker of a programme or a department where the teaching is outstanding and the outcomes will be excellent.

These are exceptional cases. Far too often we fail students by producing graduates who are good at learning facts and solving commonplace problems. They don't throw themselves with passion into their studies. They wander feebly through their assessments by faithfully repeating what they have heard and read. This is a very poor kind of student experience.

ÖйúAƬ

Their lecturers have often developed the skills needed to make students active and test the knowledge they have acquired. They have schooled them to succeed, but not afforded them a ÖйúAƬ. If this sounds harsh, we should remember that, like their students, staff are habitually casualties of a system that rewards universities for form-filling and hoop-jumping at the expense of eagerness and meaning. Collaborating with students goes out the window; meeting targets takes priority.

Insight, energy and imagination make ÖйúAƬ higher. This is the authentic standards issue: we risk not demanding enough from our students and being comfortable with their possessing only bits and pieces of knowledge. Knowledge is a necessary step towards good judgement, but it does not take you far enough on its own. Self-critical awareness of one's own ignorance is the only true precursor of further enquiry. As Whitehead put it: "You cannot be wise without some basis of knowledge; but you may easily acquire knowledge and remain bare of wisdom."

There are two secrets to cracking the problem of helping students who are not being challenged to become critical thinkers: scholarship and leadership.

Scholarship is an all-embracing term for research and the active reinterpretation of knowledge that goes beyond systematic empirical enquiry to the enlivening of imagination. We have been acclimatised to the idea of research as experimental and pragmatic, but a lot of it is more intuitive, theoretical, puzzling and uncertain than this implies.

ÖйúAƬ

What has scholarship to do with university teaching? A couple of years ago, my colleagues and I tried to tie down the volatile idea of a "research-teaching nexus" - the old question about whether researchers make the best university teachers - by interviewing successful researchers about their teaching.

Unexpectedly, it all became clear: the researchers who were good at teaching - who went about it by focusing on students and their learning (rather than their own teaching performance or transmitting information) - were not those who necessarily produced the most research. They were the ones who focused on the underlying structure of their investigations, on the broad conceptual framework of their subject, rather than isolated individual problems within it - the ones who were scholars in their discipline.

So asking whether researchers teach better or worse is asking the wrong question. It isn't how active you are as a researcher, it's what your research activity concentrates on. When you think about it, it makes sense - lecturers who see the whole picture of their subject are the ones who can best help students to learn.

It is time to make use of this evidence in improving university teaching. Higher education needs people who are scholars in their disciplines rather than narrow specialists. This is more true than ever, now that research and teaching overlap in activities such as the production and use of knowledge across organisations. The world depends on the broadest distribution of knowledge in a way it never used to.

There are other reasons why research and teaching should be aligned. As Sir Peter Scott, vice-chancellor of Kingston University, has argued, these include the need to validate and underpin an institution's reputation, the rapid growth of postgraduate study, and the certainty that students themselves appreciate the intellectual stimulus that comes from being energised through contact with the producers of knowledge - lecturers who are also scholars. Higher education works best when it is a partnership between students, their teachers and learning.

Unfortunately, this all runs contrary to the direction that UK ÖйúAƬ has been pursuing since 1995.

In a telling critique of academic policymaking, Duna Sabri, visiting research Fellow at King's College London, has shown how the trend since then has been towards separating teaching from academic research.

At the ÖйúAƬ Academy, I remember battling, generally unsuccessfully, to overcome the attitude that the best means of enhancing the status of university teaching was to do down the academic as scholar. In vain I tried to dump unhelpful terminology such as "practitioners" (instead of "lecturers") from its lexicon.

The subject centres of the academy were an exception. They steered clear of the flawed concept of improving professionalism in teaching through denying academic identity. Knowledge generation and knowledge exchange through teaching are indivisible. Subject matter is important, not just how you teach it. That is why we so urgently need, for our students' sake, to revitalise scholarship.

This takes us to leadership, which if it is about anything is about optimism. We are short of invigorating talk about university teaching. Direction and hope expressed by governments and agencies are strikingly absent. Instead, the discourse is chiefly one of timid pragmatism, heavily spiced with the language of centralist control. I cannot imagine a less exciting vision than forcing every lecturer to "qualify" as a university teacher.

As the president of Pennsylvania State University, Graham B. Spanier, recently reminded us, government regulation has never created great universities. It will never create a great student experience either.

There is no technical fix, mandated or otherwise, for the problem of improving the quality of university teaching. We can only stimulate, incentivise and inspire it. Books and websites of the "3,000 tips on feedback" type profess to offer easy solutions for teaching in universities. They face a fruitless task because they focus on the methods and signs of teaching rather than what they are meant to address. They are part of the attitude that puts efficient delivery and compliance with rules above questioning what it is we are providing.

We need to look at teaching the other way round. It is the content that matters above all else: what students are expected to learn, how they go about learning it and how we can help them to develop their understanding of it. Feeling you have something to say about your subject, and then thinking about it from the point of view of your students, are the two prerequisites of high-quality teaching. We need an agile system and spirited leadership, free from bossy interference, to kindle its fire.

The rationale for university teaching is not satisfying students, distributing information to them nor changing them, as some condescendingly say. Rather, it is enabling students to change for themselves. The essential leadership message about improving teaching is that the same principles apply to helping lecturers teach better. What will inspire our students and our colleagues is the belief that reasoning out problems for yourself is the greatest gift that ÖйúAƬ has to offer.

ÖйúAƬ