We could be living in a golden age of British textbooks - but things don't seem to have worked out quite like that. "It is ironic," comments Stephen Kennedy, a veteran of more than a quarter-century of textbook publishing at Palgrave Macmillan. "Traditionally, British academics have been very good at writing textbooks. And we've had a big expansion in student numbers and more emphasis on teaching, so textbooks should have become more important - yet just at that time there arose more and more disincentives for people in the UK to write them."

Kennedy is eloquent about what textbooks can offer their authors and the development of knowledge. "To make a difference in the subject and change the way people think is a key reason people go into academic life. But you are unlikely to achieve that by writing a journal article or a monograph that's going to be read by a handful of people, most of whom will be reviewing it to pick holes in it. It's much better to write for the next generation."

Kennedy has no time for the notion that writing a textbook is a mundane task for second raters. "Academics have been brought up on the idea that textbooks are frivolous compared with journal articles and monographs, but many of my authors who have done both have told me that writing a textbook was the hardest thing they'd ever done," he says.

"You can't hide. In a journal article, if you come to a difficult bit, you can dodge round the edge of it and talk about something you do know about. In a good textbook, you have to address the full range of debates, think through the connections between all sorts of contradictory elements of the subject. It's really hard work and takes a lot of skills."

ÖйúAƬ

Bill Jones, who teaches politics at Liverpool Hope University and is a co-author of Politics UK (now in its sixth edition), tends to agree. He says: "You have to really nail the topic in your own mind to be able to write clearly about it, and that became something I enjoyed doing. I also see my role as a conduit between the esoteric researchers and the students and teachers in the class and lecture room.

"There's a kind of snobbery in academe about textbooks. 'I suppose someone has to write them,' I recall a rather stuffy academic saying to a textbook author colleague. But to do it well, address the students' needs and outsell your competition requires quite a bit of imagination, knowledge, understanding and clear thinking. Many academics cannot write good textbooks - just as many of them cannot lecture or teach properly."

ÖйúAƬ

But what of the rewards? Kennedy claims that he has "published a lot of people who've produced good value-added textbooks and have ended up with chairs earlier than they would probably otherwise have done" (although by no means everyone considers it a good career move). And, in some disciplines, textbooks can act as a calling card for generating other lucrative work, even if the direct financial returns are fairly meagre.

John Hayes, professor of management studies at Leeds Business School, developed a distance-learning masters programme for the BBC through the Institute of Communications. So he was able to draw on research he had already done to write The Theory and Practice of Change Management, aimed at MBA students and final-year undergraduates as well as practitioners. The book in turn helped to raise his profile and led to valuable consultancy work and visiting appointments at German and Danish business schools.

Yet the fact remains that, unlike in the US, where cover prices are higher and markets far larger, there tends not to be much money in textbooks for British academics. Advances, if any, tend to amount to a few hundred pounds. While royalties can provide a useful additional income stream, a day job remains necessary.

"If we calculate it at hourly rates," says Michael Moran, professor of politics at the University of Manchester, "I reckon I'd be better off imitating my Irish ancestors and digging ditches for a living ... The main incentive is that someone might actually read the thing; so many monographs are bought by a couple of hundred academic libraries and just languish on the shelves."

But although publishers and authors alike argue that textbooks can be intellectually stimulating, a major contribution to the development of a subject and even, in exceptional cases, good money earners, not all in the garden is rosy. In February, the Publishers Association conference, "What does ÖйúAƬ want from publishers?" (co-sponsored by Times ÖйúAƬ) focused particularly on "the future of the textbook". Although the publishers seemed not, or not yet, to be too downhearted, there are certainly major challenges ahead.

A trenchant presentation by Roland Fox, a senior lecturer in finance at the University of Salford, set out the threats to the traditional textbook in what he called "the new world order". While "textbook sales are relatively OK at the moment", he explained afterwards, "it is difficult to see scenarios for the future that support the existing business model, ie, paper-based sales of new books".

Fox lists a number of key issues for publishers trying to turn a profit. Students now, in effect, have free access to journals and, with all their other financial commitments, don't want to pay for anything. Some are actively hostile to books - although these tend to be those destined to fail their exams. And increasing numbers of overseas students and wider ranges of ability on campuses make it far more difficult to produce a textbook in a style appropriate for all potential readers.

There is increasing competition from the expanding second-hand market, and also from university reprographic departments and others that bring together information "similar but generally inferior" to textbooks in the form of student course packs. While publishers obviously try to differentiate between editions of books selling at very different prices in different markets, students are learning to shop around and get the best bargain (aided by websites that deconstruct what are often, in reality, slightly different versions of the same book).

ÖйúAƬ

So is the future electronic? As of now, it was agreed at the PA conference, textbooks are the sector of academic publishing least affected by the digital revolution (the battle for e-journals and electronic reference material has in essence been won). But although the e-book revolution has been two years away for the past ten years, perhaps it is really at last upon us.

E-textbooks may cost more and attract VAT, but they must represent the future, it was argued at the conference, because libraries can't store 100 paper copies of the same textbook and we can't expect all students to buy one.

Here again Fox was sceptical, from a business point of view, since he predicted that, based on the music industry's experience, we could expect four out of five downloads of any e-book to be illegal. His only solution was to set up a portal-based system. "If publishers deal directly with universities, who prepay for services as they do for journal subscriptions, databases and such," he said, "then the problem of illegal downloading disappears, as students will want the proper version if it has already been paid for." Students could then be provided with a textbook for each course as part of their fee, which would almost certainly have a positive impact on their results.

ÖйúAƬ

But publishers' incentives to produce textbooks are only one aspect of the problem. What about the incentives for academics to write them? Today, of course, there are major disincentives, notably the research assessment exercise and the audit culture it generates. Moran has written about how the RAE "has led to the extinction of many life forms in academia". While no one will regret the loss of some - "freeloaders, idlers, chancers" - he worries that "some endangered species need protecting". Prominent among them are the textbook writers, at a time when "confessing that you are writing a textbook is the surest way to be struck off the research dean's Christmas card list".

Philip Langeskov, acquisitions editor for literature, linguistics and politics at Pearson Education, feels this scarcity acutely. "It is quite difficult to get people to write textbooks in the disciplines I work in," he reports, "and, increasingly so, even in the past three years. If you want advancement in the current climate, writing a textbook seems to be low down the list of priorities. I think that is a great shame.

"I have been looking for an author for a particular, fairly mainstream political textbook project for the past six months, and have spoken to 15 or 20 people. Almost all have said: 'It's an important book, one that the discipline requires or could make good use of - and one that I could write - but I'm not going to do it.' Some say they are too busy, but quite a few say they have been specifically told by their department that they are not to write textbooks."

Matt Lloyd, editorial director at Cambridge University Press, commissions books on earth sciences and oversees the life sciences list. Since career progression in science is based firmly on research, he also tends to find it difficult to get British academics to write textbooks in these areas. The RAE has only exacerbated this trend.

Publishers are responding to the challenges in several ways. One is to adapt US textbooks to the British market. This may not matter in geology, but there is surely something strange and worrying in the idea of British undergraduates getting their first taste of politics or international relations from US academics who inevitably see the world rather differently from their European counterparts.

Another method is to tailor textbooks to the specific needs of individual lecturers and courses. In 2003, Liz Sproat, director of custom publishing at Pearson Education, became the first European publisher to offer academics the chance to create their own. They may decide to use an existing textbook amplified by a chapter or two from elsewhere to plug a gap in the coverage. Or they may opt for something more elaborate, incorporating their own material, sections from several Pearson books, case studies and third-party input from sources such as journal articles, although usually nothing from the publications of direct competitors.

The resulting book can have the name of the course and academic on the cover as well as a bespoke image and university branding if required. "Most academics," argues Sproat, "want to teach in the way that is most logical to them and not to follow in another author's tracks. They are also concerned that students won't buy a traditional textbook; it's much easier if they can claim the book is absolutely relevant and essential to the course. If they put 20 books on the reading list, the students don't buy even one; but this way they can bring together chapters from all of them." The mix can sometimes be highly unexpected, as when one lecturer decided to incorporate a chapter on discourse analysis from a book on linguistics into his specially created politics textbook.

Tough competition and other challenges have also streamlined publishing processes. Stuart Hay, textbook development manager at Pearson Education, explains how his company gets authors to submit their books in four stages. They are then given detailed comments by a team of specialist advisers who are teaching on relevant courses - and sometimes get feedback from students as well. These are asked about style, content and pedagogic methods, but the crucial question, of course, is: "Would you adopt this book?"

Hay or one of his development editors then sends all this back to the author, together with their own comments as "naive readers" (where they got bored, what seemed unnecessary) and a summary flagging up points mentioned by more than one adviser. "Authors love it but are totally surprised by it," he says. "The first batch of chapters is often heavily criticised, which can be painful and ego-bruising, but then slowly the response improves."

The process ensures that the finished textbook is perfectly adapted to the needs of the market, while also providing a level of guidance on how to write that one would be lucky to get on a creative writing course and would never receive for a paper submitted to a peer-reviewed journal.

So what is the future of the textbook, and how can we keep up the stream of potential authors? "My feeling is that the traditional hard-copy textbook is here to stay, at least in the short to medium term," says Lloyd, "unless a piece of portable technology comes along that makes it easier to read and digest large amounts of text on screen. The coupling of a traditional hard-copy textbook - with all the advantages of portability, ease of reading and presentation of content on a page - with online supplementary materials is a powerful combination."

And the reluctance of British academics may be set to change as universities are forced to compete for students who are more interested in the quality of teaching than in research citations. "In the States," Lloyd observes, "it is crucial for departments to attract students to their courses, because they are a key source of revenue. And using a professor who has written textbooks as a way of doing this is much better established. As competition between universities hots up because of student fees, we may get to see the same thing over here."

If we want to train a new generation of researchers, Pearson Education's Langeskov argues, we need textbooks. But perhaps we also need to remove some of the obstacles to their being written.

ÖйúAƬ

"Somehow the academy has to make an allowance for the value of educational materials," he points out, "and I wonder whether that happens enough - whether the rewards for being a good teacher, whether in the classroom or by writing a good textbook, are sufficiently apparent. I wonder if the balance is quite right."