This week, the head of the General Medical Council urged the NHS to shed its paternalistic culture. Linden West explains the urgency of the situation.

Manchester GP Harold Shipman embarks on an epic killing spree. Kent GP Clifford Ayling is found guilty of the serial sex abuse of female patients. Both are extreme cases, but rarely does a week pass when one doctor or another is not being dragged across the coals by the media. In many cases, reports focus on clinical effectiveness and involve accusations that doctors lack empathy, are poor communicators and worse listeners. Medics and medicine, it seems, are in poor health.

What is wrong with the profession? That was a key question at the heart of an in-depth, biographical study I conducted into those who should know best - doctors themselves. Some 25 general practitioners were interviewed up to six times over a period of four years. Questions focused on the nature of their role in changing and challenging times. The group comprised single-handers and members of group practices; men and women; the newly qualified and those approaching retirement; blacks, whites and Asians. All worked in difficult inner-city areas, where health needs are greatest and resources, including numbers of doctors, most severely stretched.

The study produced a series of insights into the health of the profession. It shows poor morale, defensiveness (and some of its causes); racism that blights the careers of many "minority" doctors who work in parts of cities that white colleagues avoid; inadequacy of initial training and feelings of impotence; and the impact of patients' disturbance on the emotional health of doctors in a profession where mental health issues - for doctors as well as patients - have a low priority.

ÖйúAƬ

Most of the doctors in the study work in inner London. The capital has the highest incidence of serious mental illness in the United Kingdom. The poorer you are and the more run-down the area in which you live, the more stretched the services available. This situation has a disturbing impact on the morale of many inner-city doctors, particularly the large number of single-handers. Michael Balint, the distinguished psychoanalyst, once observed that disturbance disturbs: doctors have to find a way of dealing with and transcending this. Incidents of depression, alcoholism and suicide among doctors are on the increase, yet many find it hard to discuss their emotions. Doctors, it seems, are not good patients. Their stories reveal, time and again, how much medicine remains a deeply individualistic, masculine culture, where it is hard, even dangerous, to talk about emotional difficulties.

There is an omnipotent myth at work here: the idea that doctors ought to cope and should know best. Patients expect it, doctors tell themselves. Many of the doctors said that coming to terms with their own inadequacy and uncertainty was the hardest task of all. This is a serious problem, given that reality often disappoints, and many struggle to cope with their negative feelings.

ÖйúAƬ

Gender plays an important role, too: all doctors can easily succumb to periods of depression, even despair, but some of the older male GPs found it hardest to cope with feelings of failure and inadequacy. One doctor was haunted by a minor operation that went wrong 20 years ago. The bad feelings remained unresolved because he had been taught to bottle them up rather than explore them.

The GPs felt they were on the edge of their profession, given that specialist, scientific knowledge tends to be deified, high-tech medicine revered and general practice frequently disparaged as an occupational backwater. The prominence given to hospital-based medicine means that GPs spoke of a profound neglect in their training of the skills and understanding more central to general practice. Patients want the most effective treatment, but what is also needed is a cultural and emotional literacy, biographical insight alongside science, the ability to empathise with patients and their particularities as well as knowledge of the latest epidemiology. Despite the growth of "effective communication" modules in medical schools, many mainstream medics (and some trainees) disparage cultural and psychological perspectives on health as subjective and unimportant.

A large part of the problem is the power of science in medicine. Two GP academics, Trisha Greenhalgh and Brian Hurwitz, have written about the relentless substitution during medical training of skills "that are fundamentally linguistic, empathic and interpretative for those deemed 'scientific' - that is, eminently measurable but reductionist".

This, they conclude, is a deeply unsatisfactory feature of the modern medical curriculum. Experience-based insight, for instance, is often derided as anecdotal and particular when compared with evidence-based medicine derived from large population samples. In the field of continuing medical education, Balint's approach and self-directed learning groups - using case study methods designed, in part, to enable doctors to consider all aspects of a patient's problem as well as the emotional impact of a problem on the doctor - remain a "minority sport", underdeveloped, under-resourced and under-supported.

ÖйúAƬ

The doctors' stories revealed abundant evidence of damaging splits in training and the literature between psychological and physical medicine, scientific and personal experience, and the patient's particular narrative and wider evidence. Despite claims that medicine is encompassing multiple ways of knowing a world, the objective and arguably male way - taking us into the real world and out of ourselves - remains dominant.

Many of the stories also reveal that it is precisely the capacity to learn from "otherness" and the patient within that lies at the core of effective practice. For example, East London-based Aidene Croft explained that being lesbian was important to her work, but that she did not fit easily into the male and predominantly heterosexual culture of medics. Critical of the neglect of psychological medicine in her training, she said she learnt to be a doctor only after she left medical school. She suffered a breakdown, underwent psychotherapy and used this, and her experiences of being silenced, to connect with many of her marginalised patients.

Croft, like others in the study, can be seen as a lifelong learner in the deepest sense. She developed an eclectic approach to her work, drawing on a range of therapies. She researched mental illness in response to an epidemic of mental health problems in the communities she served. Her patients knew, she said, that she understood what it was to be an outsider and could empathise with their distress. Most general practice was messy and imprecise - what mattered was the capacity to listen to a patient's story, in all its dimensions, and to relate this, in dialogue with the patient, to wider evidence.

Croft's narrative, like many others in the study, illuminated the neglect in medicine of the learned humanity lying at the heart of all good doctor-patient relationships.

ÖйúAƬ

Which brings us back to the problem of bad communication. This is often the result of doctors' fear of emotion - their own and those of their patients. At one end of the equation, in the destructive and arrogant behaviour of doctors such as Shipman and Ayling, there may well lurk a deeply defended denial of the doctor's humanity. Their two stories, however extreme, hold up a psychological mirror to the whole profession, asking us to reflect on the role of emotions in being a doctor and the potentially dangerous consequences of neglecting them.

Linden West is senior lecturer in education at the University of Kent. He also works as a psychotherapist. His book, Doctors on the Edge; General Practitioners, Health and Learning in the Inner-City , is published this month by Free Association Books ( ), ?16.95.

ÖйúAƬ