You can make history or study it, and a few lucky people such as Winston Churchill did both. But there is a third and far more terrible possibility, invented in the 19th century and flourishing and expiring in the 20th. It is to make History with a capital H - a cause to live and die for - and it was once an indispensable impulse of the totalitarian idea. History is your master; you shake your manacles gratefully, rejoice in submission to its laws, and obey.

This mood can still be recalled by anyone in their middle years or beyond, and its origins are German. At the University of Berlin after 1818 G.W.F. Hegel was its supreme theorist; and the young Karl Marx, when he discovered communism in Paris in the early 1840s, excitedly called it the solution to the riddle of history. By the 1930s thousands of intellectuals in many lands had answered the call.

Cecil Day-Lewis summed it up in his communist youth in a poem called The Road These Times Must Take (1934), which he did not reprint in his later years as Poet Laureate. Why, he asked, does meeting a communist make you feel small? "There fall from him shadows of what he is building", and he is "the future walking to meet us all".

Years later, in his memoir Witness (1952), Whittaker Chambers, the American writer, editor and Soviet spy, recalled how in his New York youth the promise of communism had sounded as beguiling to his ear as the serpent's whisper in the Garden of Eden: "Ye shall be as the gods." It was a revelation that conferred power. Man's mind had replaced God as the supreme creative intelligence, and the October Revolution of 1917 had thrown down an inescapable challenge: "Have you the moral strength to take upon yourself the crimes of history so that man may close his chronicle of age-old, senseless suffering and replace it with a purpose and a plan?" Or, as Bertolt Brecht put it uncompromisingly in Die Massnahme (1930), "embrace the butcher".

ÖйúAƬ

Adolf Hitler, who often spoke privately about how much he owed to Marxism, borrowed its fundamental doctrine of the inevitable laws of history and adapted them to the idea of the nation. There is a dazzling account of his gifts as a theorist by the historian Arnold J. Toynbee, called "A Lecture by Hitler", which tells of a private visit to the Chancellery in February 1936 at Hitler's invitation, a week before he remilitarised the Rhineland. As a professor in London, he had heard a good many historical lectures, but nothing like this.

Toynbee eventually collected his memories in Acquaintances (1967), and there he recalled how Hitler delivered his lecture in private with no one present but a handful of Nazis, who sat and listened in respectful silence. It lasted for more than two hours - with only one interruption, from Toynbee himself - and was an eloquent recital of European history since the 6th-century Merovingians, celebrating 1,000 years of German guardianship of Europe against the hordes of Asia, all delivered without notes and with a spontaneous lucidity Toynbee had never known in all his academic career. National Socialism, Hitler announced, would rival Marxism and outdo it; it would reject cosmopolitanism in favour of the oncoming and inevitable hegemony of the German people.

ÖйúAƬ

As a performance the lecture was marred only by moments of hysteria. Whenever Hitler mentioned Russia, his voice would rise to a scream, reminding Toynbee irreverently of a famous monkey in London Zoo that would fly into a rage whenever a spectator uttered the word "policeman". Hitler's obsession with Russia had little to do with communism, a doctrine to which in conversation he often gratefully acknowledged a debt. In Spandau: the Secret Diaries (1976), for example, Albert Speer told how Hitler remarked to him in January 1943, at the height of the Soviet campaign, that General Franco headed "a reactionary crew" in Spain where all the idealism lay with the Reds, and added that one day he would begin the Spanish Civil War again "with us on the other side", fighting reaction shoulder to shoulder with the communists. The Slavic hordes of Asia, by contrast, whether Tsarist or communist, blocked his path and must go.

With the Soviet Union's collapse in 1991, historical inevitability ceased to be fashionable, and nobody talks now of scientific socialism and few of the coming crisis of capitalism. A dogma that once seduced the intelligentsia lies shattered like a museum piece. On the other hand, it can be of compelling interest to fit the pieces into place, and that is what the Brookings Institution in Washington did with the scattered papers of Isaiah Berlin in The Soviet Mind: Russian Culture under Communism (2004). They laid ungathered, even unpublished, for years after his death in 1997, composed by someone who witnessed two revolutions in 1917 as a boy, and who in 1945 worked for six months in the British Embassy in Moscow.

Berlin was born in Riga in 1909, and during the First World War his family moved to Petrograd, out of the path of the Imperial German army. Russian and German were his native languages. During the war against Hitler he worked in the British Embassy in Washington, and his best-remembered book is The Hedgehog and the Fox (1953), which divides writers and thinkers into two categories: "hedgehogs", who relate everything to a central, all-embracing system, and "foxes", who are fascinated by an infinite variety of things. "The fox knows many things, but the hedgehog knows one big thing."

Joseph Stalin and Hitler, who believed that history was all class or all race, were classic hedgehogs. They believed in one big thing. Intellectuals like to look complex; but their deeper reflections can be simple, and they love, above all things, a terminology that looks complex but is nonetheless quickly intelligible and easily portable. An ideology needs to be ready for instant use, after all, a requirement natural to advanced consumer societies. The pocket razor is an instance; so is domestic lighting. When you flick a switch, light floods instantly over all the walls, an effect unknown to previous generations and yet easily taken for granted.

Life, by contrast, tends to be complex, and Berlin, who lived to be the first master of Wolfson College, Oxford and president of the British Academy, had ample opportunity to note its complexity. He was appalled by the gullibility of those who comfortably inhabited lands that had never known totalitarian rule. Such ignorance can be deliberate and cultivated. Few know, or wish to be told, that when the Nazis occupied eastern lands in 1941, notably Poland, Ukraine and the Baltic states, they borrowed extermination techniques from their Soviet allies and later admitted it; or that Rudolf Hoess, writing his memoir Commandant of Auschwitz (published in 1958) as he awaited execution in a Polish prison after the war, told how he and his Nazi colleagues gathered information during the war from escapees of Soviet camps to use in their own shorter, sharper programme of death. The Holocaust was inspired by Stalin, and Hitler in conversation freely admitted that National Socialism was based on Marx. He also praised the Soviet leader as a genius. So there is still work to be done with the theme of Nazi-Soviet collaboration, including the Soviet use of Nazi camps such as Sachsenhausen and Buchenwald for their original purpose during the Soviet occupation that followed victory in 1945. Two rival dictators emulated each other as they sought to destroy each other.

Nor is it widely known that the first history of socialism in any language, Alfred Sudre's Histoire du Communisme (1849), which went through numerous translations and was promptly awarded a prize by the French Academy, thought it a reactionary idea. So socialism was not always seen as left wing, and anyone in Victorian times who contemplated its demand for an omnipotent state might well have been reminded of the monopolies of King Louis XIV. It offered order, stability and a return to the traditional values of the family and the tribe.

ÖйúAƬ

V.I. Lenin and Stalin, and after them Hitler and Mao Zedong, proudly saw themselves as the active instruments of History. Hitler's historic task was to destroy Zionism, Stalin's to destroy the bourgeoisie; and Berlin's case was that their dogmas bound them to create the evidence by which they chose to act. They predicted history by the infallible dictatorial method of ordering it to be done. They created truth. For the Nazis, a Jewish world conspiracy must exist, whether the evidence could be found or not; for Stalin, the bourgeoisie must be destroyed because Marx had said so. History became fact by the will of those who ruled. In an article written by Berlin in 1952 under the pen name "O. Utis", which is Greek for "nobody" - Stalin still had a year to live, after all - and called "The Artificial Dialectic", we are shown with shattering clarity how it was done.

As a dedicated Marxist, Stalin feared that the Marxian dialectic - thesis, antithesis, synthesis - might some day turn against him and destroy the system he had inherited from Lenin. He therefore determined to create his own. "As others produced artificial rubber and mechanical brains," wrote Berlin, so did Stalin in his purges create "an artificial dialectic whose results the experimenter himself could in a large degree control and predict." He was like a marksman painting the target around the place where the bullet had already struck.

ÖйúAƬ

Hitler, deeply impressed by his example, followed him. In the summer of 1941, when he resolved to attack his Soviet ally, he persuaded himself that the Jews had treacherously devised a way to land Germany once again in a war on two fronts. As Berlin put it, summarising Stalin, "human skill would be employed in aiding the cosmos to fulfil even more faithfully its own inner laws". Dictators created History, and history, in the long run, hit back.

The Stalinist terror, which came first, may have been three times as extensive as the Nazi terror, which is known to have killed some 9 million. It was not the quirk of a dictator's mind: it was originally devised by a Rhineland exile writing in the British Museum in London in the mid-19th century, and its toll was vast.

The Soviet Mind reverberates with the grief of those who suffered and those who survived: Osip Mandelstam, a poet who died in a Siberian camp in 1938; the novelist Boris Pasternak; and the poet Anna Akhmatova, whom Berlin met during his stint in the British Embassy at the end of the war. Their secret Leningrad meeting in 1945 was dangerously interrupted by Randoph Churchill, of all people, shouting indiscreetly from the courtyard below, and it is moving to learn that Akhmatova in private recited two cantos of Lord Byron's Don Juan from memory, to Berlin's amazement and embarrassment. He pays homage to them all, living and dead, revisiting the land of his childhood after two revolutions and a devastating war as one who had never ceased to admire the intellectual zest of "this most imaginative and least narrow of peoples". Though Russian was his first language he was a stranger there, an English academic of Latvian birth who had spent his adult life in England and the US. His diplomatic immunity protected him but not them, and their courage astounded him. This was an intelligentsia under deep freeze, though minds still stirred under the socialist permafrost among those prudent enough to avoid contention and brave enough to meet in secret places.

The Soviet collapse, in the end, was a collapse of theory. Stalin had believed that the dialectic of History could not be cheated of the mass sacrifice demanded by Marx, and he massacred by the tens of millions. So a theory devised in Western Europe in an earlier century was turned into a blueprint for Eastern Europe, with catastrophic results. Strange to recall that in the event the Soviet Union did not die because it was brutal, however, but because it failed to produce - a supreme irony, since Marx had claimed to be the first to link the theory of class to theories of production. The evidence that socialist economics did not work, abundant as it was, did not count until, more than 30 years after Stalin's death, it was too late to amend and reform it. By the 1980s, its economy hopelessly outpaced, the Soviet system had nothing to do but die.

The lessons of the Bolshevik failure are still to be pondered. Only National Socialism, as a phenomenon, is more improbable. But Bolshevik rule lasted far longer and conquered far wider, and it was more clearly than Hitler's the creation of an intelligentsia in love with theory; and just because it was a theory, it attracted intellectuals around the world and contrived to look all-knowing. It flourished as an abstraction and because it was that.

Events, however, were disobliging, and no class wars - never a one - followed any industrial revolution anywhere on earth. As a theory of history, Marxism proved a catastrophic dud. Far from impoverishing the working classes, what is more, the free market offered them a chance to grow richer by the millions, and they took it. If history is about what happened, History proved to be about what never happened. It was a failure in prediction. But all that took a tragic century and more to unravel after Marx's Communist Manifesto (1848), and even the murderous expectations of Marx and his disciples did not altogether foretell the enormity of what Stalin and Hitler did.

ÖйúAƬ

By now there are no excuses left to those who once believed in it all. Planned economies led to vast crimes; then History spectacularly failed. Now there is only the hanging judge of history, as Lord Acton dramatically called it, and its unblinking gaze, to tell what happened and why.