Einstein’s quest for unification

The definitive scientific biography of Einstein, Subtle is the Lord…, which was written by Abraham Pais in 1982, delivered an unequivocal verdict on Einstein’s quest for a unified field theory. Pais wrote that the time for unification had not come, and that Einstein’s work “led to no results of physical interest”. But a lot of water has flowed under the bridge of unification since then, allowing us to look back with perhaps more indulgence as we celebrate the centenary of Einstein’s 1905 papers.

Let us briefly recall the relevant physics that was known in the 1920s, when Einstein embarked on his quest. The only known subatomic particles were the proton and the electron: the neutron and the neutrino, for example, were not predicted or discovered until the 1930s. Most “fundamental” physicists were striving to understand quantum physics – an endeavour from which Einstein stood apart. The structure of the nucleus was regarded as an interesting but secondary problem, and the unification of forces was considered, in the words of Pais, a minor issue.

For Einstein and his few unification-minded colleagues the big issue was to unify general relativity – a theory of gravity – with Maxwell’s electrodynamics. Theodor Kaluza and Oskar Klein proposed starting from a 5D theory, which contained an extra “compactified” spatial dimension in addition to the three spatial and one temporal dimensions of everyday experience. Electromagnetism then emerged naturally from this extra dimension.

Perhaps more so than Pais, we now recognize these early theories as breakthroughs  in unification because of their many echoes in the supergravity and string theories of the past 20 years. Einstein was an early enthusiast; as he wrote to Kaluza in April 1919, “The idea of achieving unification by means of a five-dimensional cylinder world would never have dawned on me…At first glance I like your idea enormously”. Kaluza published his idea in 1921, which Einstein pursued in his first unification paper with Jacob Grommer the following year. Indeed Einstein was to return to 5D theories every few years for the rest of his life.

However, even Einstein had to admit that his unification papers were not always ground breaking. For example, after some initial confusion he recognized that the two papers he wrote in 1927 were equivalent to the work of Klein. But he might have been happy to know that some of today’s particle physicists will search for Kaluza-Klein excitations using the Large Hadron Collider at CERN. Einstein had hoped to identify quantum fields with such higher components that only arose in the 5D theories.

Another recurring theme in Einstein’s quest for unification was to generalize the “metric” of relativity – the symmetric tensor that describes the curvature of space-time – so that it could also describe the electromagnetic field. He pursued many apparently blind alleys, such as asymmetric generalizations of the metric, and even postulated that there might be no tensor at all. As Einstein himself said in a letter to Klein in 1917, “this process of deepening the theory has no limits”.

Unfortunately, these ideas were unsuccessful. For example, in his first unification paper in 1925 the antisymmetric part of his tensor field was not suitable for describing all the components of the electric and magnetic fields. Indeed, none of Einstein’s unification attempts ever reproduced the free-field Maxwell equations. In Einstein’s defence, it should be mentioned that we now recognize that other types of antisymmetric tensor fields emerge naturally from string theory. However, this type of theory had not been invented in Einstein’s day.

A more basic problem with many of Einstein’s proposals was that they did not include the general theory of relativity itself. However, in his final years following 1945 he returned to a theory with a fundamental tensor that was not symmetric and would include both the metric and the electromagnetic tensor, which avoided some of these problems.

No stone left unturned

It is difficult to accuse Einstein of leaving stones unturned – no matter how unpromising they might appear. For example, in the early 1940s he even toyed with the idea that nature might not be described by partial differential equations. Modern theorists can hardly be accused of excessive conservatism, but even they have not revived this startling speculation!

What is most impressive about Einstein’s quest for unification was his persistent indefatigability. He tried many different ideas, and often returned to earlier theoretical haunts, such as Kaluza-Klein theories, with something new to say. However, the truth is that he was adrift from many of the most important developments in physics at the time. For instance, he was famously sceptical – if not downright hostile – towards quantum physics, and he does not seem to have followed closely the discoveries of new particles and interactions. More surprisingly, perhaps, he seems to have missed out on some of the most far-reaching new theoretical ideas of that period, which now play key roles in modern approaches to unification.

For example, Einstein recognized Hermann Weyl’s seminal 1918 work on scale transformations in four dimensions, even paying it the backhanded compliment that “apart from the agreement with reality, it is at any rate a grandiose achievement of the mind”. Weyl’s ideas led to the discovery in the late 1920s of local phase transformations, which laid the foundations for the gauge theories of the weak and electromagnetic interactions in the 1950s and beyond. However, Einstein was never involved personally in these far-reaching developments.

He also seems to have been affected by frequent mood swings during his quest for unification. On several occasions he switched rapidly from unwarranted optimism about the prospects of a new idea to complete rejection. More alarmingly, his mood often swung in the full glare of publicity. For many years a new scientific paper by Einstein was a major public event, with hundreds of journalists hanging on the utterances of the great man. The closest present-day parallel would be Stephen Hawking and his recent comments on black holes and quantum mechanics.

Einstein’s legacy

Why were Einstein’s papers on unification not more successful? It is surely insufficient simply to say that only young theorists have brilliant new ideas. The many distractions of fame in his later years should also not get all the blame. Einstein himself wrote in his early years that “formal points of view…fail almost always as heuristic aids”. But later he seems to have abandoned this insight in his quest for unification, and instead was seduced more by mathematical novelty than by physical intuition.

It could be, however, that Einstein was simply ahead of his time, since even if he had been following contemporary physics more closely, the information available before his death was probably insufficient to make significant progress in unification. For example, the unification of the weak and electromagnetic interactions in the 1960s required many unforeseen experimental discoveries as well as new theoretical ideas. Even now, the unification of gravity with  the other interactions – which was Einstein’s true dream – still eludes us.

Following Einstein, most theoretical physicists assign a central role to geometrical ideas. Most of the particle-physics community believes, for example, that string theory provides the appropriate framework for realizing Einstein’s dream. Here, fascinating generalizations of Kaluza and Klein’s hidden dimensions, such as “Calabi-Yau manifolds”, are able to dispose of the several extra dimensions required by the theory. However, not all general relativists are convinced, and there is absolutely no experimental evidence for string theory. Are we also in danger of being seduced by formal beauty?

Although some of the unification ideas pursued by Einstein are now recognizable  in developments such as string theory, this is not to say that Einstein’s work actually inspired these modern unification attempts. It seems to me that the real significance of Einstein’s quest for unification lies in its quixotic ambition. Einstein, more than any of his contemporaries, put unification on the theoretical map and established it as a respectable intellectual objective. Even if we do not have all the necessary theoretical tools or experimental information, unification is the “holy grail” towards which our efforts should be directed.

About the author

John Ellis is in the Theory Division at CERN, Geneva, Switzerland.