# Boltzmann, Gibbs, and equilibrium statistical mechanics

# Boltzmann, Gibbs, and equilibrium statistical mechanics

# Abstract and Keywords

This chapter discusses equilibrium statistical mechanics for systems more complicated than monatomic gases, as well as the problem of the trend towards equilibrium of these systems. Ludwig Boltzmann is credited for having begun this branch of statistical mechanics with a basic paper written in 1884, in which he formulated the hypothesis that some among the possible steady distributions can be interpreted as macroscopic equilibrium states. This fundamental work by Boltzmann was taken up again, widened, and expounded in a classical treatise by Josiah Willard Gibbs. In his paper, Boltzmann described statistical families of steady distributions, which he called orthodes. Boltzmann showed that there are at least two ensembles of this kind, the ergode (Gibbs's microcanonical ensemble) and the holode (Gibbs's canonical ensemble). This article also explains why statistical mechanics is usually attributed to Gibbs and not to Boltzmann, the problem of trend to equilibrium and ergodic theory, and Max Planck's work on statistical mechanics.

*Keywords:*
gases, Josiah Willard Gibbs, Max Planck, equilibrium statistical mechanics, equilibrium states, orthodes, ergode, holode

# 7.1 Introduction

In this chapter we shall discuss equilibrium statistical mechanics for systems more complicated than the monatomic gases considered so far, as well as the problem of the trend towards equilibrium of these systems.

Once more must be ascribed to Boltzmann the merit of having begun this branch of statistical mechanics with a basic paper [1], written in 1884 and much less frequently quoted than his other contributions. In this paper he formulated the hypothesis that some among the possible steady distributions can be interpreted as macroscopic equilibrium states. This fundamental work by Boltzmann was taken up again, widened, and expounded in a classical treatise by Gibbs [2], and it is the terminology introduced by Gibbs that is currently used. As a matter of fact a statistical ensemble (in Gibbs’s terminology) is called a *monode* by Boltzmann. The question posed in the above-mentioned paper [1] is the following: what statistical families of steady distributions have the property that, when an infinitesimal change is made in their parameters, the infinitesimal changes in the average total energy of the system *E*, of the pressure *p*, and of the volume *V* are such that (d*E* + *p*d*V*)/*T* (where *T* is the average kinetic energy per particle) is an exact differential (at least in the thermodynamic limit, when *V* → ∞, *N* → ∞, whereas *N*/*V* remains bounded)? These families are called *orthodes* by Boltzmann. The answer given by Boltzmann to his own question is that there are at least two ensembles of this kind, the *ergode* (Gibbs’s *microcanonical ensemble*) and the *holode* (Gibbs’s *canonical ensemble*).

Although Boltzmann originated [1]] the study of equilibrium states for more general situations than that, already considered by Maxwell, of a dilute gas in the absence of external forces, it is not with his name but that of Gibbs that one usually associates the methods of this area (the most completely developed one) of statistical mechanics. Even the terminology (microcanonical, canonical, grand canonical ensembles) is that due to Gibbs, while the first two ensembles were clearly defined (with different names) and used by Boltzmann. It is then beyond doubt, in the words of Klein [3], that “it was Boltzmann, and not Maxwell or Gibbs, who worked out precisely *how* the second law is related to probability, creating the subject of statistical mechanics”.

(p.135) We may add, for the sake of clarity, that Gibbs invented the name of the subject. Gibbs was of course aware of Boltzmann’s priority, as we shall indicate later. Why then, beyond some generic acknowledgements to Boltzmann, it is Gibbs’s name that emerges? It is an interesting question, to which one may be tempted to give an easy answer.

Before embarking on this discussion, let us first say something about Gibbs.

# 7.2 A great American scientist of the nineteenth century: J.W. Gibbs

Let us recall, with the help of two fine papers by M. Klein [4, 5], who was that “Mr. Josiah Willard Gibbs of New Haven” who was appointed Professor of Mathematical Physics by the Yale Corporation on 13 July 1871. He was born on 11 February 1839 (the same month as Boltzmann, five years earlier), the only son and the fourth of the five children of Mary Anna Van Cleve and Josiah Willard Gibbs the elder. His father was a distinguished philologist who had graduated from Yale in 1809 and served as Professor of Sacred Literature in the Divinity School of the same University from 1826 until his death in 1861. Four generations of Gibbs and Willard sons had previously graduated from Harvard College, including Samuel Willards, in 1659, who was acting president of that institution at one time. Gibbs’s mother’s family background was also impressive: it included a series of Yale graduates, one of whom served as the first president of what is today Princeton University (then College of New Jersey), and several graduates from the latter institution who followed careers in science.

Gibbs graduated from Yale College in 1858, having won a series of prizes and scholarships for excellence in Latin and mathematics. Gibbs continued his studies in Yale’s new graduate school and received his PhD in 1863, one of the first scholars to be awarded this degree by an American University. His doctorate was in engineering and his dissertation, entitled “On the forms of the teeth of wheels in spur gearing”, was a sort of exercise in geometry and kinematics. He was then appointed a tutor in Yale College, where he gave elementary instruction in Latin and natural philosophy (physics) for three years. In the meantime he continued to work on engineering problems, and in 1866 he obtained a patent on an improved brake for railway cars. In the same year, however, he presented a paper to the Connecticut Academy of Arts and Sciences. The paper deals with the quantities used in mechanics, but contains a remarkably clear discussion of the dual roles played by the concept of mass in classical mechanics—inertial mass and gravitational mass—and of the confusion caused by defining mass as quantity of matter.

In the same year, in the month of August, Gibbs sailed for Europe for what turned out to be his only extended absence from his native city. He travelled with his sisters Anna and Julia, his parents and his two other sisters being no longer alive. He spent a year each at the universities of Paris, Berlin, and Heidelberg, attending a variety of lectures and reading widely in both mathematics and physics. The list of scientists whose lectures he attended is impressive, since it includes Liouvillie, Darboux, Kronecker, Weierstrass, Helmholtz, and Kirchhoff, but there is no indication that he was a research student anywhere or had yet begun any research of his own or was planning to do so.

(p.136) Two years after he came back to New Haven, Gibbs had no regular employment and his future activities were not clear either. He was evidently able to manage on the money inherited from his father. He never married, and continued to live in the family home with his unmarried sister Anna, and with Julia, her husband, and their growing family.

His financial independence and his scientific abilities must have been known within the Yale community, since he was appointed to the newly created professorship of mathematical physics in 1871, the official record including the unusual specification “without salary”. It is true that he taught only one or two students a year during his first decade as a professor. Two years later he received an offer of $1800 a year from Bowdoin College, which he refused. Only in 1880, when he was tempted to leave because of an offer from the new and appealing Johns Hopkins University, did his own university offer a salary; it was only two-thirds of what Hopkins would have paid, but was enough to convince him to stay.

Gibbs’s appointment in 1871 preceded his first published research by two years; this was not unusual at that time in America. We remind the reader that Boltzmann, five years younger than Gibbs, had published papers when he was a student and obtained a chair at the age of 25, in 1869 (see Chapter 1); in 1871 he was about to publish his most famous paper.

Gibbs’s first paper [6] was on thermodynamics and immediately demonstrated his mastery of the field. The choice of the subject shows no correlation to the lectures he had attended in Europe, and Gibbs, a very laconic writer, gives no hint as to the reasons for his choice, though his interest in steam engines between 1871 and 1873 might provide a clue.

The title of the paper, “Graphical methods in the thermodynamics of fluids” is not very promising, but its content quietly changed the content of thermodynamics by using entropy as an independent variable, something that not even Clausius had ever done. In particular he analysed the entropy–volume diagram and its “substantial advantages over any other method” because it shows the region of simultaneous coexistence of the vapour, liquid, and solid phases of a substance, a region which reduces to a point in the more usual pressure–temperature plane. We do not know, as Klein [5] points out, whether Gibbs learned of Thomas Andrews’s recent (1869) discovery of the continuity of the two fluid states of matter from Andrews’s paper itself [7] or from Maxwell’s *Theory of heat* [8].

Gibbs’s interests are much more apparent in his second paper [9], which appeared a few months later. Although the title might suggest an extension of his previous representation, he had clearly turned from methods to explanations. The problem Gibbs treated is the characterization of the equilibrium state of a material system. This state can be solid, liquid, gaseous, or a combination of these phases according to the circumstances, and was represented as a point of a surface of a space whose coordinate axes are energy, entropy and volume. This surface represents the fundamental thermodynamic equation of the body. Gibbs established the relationships between the geometry of the surface and the conditions for thermodynamic equilibrium and its stability. He showed that for two phases of the same substance to be in equilibrium with each other, not only must they have the same temperature *T* and pressure *p*, but also their internal energies *E* _{k} (*k* = 1, 2), entropies *S* _{k} (*k* = 1, 2), and volumes *V* _{k} (*k* = 1, 2) must
(p.137)
satisfy the equation

*Theory of heat*and corresponded on the subject of Andrews’s diagrams with James Thomson (1822–92), the elder brother of William Thomson, whom we have already met more than once. Thomson, a colleague of Andrews in Belfast, suggested that the isothermal curves for a fluid below its critical temperature, for the transition from the gaseous to the fluid state to be possible (in the traditional pressure–volume diagram), should really show a minimum and a maximum [10] rather than a straight line segment parallel to the volume axis, as showed by Andrews’s data. Between those extrema, the states pictured on the isotherm would be unstable—since temperature and volume would be increasing together—but such a curve would account for metastable states of supercooling and superheating, and provide what Thomson called “theoretical continuity”. In fact these isotherms proposed by Thomson looked very much like those that Johannes Diderick van der Waals (1837–1923) would derive in his dissertation two years later [11]. Maxwell’s question was: where must one draw the straight line segment that cuts across Thomson’s loop? Or, in physical terms, what is the condition determining the pressure at which gas and liquid can coexist in equilibrium?

In his letter to Thomson, Maxwell proposed a way of answering this question and repeated it in his book [8], arguing that the difference in internal energy between the two phases must be a maximum at the pressure where they coexist. He would change his mind after reading Gibbs’s paper.

When he analysed the conditions for the stability of thermodynamic equilibrium states [9], Gibbs arrived at a new understanding of the significance of the critical point. The critical state not only indicated where the two fluid phases became one, but it also marked the limits of instability associated with the two phase system. Gibbs’s analysis also led him to a series of new explicit conditions that must be fulfilled at the critical point and that can serve to characterize it.

As remarked by Klein [5], Gibbs could expect his work to be circulated far and wide by the fact that it appeared in the *Transactions of the Connecticut Academy of Arts and Sciences*, a society of which Gibbs had been a member since 1858. Although this academy was of a local character, being centred in New Haven, it had been in existence since 1799 and had developed a regular programme for the exchange of its *Transactions* with similar journals published by some 170 other learned societies, ranging from Quebec and Melbourne to Naples and Moscow. We know however that he did not rely on reaching only those potential readers who might happen to pick up the *Transactions* in their own local academies: he sent copies of his papers directly to some 75 scientists at home and abroad [12]. We cannot tell how many of those actually read his first two papers, but we do know of one, the crucial one, Maxwell. He read Gibbs with enthusiasm and profit. In fact, he had misused the term entropy in the first edition of his book [8], where he had followed his friend Tait. The error was corrected in later editions, after Maxwell had learned the proper definition from Gibbs’s papers.

(p.138) Maxwell talked about Gibbs’s work to his colleagues in Cambridge in several fields, and wrote about it to others, recommending it highly. He especially appreciated the geometric approach contained in this work and was fascinated so much by the thermodynamic surface introduced in Gibbs’s second paper that he actually constructed such a surface showing the thermodynamic properties of water, and sent a plaster-cast of it to Gibbs. He went so far as to discuss the surface at considerable length in the 1875 edition of his book [13], though that edition appeared in a series described by its publisher as consisting of “text-books of science adapted for the use of artisans and of students in public and science schools”.

In his lecture “On the dynamical evidence of the molecular constitution of bodies”, delivered at the Chemical Society on 18 February 1875 [14], Maxwell says:

The purely thermodynamical relations of the different states of matter do not belong to our subject, as they are independent of particular theories about bodies. I must not, however, omit to mention a most important American contribution to this part of thermodynamics by Prof. Willard Gibbs, of Yale College, U.S., who has given us a remarkably simple and thoroughly satisfactory method of representing the relations of the different states of matter by means of a model. By means of this model, problems which had long resisted the efforts of myself and others may be solved at once.

In this famous lecture Maxwell introduced his own ingenious method of demonstrating where to draw the horizontal line in the Thomson–van der Waals curve (the so-called *Maxwell rule*, according to which the area above the segment must equal that below) and paid a handsome tribute to van der Waals as well, because the attack of the Dutch scientist on the difficult question of a molecular explanation of the continuity of the liquid and gaseous states “is so able and so brave, that it cannot fail to give a notable impulse to molecular science. It has certainly directed the attention of more than one inquirer to the study of the Low-Dutch language in which it is written.”

The Connecticut Academy had some twenty members and had regular meetings. At one of these, in June 1874, Gibbs gave a talk, presumably not too exciting for his audience, on the application of the principles of thermodynamics to the study of thermodynamic equilibrium, which was gradually worked out into a long memoir “On the equilibrium of heterogeneous substances” [15]. This occupies about 300 pages of his collected papers and we can subscribe to the words of M.J. Klein, according to whom it “surely ranks as one of the true masterworks in the history of physical science” [5].

In his memoir, Gibbs greatly enlarged the domain covered by thermodynamics: in fact, he treated chemical, elastic, surface, and electrochemical phenomena by a single, unifying method. He described the basic ideas underlying this work in a lengthy abstract (17 pages) which appeared in the *American Journal of Science* [16], which surely reached a much wider audience than the *Transactions of the Connecticut Academy of Arts and Sciences*. The motivation for his work is clearly stated in the following sentences:

It is an inference naturally suggested by the general increase of entropy which accompanies the changes occurring in any isolated material system that when the entropy of the system has reached a maximum, the system will be in a state of equilibrium. Although this principle has by no means escaped the attention of physicists, its importance does not appear to have been duly (p.139) appreciated. Little has been done to develop the principle as a foundation for the general theory of thermodynamic equilibrium.

Gibbs’s memoir set forth exactly that development. In particular, the general criterion for equilibrium was stated simply and precisely: “For the equilibrium of any isolated system it is necessary and sufficient that in all possible variations of the state of the system which do not alter its energy, the variation of its entropy shall either vanish or be negative.”

In order to work out the consequences of this general criterion, and to explore their implications allowing for the variety and complexity that thermodynamic systems can have, Gibbs introduced chemical potentials from the outset. These intensive variables must be constant throughout a heterogeneous system in equilibrium and play a role similar to that of temperature and pressure. Starting from these considerations, Gibbs derived his famous phase rule, which specifies the number of independent variables (degrees of freedom) in a system of a certain number of coexistent phases having a specified number of chemical components [15]. Although this phase rule proved to be a milestone for understanding an incredible amount of experimental material, Gibbs did not underline its importance in any special way.

It was van der Waals who first saw the great power of that simple and basic rule. It took, as Pierre-Maurice Duhem (1861–1916) commented, “a remarkable perspicacity” on van der Waals’s part to perceive the phase rule “among the algebraic formulas where Gibbs had to some extent hidden it” [17]. Duhem also wondered how many more such seeds that might have grown into whole programmes of research “had remained sterile because no physicist or chemist had noticed them under the algebraic shell that concealed them?” [17]

This shows that Gibbs’s memoir had become widely known in Europe and had received the recognition it merited. It was translated into German by W. Ostwald, into French by Henry-Louis le Chatelier (1850–1936). Duhem wrote a letter to Gibbs on 29 May 1900, stating that this memoir crowned the nineteenth century in much the same way that Lagrange’s *Mécanique analytique* crowned the eighteenth. Another opinion of the same memoir is due to Gibbs’s student, Edwin Bidwell Wilson, who compared his achievement in it to starting with a knowledge of only the first book of Euclid and developing all the rest for oneself [18].

Gibbs’s succinct and abstract style and his unwillingness to include examples and applications to particular experimental situations made his work very difficult to read. Such famous scientists as Helmholtz and Planck developed their own thermodynamic methods in an independent fashion and remained quite unaware of the treasures buried in the third volume of the *Transactions of the Connecticut Academy of Arts and Sciences*.

Gibbs did not write any other major paper on thermodynamics and limited himself to a few short papers elaborating several points in his long memoir [15]. Perhaps he thought that he had said all he needed to say on the subject. He had shifted his attention to other issues of physics and mathematics (vector analysis, calculations of orbits, investigations into the electromagnetic theory of light, a new variational principle for mechanics, the famous Gibbs phenomenon in the Fourier series [12]). Thus he rejected all suggestions to write a treatise on thermodynamics that would expand his ideas and make his work
(p.140)
more accessible. Among the people who urged him to do this was Lord Rayleigh, who on 5 June 1892 wrote to Gibbs that the original memoir was “too condensed and too difficult for most, I might say all, readers”. Gibbs’s answer is rather surprising: he now thought that his memoir seemed “too *long*”, and showed a lack of a “sense of the value of time, of my own and others, when I wrote it”. These letters, as well as the one by Duhem quoted below, are part of the Gibbs Collection in the Yale University Library.

Gibbs agreed to a republication of his writings of thermodynamics only shortly before his death. He planned to add some new material to the book, in the form of additional chapters. The bare titles of two of these chapters, “On similarity in thermodynamics” and “On entropy as mixed-up-ness” [19] whet our appetite to know more about their content, but we can only guess. The first might have dealt with what is nowadays called the law of corresponding states, the second with the mixing process that Gibbs used, as we shall discuss below, to explain the trend of an isolated system to thermodynamic equilibrium.

# 7.3 Why is statistical mechanics usually attributed to Gibbs and not to Boltzmann?

At the time when he developed his exposition [2] of statistical mechanics, Gibbs was at the end of a scientific career devoted to the study and application of thermodynamics, in which abstract thoughts were illuminated by several geometric representations, but not by images based on mechanical models typical of atomism. The energeticists, who, as we have seen in Chapter 1 and shall discuss in more detail in Chapter 11, opposed and even scorned the use of molecular ideas, particularly valued Gibbs for having avoided these ideas in his work. In particular, G. Helm praised his thermodynamic writings because they “established the strict consequences of the two laws with no hankering or yearning after mechanics” [20]. Yet Gibbs in his new book was discussing the principles of statistical mechanics.

Although he frequently mocked the procedures typical of mathematicians [21], Gibbs was no less severe than they were in applying stringent logic and preoccupied with avoiding publication of incomplete results. To summarize, he had a character and a way of proceeding that were almost diametrically opposed to Boltzmann’s, who had his intuition, his faith in mechanical models, and his enthusiasm as winning tools, with the obvious consequence of a very large number of papers, of preoccupying length. On this aspect, Maxwell had written several years before (1873) to Tait [22]:

By the study of Boltzmann I have been unable to understand him. He could not understand me on account of my shortness, and his length was and is an equal stumbling-block to me. Hence I am very much inclined to join the glorious company of supplanters and to put the whole business in about six lines.

To be sure, Maxwell had died five years before Boltzmann’s paper [1], but if this was Maxwell’s opinion, we can imagine what opinions people might have had on Boltzmann’s style and statements in times and places more hostile to the atomic theory of matter. This is what we have earlier called an easy explanation of the fact that (p.141) Boltzmann’s paper [1] was practically forgotten and Gibbs’s treatise became the standard reference for equilibrium statistical mechanics.

In his paper considering Boltzmann’s paper, Gallavotti [23] claims that the reason for its obscurity is due rather to the fact, hinted at in previous chapters, that Boltzmann’s work is known only through the popularization of the Ehrenfests’ encyclopaedia article [24], which is as good a treatise on the foundations of statistical mechanics as it is in having little to do with many of Boltzmann’s key ideas. In fact, people who have had a chance of talking with illustrious physicists who had been students of Paul Ehrenfest may realize, by reading Boltzmann and the encyclopedia article [24], that when such physicists said that they were reporting Boltzmann’s views, they were really talking about that article. Gallavotti [23] should be mentioned here, not only because he appears to be the second author after Klein to underline the importance and the basic role of Boltzmann’s paper [1], but also because he proposes a new etymology of the words *monode, ergode, holode*. This has some relevance, even if these terms have disappeared from common usage, because the second of them has given rise to the much used adjective *ergodic*, which we shall meet later in connection with the ergodic problem. It is commonly surmised that the second part of these words comes from the Greek term ŏδóς (path). Gallavotti argues that it comes from εĩδος (aspect, form, method, way, state; related to the Sanskrit word *vedah*). This is very interesting, because it would suggest something related to a state rather than an evolution, but it is a little puzzling; the present author must confess that he has not the necessary philological competence to discuss this point and in particular why Boltzmann rendered the end of the above German words with *ode* rather than *ide* or *ede*. It is true that the perfect tense of the related verb εĩδον (to see) transforms the ε into an ο. On the other hand, Boltzmann should have written “monhode” if he thought of a relation with ŏδóς (as in “hodograph”). Presumably we must accept Gallavotti’s interpretation, unless more cogent arguments are suggested against it; in fact he must have discussed it with his father Carlo Gallavotti (1909–92), a distinguished expert on Greek literature, as indicated in the acknowledgements [23].

We can tentatively add a third explanation for the unfamiliarity of Boltzmann’s work, by introducing the role played by Niels Bohr. He was probably the most influential scientist of this century (in the words of Max Born, when proposing both Einstein and him as foreign members of the Göttingen Academy of Sciences: “His influence on theoretical and experimental research of our time is greater than that of any other physicist.”) Bohr had a great opinion of Gibbs, expressed over and over, because he had introduced statistical ensembles, but did not think highly of Boltzmann, and he may have induced mistrust of Boltzmann’s work in a large number of physicists of the twentieth century. It is clear that he had not read Boltzmann, who had been the originator of the first two kinds of ensemble. This is confirmed by his co-worker L. Rosenfeld in an interview on Bohr given to T. Kuhn and J.L. Heilbron, and kept in the Niels Bohr Archive in Copenhagen. These are the relevant sentences of Rosenfeld: “I don’t think that he had ever read Boltzmann, or at any rate not Boltzmann’s papers. He had read, I think, and studied very carefully Boltzmann’s Lectures on gas theory” and “He said that Boltzmann had spoiled things by insisting on the properties of mechanical systems.”

(p.142) Apparently Bohr was not alone, for Einstein once said to one of his students: “Boltzmann’s work is not easy to read. There are great physicists who have not understood it.” [25]

Bohr’s opinions go back to his youth, because he had already expressed reservations about Boltzmann’s views in 1912, when lecturing on the statistical foundations of thermodynamics in Copenhagen [26].

# 7.4 Gibbs’s treatise

In his treatise [2], published as one of the volumes in the Yale Bicentennial Series, Gibbs shows himself to be worrying mostly about the degree of generality of his exposition. Thus he abstains from any assumption on the microscopic constitution of matter, and renounces following the time evolution of a particular system. He sets himself the task of determining the way in which an ensemble of systems of the same kind distributes among the possible phases, starting from the distribution at a given instant of time. His theory is thus a branch of rational mechanics, a sort of projection of the latter discipline on to thermodynamics, a projection performed with analogies, the validity of which is discussed by Gibbs himself.

It is to be remarked however that Gibbs, notwithstanding all his above-mentioned preoccupation about the generality of the principles upon which statistical mechanics should be founded and for just this reason not so free with his references, pays a tribute to Boltzmann, underlining the fact that the Austrian scientist was the first, beginning in 1871, to consider explicitly the phase distribution of a large number of systems, as well as to study the evolution of this distribution through Liouville’s theorem [2, Introduction and *passim*].

Gibbs’s book must have been a real surprise to the scientific community, particularly since it would have seemed to represent not just a change of direction on the author’s part, but an actual reversal. In fact, in the introduction he states that only the principles of statistical mechanics could supply the “rational foundation of thermodynamics” and that the laws of the latter discipline were only an “incomplete expression” of these more general principles. Yet it was not the first time that he was referring to molecular behaviour: one may mention his treatment of the reaction of NO_{2} to N_{2}O_{4} [15], his mention of “the sphere of molecular action” in the analysis of the thermodynamics of capillarity [15], and the famous Gibbs paradox [15], to which Grad alluded in a passage quoted in Chapter 5.

Actually this paradox is no paradox to those who have understood the true meaning of entropy according to statistical mechanics. It arises when two gases are allowed to mix at constant temperature and pressure; then there is an increase of entropy, which is natural, because as we know, the state in which the molecules of the two gases are intimately mixed is more probable than the state in which the two gases are completely separated. The amount of increase in entropy is “independent of the degree of similarity or dissimilarity” between the two gases, unless they are identical—the same gas—in which case there is no entropy increase at all. Gibbs’s description of this situation is explicitly molecular and we can perhaps find here the first seed that would grow into the (p.143) treatise of 1902. The reason why there is no paradox is that a given system cannot always be assigned a unique value, its “entropy”. It may have many different entropies, among which we can choose according to our interests, the particular phenomenon under study, the degree of precision available or arbitrarily decided upon, or the method of description which is employed. For example, an aeronautical engineer studying the motion of air past an aircraft can think of the entropy of the air itself, as if all its molecules were identical, although we know that it is a mixture of gases, mainly nitrogen and oxygen, the molecules of which are decidedly different. If a nuclear engineer wants to separate two isotopes of uranium, he will deal with extremely similar molecules, but he will need an entropy that takes into account the slight difference between them, the only one that can be used for the purpose of separating the two isotopes. Hence there is a hierarchy of entropies which take more and more detail into account. When the molecules are really identical, we can only conceptually think of them (in classical mechanics) as different by applying to each of them a label; then this highly detailed entropy (Gibbs entropy) will remain constant in time, because any state will be equally probable when measured in terms of it. Thus the paradox arises because entropy is a property not of the system but of our description.

The example provided by his paradox must have convinced Gibbs of the importance of studying statistical mechanics. In fact he wrote [15]: “In other words, the impossibility of an uncompensated decrease of entropy seems to be reduced to improbability”, the sentence that Boltzmann would choose as a motto for his second volume of his lectures on gas theory [27].

By 1892 Gibbs was devoting much of his energy to writing up the results of his years of work on statistical mechanics [5]. In the above-mentioned letter to Lord Rayleigh, he wrote:

Just now I am trying to get ready for publication something on thermodynamics from the a priori point of view, or rather on ‘statistical mechanics’, of which the principal interest would be in its application to thermodynamics—in the line therefore of the work of Maxwell and Boltzmann. I do not know that I shall have anything particularly new in substance, but shall be contented if I can so choose my standpoint (as seems to me possible) as to get a simpler view of the subject.

It would not be appropriate here to make a detailed exposition of Gibbs’s short treatise. We shall restrict ourselves to briefly examining his ideas on the trend to equilibrium and his thermodynamic analogies. He did not quote Boltzmann’s equation or his combinatorial method, or Boltzmann’s latest elaborations and clarifications of his views on irreversibility. There are no indications that he read Boltzmann’s lectures on gas theory.

In order to make Gibbs’s passages quoted below more easily understandable, we recall that he uses the term “index of probability” to mean the logarithm of the probability density in phase space, *P*, that we considered in Chapter 4. On the problem of trend to equilibrium, we can read [2, Chapter XII]:

It would seem, therefore, that we might find a sort of measure of the deviation of an ensemble from statistical equilibrium in the excess of the average index above the minimum which is consistent with the condition of invariability of the distribution with respect to the constant functions of phase. But we have seen that the index of probability is constant in time for each system of the (p.144) ensemble. The average index is therefore constant, and we find by this method no approach toward statistical equilibrium in the course of time.

Here Gibbs deals with the fact that at the level of the Liouville equation one cannot talk of a trend to equilibrium, if all the states (the phases) compatible with a given energy can be reached during the time evolution of the system. In order to avoid this deadlock, Gibbs invokes the example of a liquid mass, made of two immiscible components having different colours, white and black, in a container of cylindrical shape: at time zero, the two liquids are clearly separated, but if we spin the cylinder, the white and black parts start tracing narrow ribbons which embrace the axis in a spiral shape, with a thickness that decreases in time: the liquid tends to form perfect mixing of white and black, but after any finite time interval, the total volume will be subdivided into two parts, strictly intertwined, of different colour. He also tries to give a more precise explanation of the trend of a given ensemble towards a state of statistical equilibrium, by imagining subdivision into equal elements Δ*V*, which are assumed to be not infinitesimal but so small that, at least initially, the probability index is essentially constant in the interior of each element. Then one can see (see Appendix 7.1) that the average index, computed through the subdivision into cells, at subsequent time instants will be lower than its initial value. This does not violate the time reversibility of the equations. It seems appropriate to make two comments here: first, that the analogy with a liquid is justified by the fact that the probability density in phase space behaves like the density of an incompressible fluid, which is a good model for a liquid (see Chapter 4); second, the idea of convergence to equilibrium that Gibbs tries to convey here is akin to the concept of weak convergence in mathematics.

In order to find a basis for thermodynamics in analytic mechanics, Gibbs then studies the coincidence existing between the equations governing the statistical ensembles and the general principles of thermodynamics [2, Chapter XIV]: but, he says,

however interesting and significant this coincidence may be, we are still far from having explained the phenomena of nature with respect to these laws. For, as compared with the case of nature, the systems which we have considered are of an ideal simplicity. Although our only assumption is that we are considering conservative systems with a finite number of degrees of freedom, it would seem that this is assuming too much, so far as the bodies of nature are concerned. The phenomena of radiant heat, which certainly should not be neglected in any complete system of thermodynamics, and the electrical phenomena associated with the combination of atoms, seem to show that the hypothesis of systems of finite number of degrees of freedom is inadequate for the explanation of the properties of bodies […]. But, although these difficulties, long recognized by physicists (See Boltzmann, Sitzb. der Wiener Akad., Bd. LXIII, S. 418 (1871)), seem to prevent, in the present state of science, any satisfactory explanation of the phenomena of thermodynamics as presented to us in nature, the ideal case of systems of a finite number of degrees of freedom remains as a subject not devoid of a theoretical interest, and which may serve to point the way to the solution of the far more difficult problems presented to us by nature. And if the study of the statistical properties of such systems gives us an exact expression of laws which in the limiting case take the form of the received laws of thermodynamics, its interest is so much the greater.

Gibbs also says that the notion itself of canonical ensemble “may seem to some artificial and hardly germane to a natural exposition of the subject”. However, if we take temperature as an independent variable, it is the canonical ensemble that gives (p.145) the best mathematical representation of a body at a given temperature, whereas it is natural to have recourse to the microcanonical ensemble when energy (a natural notion in mechanics) is considered as an independent variable.

# 7.5 French scientists on statistical mechanics

It is to be remarked that Hadamard, in 1906 [28], wrote an analysis of Gibbs’s treatise, with particular attention to the possibility of explaining the irreversible processes within statistical mechanics.

Hadamard recalls that the equations of analytical mechanics possess, generally speaking, the so-called Poisson stability, according to which the representative point of a system passes again infinitely many times in a neighbourhood as small as we like of any position which it has already occupied, apart from a zero volume set (Poincaré’s recurrence theorem, see Chapter 5). He contrasts the analysis of Boltzmann, based on the occurrence of numerous collisions, which may occur in very short time intervals, with Gibbs’s, which considers systems without interactions and may be applied only to sufficiently long time intervals *t* − *t* _{0} for two representative points in phase space, close at time *t* _{0}, to be very distant at time *t*. For Hadamard there is no doubt about the fact that the objection based on the reversibility of the equations of motion is not tenable, i.e. it cannot be used to prove the impossibility of systematic irreversible laws, which on the contrary seem to be plausible, as shown by Hadamard’s analogy based on card shuffling. Hadamard himself underlines the sensitivity of the solution to a change in the initial data.

According to Hadamard, Gibbs does not exhibit rigorous proofs of the existence of irreversible processes, since a satisfactory definition of randomness of a system is lacking. One may think that the “organized” distributions are the exceptional ones in the sense of Poincaré’s recurrence theorem. The conclusion is the following:

If

His a certain function of the distribution of the systemsS, the following property:The quantity H is increasingcan be transformed into a statement of the following kind (in which I no longer see handles for the reversibility objection): LetH_{1}andH_{2}<H_{1}be two values ofH, Ta suitably chosen time interval. Let us denote byM_{1}the motions for which, in the intervalT, the quantityHtakes at least once the valueH_{1}; byM_{2}the motions such that (during the same time interval) the quantityHtakes at least once the valueH_{2}; byM_{3}those that satisfy both of the previous conditions; in other words that are at the same timeM_{1}andM_{2}motions.The M_{3}’S are exceptional among the M_{1}’s, but not among the M_{2}’s. I have no doubt, as far as I am concerned, about a statement of this kind after the deductions of Gibbs and Boltzmann; were their conclusion wrong, the error could only concern the expression of the quantityH.

Another great French mathematician, Henri Poincaré, gave his opinions more than once on the kinetic theory of gases, but as we saw in Chapter 5, more with scepticism than with adherence to Boltzmann’s and Gibbs’s ideas. Concerning Poincaré and Boltzmann, it is ironic that they did not understand each other; they are probably the two scientists who most shaped our ideas about the behaviour of complex classical systems. We have no direct evidence that they met, but they must have done so, since they were both at the meeting in St Louis in 1904 (see Chapter 1).

(p.146) We recall here the tribute, actually too hasty, by Poincaré before the French Academy of Sciences [29]:

Boltzmann, who died tragically, had been teaching for a long time in Vienna; he had become known especially for his researches on the kinetic theory of gases. If the world obeys the laws of mechanics that allow us to proceed both forward and backward in time, why does it constantly tend toward uniformity without any chance that one may bring it back? This was the problem to be solved which he had devoted himself to, and not without some success.

In an article by Poincaré published in the same year [30], we read:

The kinetic theory of gases still leaves several embarrassing points to be clarified for those who are used to mathematical rigour; several results which have not been made sufficiently precise show up in a paradoxical form and seem to engender contradictions which, on the other hand, are only apparent. Thus the notions of

molar geordnet(ordered at a molar level) ormolekular geordnet(ordered at a molecular level) do not seem, in my opinion, to have been defined with sufficient precision. One of the points that were most embarrassing to me was the following: we must prove that entropy decreases [present author’s remark: the sign chosen is the same as for Boltzmann’sH-function]; but Gibbs’s arguments seem to assume that after letting the external conditions vary, one waits for the system to go back to [a steady] regime before letting them vary again. This assumption is essential, or one could arrive at results at variance with Carnot’s principle by letting external conditions vary too quickly for the steady regime to prevail. I wanted to clarify this question…

Poincaré continues with his analysis and arrives at the point of discussing a coarse-grained and a fine-grained entropy. The latter is greater than the former, which is the quantity considered by physicists and decreases with time, whereas the other remains constant.

# 7.6 The problem of trend to equilibrium and ergodic theory

It seems clear that Poincaré’s analysis inspired the Ehrenfests in their criticism of Gibbs, who never clarified the difference between the two entropies in a neat way. Although this criticism is relevant, one cannot ignore the fact that Gibbs himself was aware of the difficulties, as shown by the following sentences concerning the motion of a liquid containing colouring matter [2]:

Now the state in which the density of the coloring matter is uniform, i.e. the state of perfect mixture, which is a sort of state of equilibrium in this respect that the distribution of the coloring matter in space is not affected by the internal motions of the liquid, is characterized by a minimum value of the average square of the density of the coloring matter. Let us suppose, however, that the coloring matter is distributed with a variable density. If we give the liquid any motion whatever, subject only to the hydrodynamic law of incompressibility,—it may be a steady flux, or it may vary with time,—the density of the coloring matter at any same point of the liquid will he unchanged, and the average square of this density will therefore be unchanged. Yet no fact is more familiar to us than that stirring tends to bring a liquid to a state of uniform mixture, or uniform densities of the components, which is characterized by minimum values of the average squares of these densities. It is quite true that in the physical experiment the result is hastened by the process of diffusion, but the result is evidently not dependent on that process.

(p.147)The contradiction is to be traced to the notion of the

densityof the coloring matter, and the process by which this quantity is evaluated. This quantity is the limiting ratio of the quantity of the coloring matter in an element of space to the volume of that element. Now if we should take for our elements of volume, after any amount of stirring, the spaces occupied by the same portions of the liquid which originally occupied any given system of elements of volume, the densities of the coloring matter, thus estimated, would be identical to the original densities as determined by the given system of elements of volume. Moreover, if at the end of any finite amount of stirring we should take our elements of volume in any ordinary form but sufficiently small, the average square of the density of the coloring matter, as determined by such element of volume, would approximate to any required degree to its value before the stirring. But if we take any element of space of fixed position and dimensions, we may continue the stirring so long that the densities of the colored liquid, estimated for these fixed elements, will approach a uniform limit, viz., that of a perfect mixture.The case is evidently one of those, in which the limit of a limit has different values, according to the order in which we apply the processes of taking the limit.

What does all this lengthy argument mean? Essentially this: if we subdivide phase space (or, better, a hypersurface of constant energy; see Appendix 7.1) in small cells of volume Δ and indicate by (*t*) the average of *P* over one of these cells (a function of the index *i*, identifying the cell, and of time *t*), then (*t*) tends to a constant (independent of *i*) when *t* → ∞. The statement is plausible, at least for systems of points which do not exhibit attractive forces, but far from simple to prove. In mathematical terminology we would say that there is a weak convergence to a uniform state.

In fact the result stated above appears to be even stronger than ergodicity, as it is understood today. Here seems to be a good moment to recall the history and the role played by this concept in the development of statistical mechanics.

We have already stated that there is a hint at a property which must be satisfied if the statistical approach to mechanics holds. In particular, since a strict equilibrium is impossible, the analogue of thermodynamic equilibrium is a sort of motion which does not favour any region of phase space, or at least of the hypersurface of constant energy.

The Ehrenfests’ article [24] leaves much to be desired from a historical viewpoint. And this is true not only for the role played by Boltzmann in establishing the equilibrium distributions, but also because of certain wrong attributions. Thus S. Brush notices [31] that the authors attribute the expression “continuity of path” to Maxwell, whereas it was first used by J.H. Jeans in 1902 [32], 23 years after Maxwell’s death.

Thus it is better to look at what was actually written by the founding fathers, Maxwell and Boltzmann. The latter first published a detailed discussion of ergodic systems in 1871 [33]. He treats the motion of a point mass in a plane under the action of an attractive force which results from the sum of a force –*ax* along the *x*-axis and a second force –*by* along the *y*-axis. The resulting trajectory will be closed if the ratio between the two constants *a* and *b* is rational; in that case one obtains the so-called Lissajous figures well known from elementary textbooks in physics. When the aforementioned ratio is not rational, then exact recurrence may occur for only particular initial data; in general, the motion in the *xy*-plane will tend to fill a rectangle (*die ganze Fläche*), the size of which is fixed by the initial data (see Fig. 7.1). Similarly in the case of an attractive central force, for values of the total energy which allow only bounded motions (Boltzmann
(p.148)

*ar*

^{−1}+

*br*

^{−2}, for which an exact solution can be obtained), the point mass traverses the entire surface of an annular region (see Fig. 7.2) and the trajectory never becomes closed, in general (notable exceptions are, apart from special initial data, a Newtonian force and a force proportional to the distance from the attracting point). Since Boltzmann studied these simple problems as examples for the property that was required to establish equilibrium statistical mechanics, it is of interest to see what he meant. Did he really mean that the trajectory goes through

*every*point in the areas mentioned, or that it covers the area in such a way as to approach as close as one likes to every point? That is the question. Boltzmann then stated that the same property that is in question should hold for a system of

*n*points. Later in the paper he treats the thermal equilibrium of gas molecules on the following assumption [33]:

The great irregularity of the thermal motion, and the variety of forces that act on the body from outside, make it probable that the atoms, thanks to the motion that we call heat, pass through all possible positions and velocities compatible with energy conservation, and that we can accordingly apply the equations previously developed to the coordinates and velocities of the atoms of warm bodies.

The fact that Boltzmann mentions forces acting from outside seems to suggest that he knew that the trajectory was not going through *every* point, but this did not matter if it went as close as we like, since we cannot account for the multiplicity of small external forces.

Perhaps the strongest statement in favour of the ergodic hypothesis, as usually stated, can be read in Maxwell [34]: (p.149)

The only assumption which is necessary for the direct proof of the expression for the distribution function in equilibrium is that the system, if left to itself in its actual state of motion, will, sooner or later, pass through every phase which is consistent with the equation of energy.

Now it is manifest that there are cases in which this does not take place. The motion of the system not acted on by external forces possesses six equations besides the equation of energy, so that the system cannot pass through those phases which, though they satisfy the equation of energy, do not satisfy these six equations.

Again, there may be particular laws of force, as for instance that according to which the stress between two particles is proportional to the distance between them, for which the whole motion repeats itself after a finite time. In such cases a particular value of one variable corresponds to a particular value of each of the other variables, so that phases formed by sets of values of the variables which do not correspond cannot occur, though they may satisfy the several general equations.

But if we suppose that the material particles, or sore of them, occasionally encounter a fixed obstacle such as the sides of the vessel containing the particles, then, except for special forms of the surface of this obstacle, each encounter will introduce a disturbance into the motion of the system, so that it will pass from one undisturbed path to another. The two paths must both satisfy equations (p.150) of energy, and they must intersect each other in the phase for which the conditions of the encounter with the fixed obstacle are satisfied, but they are not subject to the equations of momentum. It is difficult in a case of such extreme complexity to arrive at a thoroughly satisfactory conclusion, but we may with considerable confidence assert that except for particular forms of the surface of the fixed obstacle, the system will sooner or later, after a sufficient number of encounters, pass through every phase consistent with the equation of energy.

This statement seems inescapable, because the boundary is not introducing any uncontrolled outside influence. The term *every phase* is not identical with *every point*, but using this as the basis of an argument would be really fussy. Thus Maxwell actually stated something similar to the ergodic hypothesis, while Boltzmann was not always very precise (another example is given by Brush [31]), just because he thought that a quasi-ergodic and an ergodic hypothesis, though very different mathematically, were practically equivalent.

We remark that the lengthy and extremely well-documented treatment of the ergodic hypothesis by Brush [31] mentions in passing that Boltzmann had discovered the method of ensembles before Gibbs, but he devotes no discussion to this extremely important point.

To conclude our discussion on the ergodic hypothesis, we must recall that the theory of infinite sets had not been developed in detail in those days. But all it needed was to allow time to pass. Thus it is not surprising, if we call *ergodic systems* those for which the phase space trajectory passes through *every* point, to read the remark of Borel in his supplement to the French translation to the Ehrenfests’ article [35]:

It is sufficient to be acquainted with the modern theory of sets to have the certainty that even the definition of ergodic systems is contradictory. […] this contradiction has been explicitly shown […] by A. Rosenthal and M. Plancherel […]; to tell the truth, it does not appear that this abstract hypothesis was ever really considered by physicists; …

Today, since the work of G.D. Birkhoff [36] and J. von Neumann [37], the term ergodic is equivalent to “metrically undecomposable”. This means that a dynamical system has a phase space which does not possess any non-zero measure subsets that are invariant for the motion. In other words, the set of points which cannot be joined by a trajectory has zero measure. It seems likely that many systems of physical interest possess this property, but actual proofs are scant.

# 7.7 Planck and statistical mechanics

As we have already remarked, Gibbs’s treatise was to obscure for most scientists Boltzmann’s and Maxwell’s kinetic theory, and to develop a tendency to forget the by then old lesson of Boltzmann on the Second Law, albeit that Gibbs himself had at length attracted the attention of his readers to the aleatory nature of his thermodynamic analogies. It is thus interesting to examine what Max Planck was writing in those years. He was a recent convert, if in 1897 he was still writing in the introduction to his treatise on thermodynamics [38] that “obstacles, at present unsurmountable” stood on the way of further progress of kinetic theory, and referred to the “essential difficulties […] in the mechanical interpretation of the fundamental principles of Thermodynamics”.

(p.151)
Planck’s conversion was due to the fact that Boltzmann’s techniques had played an essential role in the basic idea of the theory of light *quanta* proposed by Planck himself in 1900 (see Chapter 12).

The occasion for taking a position in favour of Boltzmann was offered by the celebration of the latter’s sixtieth birthday. Planck [39] remarks that although there was a time, when temperature was the quantity of most interest, since it is directly measurable, whereas entropy seemed to be a complicated concept, the situation had been subverted: one must give a mechanical explanation for entropy, and the definition of temperature will follow. The reason for this inversion of trend is very clear to Planck: temperature is a concept essentially related to the notion of thermal equilibrium, which acquires a meaning only in relation to irreversibility, being the final state toward which irreversible processes tend. Thus the notion of temperature necessarily leads to a study of irreversibility and hence of entropy, a primary, general concept that has a meaning for all kinds of state and hence for all state changes, whereas temperature makes its appearance only in a condition of thermal equilibrium, when entropy reaches its maximum value. He then goes on [39], praising Boltzmann but ignoring, like everybody else, the latter’s 1884 paper [1]:

It seems that Clausius and Maxwell have never tried to give a direct and general definition of entropy in mechanical terms. It was left to Boltzmann to accomplish this step, starting from the kinetic theory of gases and defining entropy in a general and univocal fashion as the logarithm of the probability of the mechanical state. In his papers on statistical mechanics, J.W. Gibbs proposes, besides Boltzmann’s definition, three new definitions founded upon the calculus of probabilities. The definitions by Gibbs aspire to a wider generality, because they do not imply any particular assumption on the mechanical system under consideration. They allow a successful extension, in principle, to systems that have, or have not, a large number of degrees of freedom and are made up of constituents of the same or different nature. To each of these definitions of entropy there corresponds a definition of temperature through d

Q=TdS. For systems with a large number of degrees of freedom, the three definitions always lead, as shown by Gibbs, to the same result, so that, e.g., for a system formed by very many molecules, there is but one definition, which agrees with the thermodynamic one.

Planck then proceeds to examining the case of equilibrium in a monatomic gas and remarks that both Boltzmann’s and Gibbs’s definitions are in agreement with thermodynamics:

Whereas Boltzmann defines entropy by the logarithm of probability, entropy, according to the first definition by Gibbs (concerning canonical ensembles), is, with a minus sign, the average value of the logarithm of probability. In the course of irreversible processes, the averaged logarithm of probability

decreasesaccording to Gibbs, whereas according to Boltzmann the logarithm of probabilityincreases. But this contradiction is just apparent, and disappears when one takes into account the fact that the two authors indicate entirely different concepts by the wordprobability. This requires a more detailed consideration. Boltzmann obtains the expression of probability by distinguishing betweeen a given state of the system and a “complexion” of the system itself. A state is given by the law of reparation of the positions and velocities, i.e. by the number of particles located in each element of the volume in phase space, by assuming that in each of the elements, considered to be equal, there is always a great number of particles. Hence a given state corresponds to many complexions. In fact when any two particles, belonging to two different (p.152) elements, interchange their coordinates, one obtains a new complexion, but the state remains the same. If then, with Boltzmann, we consider all the complexions as equally probable, the number of the complexions corresponding to a fixed state gives the probability and hence the entropy of the system in the state under consideration (apart from a constant).On the contrary, with Gibbs, the question of the uniformity of the particles does not play a more important role for the determination of entropy than for the distribution of positions and velocities. What is being taken into account here is rather the number of the complexions obtained when one considers all the possible values of the coordinates and of velocities within the assigned conditions: the values of the coordinates must be inside the volume ν under consideration, whereas the velocities may be constrained by specific conditions.

Although, concerning the calculation of entropy by Boltzmann, Planck refers to the lectures on *Gastheorie*, he devotes more space to the definition given by Gibbs and shows that his three definitions coincide when the number of molecules is large. Then he considers the general case when the particles are not identical, remarking that Boltzmann’s definitions provide formulae in agreement with thermodynamics, which Gibbs obtains only by introducing the grand canonical ensemble. He also remarks that Boltzmann’s treatment can be extended, at variance with Gibbs’s, to non-equilibrium states.