Statistical Methods And Thermodynamics Chem 472: Lecture

The coefficients pk determine the probability of observing the eigenvalue ok when the system is in state ψ . The expectation value of ô is ψ ô ψ pk ok ,(5)ki.e., the average of expectation values associated with states φk .The expectation value of any arbitrary operator Â, which does not share a common set of eigenstateswith ô, can be computed in the basis set of eigenstates of ô as follows,p ψ  ψ pk φk  φk pk p j ei(θ j θk ) φk  φj .(6)k j6 kkNote that such an expectation value is not only determined by the average of expectation values associatedwith states k (i.e., the first term in the r.h.s of Eq. (6)), but also by the second term in that equation. Suchsecond term is responsible for interferences, or coherences, between states φk and φj as determinedby the phases θk and θ j .Consider a large number of N replicas of the system, all of them described by the same state vector ψ .Note that such collection of N replica systems is also described by a pure state. Therefore, the ensembleaverages associated with the observables ô and  of such a pure state will coincide with the expectationvalues given by the equations Eq. (5) and Eq. (6), respectively.4Statistical Mixture of StatesThe collection of a large number N of independently prepared replicas of the system is called an ensemble.An ensemble of N replicas of systems is in a statistical mixture of states φk , with probabilities pk , whennk members of the ensemble are in state φk , with pk nk /N. Note that each member of the ensembleis in a specific state φk , not in a coherent superposition of states as described by Eq. (2). Therefore, theensemble averages associated with the observables ô and  are pk φk  φk ,(7) pk φk ô φk pk ok ,(8)A kando kkrespectively. Note that the ensemble average o, introduced by Eq. (8), coincides with the ensemble averageof the pure state described by Eq.(5). However, the ensemble average A, introduced by Eq. (7), does notcoincide with the corresponding ensemble average of the pure state, introduced by Eq. (6). As a matter offact, it coincides only with the first term of Eq. (6) since the second term of the r.h.s. of Eq. (6) is missing inEq. (7). Therefore, in a statistical mixture there are no contributions to the ensemble average coming frominterferences between different states (e.g., interferences between states ψk i and ψj i).The statistical mixture introduced in this section, is also equivalent to an ensemble of N replicas of thesystem in incoherent superposition of states represented as follows, ψ(ξ )i pk eiθk (ξ ) φk ,(9)kwhere the phases θk (ξ ) are distributed among the different members ξ of the ensemble according to auniform and random distribution.In the remaining of this section we introduce the most important types of ensembles by consideringsystems with only one species of molecules. Additional details for multicomponent systems are consideredlater.9

In the canonical ensemble all of the replica systems are in thermal equilibrium with a heat reservoirwhose temperature is T. This ensemble is useful for comparisons of the ensemble averages with measurements on systems with specified number of particles N, volume V and temperature T. It is central to MonteCarlo simulations, an important approximation method of Statistical Mechanics.In the microcanonical ensemble all of the replica systems have the same energy E and number of particles N. This ensemble is no very simply applicable to comparisons with systems we usually study inthe laboratory, since those are in thermal equilibrium with their surroundings. However, the microcanonical ensemble is centrally involved in Molecular Dynamics simulations which is one of the most importantapproximation methods of Statistical Mechanics.In the grand canonical ensemble all of the replica systems are in thermal equilibrium with a heat reservoir whose temperature is T and they are also in equilibrium with respect to exchange of particles witha “particle” reservoir where the temperature is T and the chemical potential of the particles is µ. Thisensemble is useful for comparisons to measurements on systems with specified µ, T and V.Exercise 1: Compute the ensemble average Ā associated with the incoherent superposition of states introduced by Eq. (9) and verify that such an average coincides with Eq. (7).5Density OperatorIn this section we show that ensemble averages for both pure and mixed states can be computed as follows,where ρ̂ is the density operatorρ̂ A Tr {ρ̂ Â},(10) pk φk φk .(11)kNote that, in particular, the density operator of an ensemble where all of the replica systems are describedby the same state vector ψ (i.e., a pure state) isρ̂ ψ ψ .(12)Eq. (10) can be proved first for a pure state ψ k ak φk , where φk i constitute a complete basis setof orthonormal states (i.e., φk0 φk δkk0 ), by computing the Tr{ρ̂ Â} in such representation as follows,A 0 φk0 ψ ψ  φk0 .(13)kSubstituting the expansion of ψi into Eq. (13) we obtain,A 0 φk0 φk ak a j φj  φk0 ,kj(14)kand since φk0 φk δkk0 ,A pk φk  φk ppk p j ei(θk θ j ) φj  φk ,(15)k j6 kkwhere we have substituted the expansion coefficients a j in accord with Eq. (4). Equation (15) is identical toEq. (6) and, therefore, Eq. (10) is identical to Eq. (6) which defines an ensemble average for a pure state.Eq. (10) can also be proved for an arbitrary mixed state defined by the density operator introduced by Eq.(11), by computing the Tr{ρ̂ Â} as follows,A 0 pk φk0 φk φk  φk0 pk φk  φk ,kkk10(16)

which is identical to Eq. (7).Exercise 2:(A) Show that Tr{ρ̂} 1 for both mixed and pure states.(B) Show that Tr{ρ̂2 } 1 for pure states.(C) Show that Tr{ρ̂2 } 1 for mixed states.Note that the Tr{ρ̂2 } is, therefore, a measurement of decoherence (i.e., lost of interference between the various different states in the ensemble). When the system is in a coherent superposition state, such as the onedescribed by Eq. (2), Tr{ρ̂2 } 1. However, Tr{ρ̂2 } 1 when the system is in an incoherent superposition ofstates such as the one described by Eq. (9).6Time-Evolution of EnsemblesThe evolution of systems in both pure and mixed states can be described according to the following equation: ρ̂[ρ̂, Ĥ ] . tih̄(17)Exercise 3: Using the equation of motion for a state vector ψ (i.e., Eq. (1)), show that Eq. (17) describesthe time evolution of ρ̂ for a pure state.Exercise 4: Using the linearity of Eq. (1), show that Eq. (17) also describes the time evolution of ρ̂ for amixed state.7Classical AnalogueMicroscopic states: Quantum statistical mechanics defines a microscopic state of a system in Hilbert spaceaccording to a well defined set of quantum numbers. Classical statistical mechanics, however, describesthe microscopic state in phase space according to a well defined set of coordinates ( x1 , .x f ) and momenta( p1 , ., p f ).Ensembles: Quantum statistical mechanics describes an ensemble according to the density operator ρ̂,introduced by Eq. (11). Classical statistical mechanics, however, describes an ensemble according to thedensity of states ρ ρ( x1 , .x f , p1 , ., p f ).Time-Evolution of Ensembles: Quantum statistical mechanics describes the time-evolution of ensemblesaccording to Eq. (17), which can be regarded as the quantum mechanical analogue of the Liouville’s theoremof classical statistical mechanics, ρ (ρ, H ) ,(18) tEq. (18) is the equation of motion for the classical density of states ρ ρ( x1 , .x f , p1 , ., p f ). Thus thename density operator for ρ appearing in Eq. (17).[ G,F ]Note that the classical analog of the commutator ih̄ is the Poisson bracket of G and F,f( G, F ) G F G F x j p j p j x j .(19)j 1Exercise 5: Prove Eq. (18) by using the fact that the state of a classical system is defined by the coordinates( x1 , .x f ) and momenta ( p1 , ., p f ) which evolve in time according to Hamilton’s equations, i.e.,dp j H ,dt x jdx j H ,dt p j11(20)

fwhere H j 1 p2j /(2m j ) V ( x1 , .x f ) is the classical Hamiltonian.Ensemble Averages: Quantum statistical mechanics describes ensemble averages according to Eq. (10).Classical statistical mechanics, however, describes ensemble averages according to the classical analog ofEq. (10),RRdx dpρ( x1 , ., x f , p1 , ., p f ) ARĀ R,(21)dx dpρ( x1 , ., x f , p1 , ., p f )where dxdp stands for a volume element in phase space.8EntropyThe entropy S̄ of an ensemble can be defined in terms of the density operator ρ̂ as follows,S kTr {ρ̂ lnρ̂},(22)where k is the Botzmann constant. Equation (22) is the Von Neumann definition of entropy. This is the mostfundamental definition of S because it is given in terms of the density operator ρ̂, which provides the mostcomplete description of an ensemble. In particular, the Gibbs entropy formula,S k pk lnpk ,(23)kcan be obtained from Eq. (22) by substituting ρ̂ in accord with Eq. (11).From Eq. (23) one can see that the entropy of a pure state is zero, while the entropy of a statisticalmixture is always positive. Therefore,S 0,(24)which is the fourth law of Thermodynamics.8.1Exercise: Entropy ExtensivityShow that the definition of entropy, introduced by Eq. (23), fulfills the requirement of extensivity (i.e., whendividing the system into fragments A and B, the entropy of the system including both fragments S AB equalsthe sum of the entropies of the fragments S A and SB ).Solution: We consider that the fragments are independent so the joint probability p jA ,jB of configurationsj A and jB of fragments A and B is equal to the product of the probabilities p jA and p jB of the configurationsof each fragment. Therefore, S AB jA jB p jA ,jB ln( p jA ,jB ) jA jB p jA p jB ln( p jA p jB ), with S A jA p jA ln( p jA ) and SB jB p jB ln( p jB ). So, S AB S A SB , since ln( p jA p jB ) ln( p jA ) ln( p jB ).We can show that there is no other function but the logarithm that fulfills that condition, as follows.Consider a function that fulfills the following condition: f ( p jA p jB ) f ( p jA ) f ( p jB ) and compute thepartial derivative with respect to p jA , as follows: f ( p j A p jB ) f ( p jA )p jB . ( p j A p jB ) p jA(25)Analogously, wecompute the partial derivatives with respect to p jB , as follows: f ( p jB ) f ( p j A p jB )p jA . ( p j A p jB ) p jB12(26)

Therefore,p jAwhere c is a constant. f ( pj )Therefore, p j A A9cpjAandRdp jA f ( p jB ) f ( p jA ) p jB c, p jA p jB f ( pjA ) p j A cR(27)dp jA p1j , giving f ( p jA ) c ln( p jA ).AMaximum-Entropy Density OperatorThe goal of this section is to obtain the density operator ρ̂, with Tr {ρ̂} 1, that maximizes the entropyS kTr{ρ̂lnρ̂} of a system characterized by an ensemble average internal energyE Tr{ρ̂ Ĥ },(28)and fix extensive properties X such as X (V, N ) (i.e., canonical and microcanonical ensembles).This is accomplished by implementing the method of Lagrange Multipliers to maximize the functionf (ρ̂) kTr{ρ̂lnρ̂} γ( E Tr {ρ̂ Ĥ }) γ0 (1 Tr {ρ̂}),where γ and γ0 are Lagrange Multipliers. We, therefore, solve for ρ̂ from the following equation! f 0, ρ̂(29)(30)Xand we obtain that the density operator that satisfies Eq. (30) must satisfy the following equation:Tr { klnρ̂ k γ Ĥ γ0 } 0.Therefore, lnρ̂ 1 γγ0Ĥ .kk(31)(32)Exponentiating both sides of Eq. (32) we obtainρ̂ exp( (1 and, since Tr{ρ̂} 1,γ0γ))exp( Ĥ ),kk(33)γ01)) ,kZ(34)Z Tr {exp( β Ĥ )},(35)exp( (1 where Z is the partition functionwith β γ/k.Substituting Eqs. (35) and (34) into Eq. (33), we obtain that the density operator that maximizes theentropy of the ensemble, subject to the contraint of average ensemble energy Ē, isρ̂ Z 1 exp( β Ĥ ).(36)Note that ρ̂ 0, twhen ρ̂ is defined according to Eq. (36) and, therefore, the system is at equilibrium.Exercise 6: Use Eqs. (17) and (36) to prove Eq. (37).13(37)

10Internal Energy and Helmholtz Free EnergySubstituting Eqs. (35) and (34) into Eq. (28) we obtain that the internal energy E can be computed from thepartition function Z as follows,! lnZE .(38) βXFurthermore, substituting Eqs. (35) and (34) into Eq. (22) we obtainS kTr{ρ̂( β Ĥ lnZ )} kβE klnZ.(39)In the next section we prove that the parameter T (kβ) 1 can be identified with the temperature of theensemble. Therefore,A E TS kTlnZ,(40)is the Helmholtz free energy, that according to Eq. (38) satisfies the following thermodynamic equation,! ( βA)E .(41) βX11TemperatureThe parameter T 1/kβ γ1 has been defined so far as nothing but the inverse of the Lagrange Multiplierγ. Note that according to Eq. (39), however, T can be defined as follows:!1 S .(42)T ENThe goal of this section is to show that T can be identified with the temperature of the system because it isthe same through out the system whenever the system is at thermal equilibrium.Consider a system at equilibrium, with ensemble average internal energy E, in the state of maximum entropyat fixed N. Consider a distribution of S, T, and E in compartments (1) and (2) as specified by the followingdiagram:(1)(2)S (1) T (1) E (1)S (2) T (2) E (2)6N1N2Thermal (Heat) ConductorConsider a small displacement of heat δE from compartment (1) to compartment (2):δE(1) δE,andδE(2) δE.(43)Since the system was originally at the state of maximum entropy, such a displacement would produce achange of entropyδS) E,N 0,(44)14

whereδS δS(1) δS(2) S(1) E(1)!δE(1)N S(2) (2) E!δE(2) N 11 T1T2 δE 0.(45)Since the inequality introduced by Eq. (45) has to be valid for any positve or negative δE, then T1 T2 .12Minimum Energy PrincipleThe minimum energy principle is a consequence of the maximum entropy principle. This can be shown byconsidering the system at thermal equilibrium described by the following diagram:(1)(2)S ( E (1) , X )S ( E (2) , X )6N1N2Thermal (Heat) ConductorConsider a small displacement of heat δE from compartment (2) to compartment (1). Since the systemwas originally at equilibrium, such a contraint in the distribution of thermal energy produces a constrainedsystem whose entropy is smaller than the entropy of the system at equilibrium. Mathematically,S( E(1) δE, X) S( E(2) δE, X) S( E(1) , X) S( E(2) , X).(46)Now consider the system at equilibrium (i.e., without any constraints) with entropy S( E, X) such thatS( E, X) S( E(1) δE, X) S( E(2) δE, X).(47)Since, according to Eqs. (47) and (46),S( E, X) S( E(1) , X) S( E(2) , X),(48)and according to Eq. (42), S E! V,N1 0,T(49)thenE E (1) E (2) .(50)Eq. (47) thus establishes that by imposing internal constraints at constant entropy the system that wasinitially at equilibrium with entropy S( E, X) moves away from such equilibrium and its internal energyincreases from E to E(1) E(2) . Mathematically,! 0,dES,Vwhich is the minimum energy principle.15(51)

As an example, consider 2 balloons filled with nitrogen at room temperature. The balloons are in contactand at thermal equilibrium. At a lower temperature, the kinetic energy of molecules would be smaller,so the total energy of the system would be smaller. Therefore, the entropy would also be smaller since1/T dS/dE 0. Thus, reducing the energy of the system would be one way of reducing the entropy.Another way would be as follows. Take the 10 slower molecules of one balloon and exchange themby the 10 faster of the other. Then, separate the balloons so they are no longer touching each other. Oneof the balloons would now be warmer and the other colder because effectively a little bit of heat has beentransferred from one to the other. Since the balloons were originally at equilibrium, that transformationalso reduced the total entropy of the system of 2 balloons, although the total internal energy remained thesame. Now, if we bring them to equilibrium with each other keeping the entropy to remain the same, we seethat the internal energy would have to be reduced. Therefore, the equilibrium could be reached by energyminimization. Another way of reaching the equilibrium would be to keep the energy the same and increasethe entropy.13Canonical and Microcanonical EnsemblesExercise 7: (A) Use Eqs. (39) and (11) to show that in a canonical ensemble the probability p j of observingthe system in quantum state j , whereH j Ej j ,(52)is the Boltzmann probability distributionp j Z 1 exp( βEj ) exp( β( Ej A)),(53)where β (kT ) 1 , with T the temperature of the ensemble and k the Boltzmann constant.(B) Show that for a microcanonical ensemble, where all of the states j have the same energy Ej E, theprobability of observing the system in state j ispj 1,Ω(54)where Ω is the total number of states. Note that p j is, therefore, independent of the particular state j in amicrocanonical ensemble.Note that according to Eqs. (23) and (54), the entropy of a microcanonical ensemble corresponds to theBoltzmann definition of entropy,S klnΩ.(55)14Equivalency of EnsemblesA very important aspect of the description of systems in terms of ensemble averages is that the properties ofthe systems should be the same as described by one or another type of ensemble. The equivalence betweenthe description provided by the microcanonical and canonical ensembles can be demonstrated most elegantlyas follows. Consider the partition function

R1: "Introduction to Modern Statistical Mechanics" by David Chandler (Oxford University Press). Ch. 3-8. Additional textbooks are available at the Kline Science and Engineering library include: R2: "Introduction to Statistical Thermodynamics" by T.L. Hill (Addison Wesley), R3: "Statistical Mechanics" by D. McQuarrie (Harper & Row),