Entropy and Temperature

Eq. 2.1 introduced the quantity $\Omega(N,V,E)$ as the number of states available to a system under the constraints of constant number of particles, $N$, volume, $V$, and energy $E$. The fundamental postulate of statistical mechanics, also called the ``rational basis'' by Chandler, is the following:

In statistical equilibrium, all states consistent with the constraints of $N$, $V$, and $E$ are equally probable.

or

$$P_\nu = 1/\Omega(N,V,E).$$ (9)

This relation is often referred to as a statement of the ``equal a priori probabilities in state space.'' Another way of saying the same thing: The probability distribution for states in the microcanonical ensemble is uniform.

The link between statistical mechanics and classical thermodynamics is given by a definition of entropy:

$$S \equiv k_B\ln\Omega$$ (10)

Note two important properties of $S$. First, it is extensive: if we consider a compound system made of subsystems $A$ and $B$ with $\Omega_A$ and $\Omega_B$ as the respective number of states, the total number of states is $\Omega_A\Omega_B$, and therefore $S = S_A + S_B$. Second, it is consistent with the second law of thermodynamics: putting any constraint on the system lowers its entropy because the constraint lowers the number of accessible states.

Temperature is defined using entropy: $1/T \equiv \left(\partial S/\partial E\right)_{N,V}$, or

$$\beta = \left(k_BT\right)^{-1} = \left(\partial\ln\Omega/\partial E\right)$$ (11)

Now we will consider constraining our system not with constant $E$, but with constant $T$. The set of all possible states satisfying constraints of $N$, $V$, and $T$ is called the canonical ensemble. Contrary to appearances based on their names, the canonical ensemble can be envisioned as a subsystem in a larger, microcanonical system. Consider such a system divided into a small subsystem $A$, surrounded by a large ``bath'' $B$. We imagine that these two subsystems are ``weakly coupled,'' meaning they exchange only thermal energy, but no particles, and their volumes remain fixed. We seek to compute the probability of finding the total system in a state such that subsystem $A$ has energy $E_A$. The entire system is microcanonical, so the total energy, $E$ is constant, as is the total number of states available to the system, $\Omega (E)$ (we omit the $N$ and $V$ for simplicity).

When $A$ has energy $E_A$, the total system energy is $E = E_A + E_B$, where $E_B$ is the energy of the bath. By constraining system $A$'s energy, we have reduced the number of states available to the whole system to $\Omega(E-E_A)$. So, the probability of observing the system in a state in which subsystem $A$ has energy $E_A$ looks like

$$P_A = \frac{\left[\begin{array}{l} \mbox{Number of states f... ...ht]} = \frac{\Omega\left(E - E_A\right)}{\Omega\left(E\right)}$$ (12)

We can expand $\Omega\left(E - E_A\right)$ in a Taylor series around $E_A = 0$:

$$P_A \propto \Omega\left(E - E_A\right) = \exp\left[\ln\Omega\left(E-E_A\right)\right]$$ (13)

$$= \exp\left[\ln\Omega\left(E\right) - E_A\frac{\partial\ln\Omega}{\partial E} + \cdots\right],$$ (14)

where the partial derivative implies we are holding $N$ and $V$ fixed. We can truncate the Taylor expansion at the first-order term, because higher order terms become less and less important as the size of subsystem $B$ becomes larger and larger. What results is the Boltzmann distribution law for energies of a system at constant temperature:

$$P_A \propto \exp\left(-\beta E_A\right)$$ (15)

The normalization condition requires that for all energies of subsystem $A$, $E_A$,

$$\sum_A P_A = 1 = \sum_A \exp\left(-\beta E_A\right) \equiv Q\left(N,V,T\right),$$ (16)

which defines the canonical partition function, $Q$. Therefore,

$$P_A = Q^{-1}\exp\left(-\beta E_A\right)$$ (17)

Because some energies can correspond to more than one microstate, we should distinguish between ``states'' and ``energy levels.'' We can express the canonical partition function as

$$Q = \sum_{\begin{array}{cc}\nu\ \mbox{states}\end{array}} e... ...mbox{levels}\end{array}} \Omega\left(E_l\right) e^{-\beta E_l}$$ (18)

where, as we have seen, $\Omega\left(E\right)$ is the number of microstates with energy $E$. Moving to the continuum limit, and assuming a reference energy of $E_{ref} = 0$,

$$Q \rightarrow \int_0^\infty dE \overline\Omega\left(E\right)e^{-\beta E}$$ (19)

where $\overline\Omega\left(E\right)$ is the density of states. What is this equation telling us? It is telling us that $Q$ is the Laplace transform of $\overline\Omega$. We know that transform pairs are unique, and hence, both $Q$ and $\overline\Omega$ contain the same information.

We recognize that for a system described by a canonical ensemble, the energy is a fluctuating quantity. And we now have the probability of observing a state with a given energy, so we can use Eq. 1 to compute the average energy, $\left<E\right>$. Consider

$$\left<E\right> = \left<E_\nu\right> = \sum_\nu P_\nu E_\nu$$ (20)

$$= \left[\sum_\nu E_\nu \exp\left(-\beta E_\nu\right)\right] \left/ \left[\sum_\nu \exp\left(-\beta E_\nu\right)\right] \right.$$ (21)

Notice that

$$\sum_\nu E_\nu \exp\left(-\beta E_\nu\right) = -\left(\partial Q \left/ \partial\beta\right.\right)$$ (22)

Recalling that $d\ln f(x) / dx = 1/f df/dx$, we see that

$$\left<E\right> = -\left(\partial Q \left/ \partial\beta\right.\right)\left/Q\right.$$ (23)

$$= -\left(\partial \ln Q \left/ \partial\beta\right.\right)_{N,V}$$ (24)

Now, let us consider the average magnitude of the fluctuations in energy in the canonical ensemble.

$$\left<\left(\delta E\right)^2\right> = \left<\left(E - \left<E\right>\right)^2\right>$$ (25)

$$= \left<E^2\right> - \left<E\right>^2$$ (26)

$$= \sum_\nu P_\nu E_\nu^2 - \left(\sum_\nu P_\nu E_\nu\right)^2$$ (27)

$$= Q^{-1}\left(\frac{\partial^2Q}{\partial\beta^2}\right)_{N,V} - Q^{-2}\left(\frac{\partial Q}{\partial\beta}\right)^2_{N,V}$$ (28)

$$= \left(\frac{\partial\ln Q}{\partial\beta^2}\right)_{N,V} = -\left(\frac{\partial\left<E\right>}{\partial\beta}\right)_{N,V}$$ (29)

Now, noting that the definition of heat capacity at constant volume, $C_v$, is

$$C_v = \left(\frac{\partial E}{\partial T}\right)$$ (30)

we see that

$$\left<\left(\delta E\right)^2\right> = k_B T^2 C_v$$ (31)

This is an interesting statement. It relates the magnitude of spontaneous fluctuations in the total energy of a system to that system's capacity to store or release energy due to changing its temperature.

The fact (Eq. 24) that the average energy in the canonical ensemble is related to a derivative of the log of the partition function implies that $\ln Q$ is an important thermodynamic quantity. So, let's go back to our undergraduate thermodynamics course(s) and recall the following statement of the 1st and 2nd Law:

$$dA = -SdT - pdV + \mu dN$$ (32)

where $A$ is the Helmholtz free energy, defined in terms of internal energy and entropy as

$$A = \left<E\right> - TS$$ (33)

Now, consider the following derivative of $A$:

$$\left(\frac{\partial\left(A/T\right)}{\partial\left(1/T\right)}\right)_{N,V} = A + \frac{1}{T}\left(\frac{\partial A}{\partial\left(1/T\right)}\right)_{N,V}$$ (34)

$$= A - T\left(\frac{\partial A}{\partial T}\right)_{N,V} = A - TS = \left<E\right>.$$ (35)

Therefore,

$$\left(\frac{\partial\left(\beta A\right)}{\partial\beta}\right)_{N,V} = \left<E\right>.$$ (36)

Considering Eq. 24, we see that

$$\ln Q + C = -\beta A$$ (37)

which does indeed suggest an important link between $\ln Q$ and the important thermodynamic quantity, the Helmholtz free energy. But what is the constant $C$? To evaluate it, consider the ``boundary condition'' as $T \rightarrow 0$:

$$Q = \sum_\nu e^{-\beta E_\nu} \stackrel{T\rightarrow 0}{\longrightarrow} e^{-\beta E_{\rm ground}}.$$ (38)

Here, we have assumed that the degeneracy of the ground state, $\Omega\left(E_{\rm ground}\right)$ is 1. This tells us that

$$\lim_{T\rightarrow 0} \ln Q = -\beta E_{\rm ground}$$ (39)

Using this fact, and combining Eqs. 33 and 37, as $T \rightarrow 0$, we see that

$$-\beta E_{\rm ground} + C = -\beta \underbrace{\left<E\right... ...}} - \underbrace{\frac{S}{k_B}}_{\rightarrow 0 (\Omega = 1)},$$ (40)

Hence, $C$ = 0. So,

$$\ln Q = -\beta A.$$ (41)

The quantity $-\beta^{-1}\ln Q$ is the Helmholtz free energy, $A$. This is denoted $F$ in Frenkel & Smit [1].