D. A. Lavis, the Concept of Probability in Statistical Mechanics

Recall the formula called Maxwell law for a gas of $N$ small, hard, perfectly elastic spheres: the number of spheres whose speed lies between $v$ and $v+dv$ is $$ f_1(v) dv = \frac{4N}{\alpha^2 \sqrt{\pi}}v^2 \exp\big(-\frac{v^2}{\alpha^2}\big). $$ The quantity is used to establish the $H$-function in the $H$-theorem: $$ H(t) = \int d^3 v f_1(v,t)\ln [f_1(v,t)]. $$ Dividing out the factor $N$, it is easy to see that $H(t)$ is nothing but the negative "entropy" in the velocity space, and the function $f_1(v,t)$ is equal to $N\rho_1(v,t)$ where $\rho_1(v,t)$ is the probability distribution of velocity over the velocity space.

Originally the quantity $f_1(v,t)$ was considered as the actual distribution of the $N$ particles of the gas. To see this, recall how it was derived. It was assumed that the $x,y,z$ velocities are independent from each other, thus $$ f(v_x,v_y,v_z) dv_x dv_y dv_z = f(v_x)dv_x f(v_y) dv_y f(v_z) dv_z $$ at the same time the distribution of velocities should be isotropic, $$ f(v_x,v_y,v_z) = f(v^2) = f(v_x^2 + v_y^2 +v_z^2) = f(v_x)f(v_y)f(v_z), $$ so that $$ \ln f(v_x^2 + v_y^2 + v_z^2) = \ln f(v_x) + \ln f(v_y) + \ln f(v_z). $$ A possible solution will be $$ f(v_i) = C e^{-B v_i^2}. $$ the constants are then determined by

1. the normalization of the velocity distribution,
2. the formula for the everage energy $\overline{\epsilon} = 3kT/2 = m\overline{v}^2/2$, where $\overline{v}^2$ is computed by

$$ \overline{v}^2 = \int_{-\infty}^{+\infty} v^2 \cdot f(v)d^3 v = 4\pi \int_0^\infty v^2 \cdot f(v) v^2 dv. $$

A technical problem for the kinetic theory of gases was reconciling the

1. reversibility of mechanical laws and the irreversibility of natural proceses as described by the second law of thermodynamics,
2. incompatibility of the second law of thermodynamics and the mechanical theory heat which is absed on a usually recurrent dynamical system (Poincare's recuurence theorem).

Since there was no probability involved and the kinetic theory was thought to be trying to describe what thermodynamic phenomena actually are, these two were actual problems.

The defence given by Boltzmann were some points that have become standard arguments in textbook nowadays:

1. The number of moclues is large for macro physical systems, so the recurrence time would be extremely large.
2. In practice a finite system is never completely isolated, so the recurrence doesn't apply.
3. One needs a clear definition, for the dynamical system, of what is meant by a macrostate.
4. The second law is a statistical law from the molecular point of view.

The immediate effect of the last point is the change of perception of the meaning of the quantity $f_1(v,t)$. From $\rho_1(v,t) = f_1(v,t)/N$ to $\rho_(v,p,t)$, the probability density function on the phase space $\Gamma$ is relatively a small step. Now probability is always accompanied by debates around its interpretations. In statistical mechanics, the presence of statistics is considered, usually, in three light: Ensemble, Ergodic, Probablistic.

While the maximum entropy method is primarily that of non-equilibrium statistical mechanics, it has a form specifically for equilibrium.

The motto is that probability distribution should model the best prediction one can make of observable phenomena, absed on the information available. The fact that there are unknown constants of motion is irrelevant since making predictions is based on available information.

Conssider a system with discrete energy levels $\{E_i\}_i$. Then one asks the following question

while the standard objectivist formulation would be

Now, given an appropriate measure of uncertainty, if one choose the probability distribution that maximizes the uncertainty relative to the available information, then this will be the best probability distribution, since it assumes as little as possible. He shows that the unique measure of uncertainty satisfying some reasonable mathematical properties is Shannon's entropy $$ S_I(p_i) = -\sum_i p_i \ln (p_i). $$ Subject to the condition $\sum_i p_i = 1$, maximizing $S_I(p_i)$ would give the same result as that hypothesized by the objectivists the uniform distribution. The objectivist, on the other hand, suppose that there is a probability associated with an experiment to determine its state. Popperian will hypothesize that $p_i=1/n$; sequence of experiments will be used to see if the hypothesis is falsified. Frequentists will define the probability being $p_i=1/n$.

For general cases, the maximum entropy method for equilibrium is identical to what is described here (Information-theoretic interpretation of thermodynamics).