Entropy

Entropy

Entropy measures disorder

Entropy is a probabilistic property

Entropy is additive

Entropy is not conserved

Resources

Entropy is a physical quantity that is primarily a measure of the thermodynamic disorder of a physical system. Entropy has the unique property in that its global value must always increase or stay the same.

This property is reflected in the second law of thermodynamics. The factthat entropy must always increase in natural processes introduces the concept of irreversibility, and defines a unique direction for the flow of time.

Entropy is a property of all physical systems, the behavior of which is described by the second law of thermodynamics (the study of heat). The first law of thermodynamics states that the total energy of an isolated system is constant; the second law states that the entropy of an isolated system must stay the same or increase. Note that entropy, unlike energy, is not conserved but can increase. A system’s entropy can also decrease, but only if it is part of a larger system whose overall entropy does increase.

French mathematician and engineer Comte Lazare Nicolas Marguerite Carnot (1753–1823) wrote indirectly about entropy when he said that the motions and movements of machines represent losses of “moment of activity,” or losses of useful energy to the system. Increases in entropy, what is now known as heat, was called caloric by French physicist, mathematician, and engineer Nicolas Leonard Sadi Carnot (who was Lazare Carnot’s son). The word entropy, first articulated in 1850 by German physicist Rudolf Clausius (1822–1888), does not correspond to any property of matter that scientists can sense, such as temperature, and so it is not easy to conceptualize. It can be roughly equated with the amount of energy in a system that is not available for work or, alternatively, with the orderliness of a system, but is not precisely given by either of these concepts. A basic intuitive grasp of what entropy means can be given by statistical mechanics, as described below. Still later, American mathematical physicist Josiah Willard Gibbs (1839– 1903), Italian mathematician and physicist Ludwig Boltzmann (1844–1906), and Scottish physicist James Clerk Maxwell (1831–1879), among others, help to define the statistics behind the subject of entropy.

On a fundamental level, entropy is related to the number of possible physical states of a system, S = k log (γ), where S represents the entropy, k is Boltzmann’s constant, and γ is the number of states of the system.

Consider a system of three independent atoms that are capable of storing energy in quanta or units of fixed size є. If there happens to be only three units of energy in this system, how many possible micro-states—that is, distinct ways of distributing the energy among the atoms—are there? This question is most easily answered, for this example, by listing all the possibilities. There are 10 possible configurations.

If n ₀ stands for the number of atoms in the system with 0є energy, then n ₁ stands for the number with є, n ₂ for the number with 2є, and n ₃ for the number with 3є. For example, in the microstates labeled 1, 2, and 3 in the figure that accompanies this article, (n ₀, n ₁, n ₂, n ₃ ) =(2, 0, 0, 3); that is, two atoms have 0є energy, no atoms have 1є or 2є, and one atom has 3є.

Each class or group of microstates corresponds to a distinct (n ₀, n ₁, n ₂, n ₃ ) distribution. There are three possible distributions, and where P stands for the number of microstates corresponding to each distribution, P can equal 3, 6, or 1. The three values of P can be verified by counting the microstates that themselves reflect the energy distributions for a system of three atoms sharing three units of energy. Again, the number of possible microstates P corresponding to each distribution.

The distribution P2 —representing the distribution (n ₀, n ₁, n ₂, n ₃ ) = (1, 1, 1, 0)—has the most possible microstates (six). If one assume that this system is constantly, randomly shifting from one microstate to another, that any microstate is equally likely to follow any other, and that one inspect the system at some randomly-chosen instant, then one is most likely to observe one of the microstates corresponding to distribution P2. Specifically, the probability of observing a microstate corresponding to distribution P2 is 0.6 (6 chances out of 10). The probability of observing distribution P1 is 0.3 (3 chances out of 10) and the probability of observing distribution P3 is 0.1 (1 chance out of 10).

The entropy of this or any other system, S, is defined as S = k ln(P _max ), where P _max is the number of microstates corresponding to the most probable distribution of the system (P _max = 6 in this example), k is the Boltzmann constant (1.3803←× 10^-16 ergs per degree C [Celsius]), and ln(.) is the natural logarithm operation. Further inspection of this equation and the three-atom example given above will clarify some of the basic properties of entropy.

Entropy measures disorder

(1) Microstates 1, 2, and 3 of the three-atom system described above—those distributions in which the energy available to the system is segregated entirely to one atom—are in some sense clearly the most orderly or organized. Yet, these three microstates (distribution 1) are also unlikely; their total probability of occurrence at any moment is only half that of distribution 2. Order is less likely than disorder.

Entropy is a probabilistic property

(2) Any system might transition, at any time, to one of its less probable states, because energy can, in a sense, go anywhere it wants to; it is simply much less likely to go some places than others. In systems containing trillions of atoms or molecules, such as a roomful of air, the probability that the system will transition to a highly organized state analogous to microstate 1 in the Figure—say, that all the energy in the room will concentrate itself in one molecule while all the rest cool to absolute zero—is extremely small. One would have to wait many trillions of years for such an event even to happen.

Entropy is additive

If two systems with entropy S ₁ and S _2, respectively, are brought into contact with each other, their combined entropy equals S ₁ + S _2. This result is assured by the logarithmic relationship between S and P _max, as follows: If the number of microstates corresponding to the most likely distribution of the first system is P _max1 and that of the second system is P _max2, then the number of possible microstates of the most likely distribution of the combined system is simply P _max1 P _max2 . It is a fundamental properties of logarithms that ln(ab ) = ln(a) + ln(b), so if S ₁₊₂ entropy of the combined system, then:

S₁₊₂ = k 1n(P_max1P_max2)

=k [1n(P_max1) + 1n(P_max2)]

=k 1n(P_max1) + k 1n(P_max2)

=S₁ + S₂

Entropy is not conserved

All that is needed to increase the entropy of an isolated system is to increase the number of micro-states its particles can occupy; say, by allowing the system to occupy more space. It is beyond the scope of this discussion to prove that the entropy of a closed system cannot ever decrease, but this can be made plausible by considering the first law of thermodynamics, which forbids energy to be created or destroyed. As long as a system has the same number of atoms and the same number of quanta of energy to share between them, it is plausible that the system possesses a minimum number of possible microstates—and a minimum entropy.

It is sometimes claimed that entropy always increases, and that the second law requires that disorder must always increase when nature is left to its own devices. This is incorrect. Note that in the above example, a system of three independent atoms is stipulated; yet atoms rarely behave independently when in proximity to each other at low temperatures. They tend to form bonds, spontaneously organizing themselves into orderly structures (molecules and crystals). Order from disorder is, therefore, just as natural a process as disorder from order. At low temperatures, self-ordering predominates; at high temperatures, entropy effects dominate (i.e., order is broken down). Furthermore, any system that is not isolated can experience decreased entropy (increasing order) at the expense of increasing entropy elsewhere. The Earth, which shares the solar system with the sun, whose entropy is increasing rapidly, is one such non-isolated system. It is therefore an error to claim, as some people do, that biological evolution—which involves spontaneously increasing order in natural molecular systems—contradicts thermodynamics. Entropy does not forbid molecular self-organization because entropy is only one property of matter; entropy does discourage self-organization, but other properties of matter encourage it, and in some circumstances (especially at relatively low temperatures, as on the surface of the Earth) will prevail.

An alternative derivation of the entropy concept, based on the properties of heat engines (devices that turn heat flows partially into mechanical work) is often presented in textbooks. This method produces a definition of entropy that seems to differ from S = k ln(P _max ), namely, d S =d Q _rev/ T, where d S is an infinitesimal (very small) change in a system’s entropy at the fixed temperature T when an infinitesimal quantity of heat, Q _rev, flows reversibly into or out of the system. However, it can be shown that these definitions are exactly equivalent; indeed, the entropy concept was originally developed from the analysis of heat engines, and the statistical interpretation given above was not invented until later.

The entropy concept is fundamental to all science that deals with heat, efficiency, the energy of systems, chemical reactions, very low temperatures, and related topics. Its physical meaning is, in essence, that the amount of work the universe can perform is always declining as its orderliness declines, and must eventually approach zero. In other words, things are running down, and there is no way to stop them.

Resources

BOOKS

Greven, Andreas, et al., eds. Entropy. Princeton, NJ: Princeton University Press, 2003.

Jost, Jurgen. Dynamical Systems: Examples of Complex Behavior. Berlin, Germany, and New York: Springer, 2005.

Kirwan, A.D. Mother Nature’s Two Laws: Ringmasters for Circus Earth (Lessons on Entropy, Energy, Critical Thinking, and the Practice of Science). Singapore and River Edge, NJ: World Scientific, 2002.

Lee, Joon Chang Lee. Thermal Physics: Entropy and Free Energies. River Edge, NJ: World Scientific Publishing Co., Inc., 2002.

K. Lee Lerner

Larry Gilman