Bayes, Thomas

views updated

Bayes, Thomas

(b. London, England, 1702; d. Tunbridge Wells, England, 17 April 1761)

probability.

Bayes was a member of the first secure generation of English religious Nonconformists. His father, Joshua Bayes, F.R.S., was a respected theologian of dissent; he was also one of the group of six ministers who were the first to be publicly ordained as Nonconformists. Privately educated, Bayes became his father’s assistant at the presbytery in Holborn, London; his mature life was spent as minister at the chapel in Tunbridge Wells. Despite his provincial circumstances, he was a wealthy bachelor with many friends. The Royal Society of London elected him a fellow in 1742. He wrote little: Divine Benevolence (1731) and Introduction to the Doctrine of Fluxions (1736) are the only works known to have been published during his lifetime. The latter is a response to Bishop Berkeley’s Analyst, a stinging attack on the logical foundations of Newton’s calculus; Bayes’s reply was perhaps the soundest retort to Berkeley then available.

Bayes is remembered for his brief “Essay Towards Solving a Problem in the Doctrine of Chances” (1763), the first attempt to establish foundations for statistical inference. Jacques Bernoulli’s Ars Conjectandi (1713) and Abraham de Moivre’s The Doctrine of chances (1718) already provided great textbooks of what we now call probability theory. Given the probability of one event, the logical principles for inferring the probabilities of related events were quite well understood. In his “Essay,” Bayes set himself the “converse problem”: “Given the number of times in which an unknown event has happened and failed: Required the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named.” “By chance,” he said, “I mean the same as probability.”

In the light of Bernoulli’s Ars conjectandi, and of a paper by John Arbuthnot (1710), there was some understanding of how to reject statistical hypotheses in the light of data; but no one had shown how to measure the probability of statistical hypotheses in the light of data. Bayes began his solution of the problem by noting that sometimes the probability of a statistical hypothesis is given before any particular events are observed; he then showed how to compute the probability of the hypothesis after some observations are made. In his own example:

Postulate: 1. I suppose the square table or plane ABCD to be so made and levelled that if either of the balls O or W be thrown upon it, there shall be the same probability that it rests upon any one equal part of the plane as another, and that it must necessarily rest somewhere upon it.

2. I suppose that the ball W shall be first thrown, and through the point where it rests a line os shall be drawn parallel to AD, and meeting CD and AB in s and o, and that afterwards the ball O shall be thrown p + q or n times, and that its resting between AD and os after a single throw be called the happening of the event M in a single trial.

For any fractions f and b (between zero and one), Bayes was concerned with the probability of assertions of the form

f ? probability of M ? b.

From his physical assumptions about the table ABCD, he inferred that the prior probability (i.e., the probability before any trials have been made) is b — f. He proved, as a theorem in direct probabilities, that the posterior probability (i.e., the probability, on the evidence, that M occurred p times and failed q times) is

A generalization on this deduction is often, anachronistically, called Bayes’s theorem or Bayes’s formula. In the case of only finitely many statistical hypotheses, H1,..., Hn, let there be prior data D and some new observed evidence E. Then the prior probability of Hi is Prob (Hi/D); the posterior probability is Prob (Hi/D and E); the theorem asserts:

Prob (Hi/D and E)

A corresponding theorem holds in the continuum; Bayes’s deduction essentially involves a special case of it.

The work so far described falls entirely within probability theory, and would now be regarded as a straightforward deduction from standard probability axioms. The striking feature of Bayes’s work is an argument found in a scholium to the paper, which does not follow from any standard axioms. Suppose we have no information about the prior probability of a statistical hypothesis. Bayes argues by analogy that, in this case, our ignorance is neither more nor less than in his example where prior probabilities are known to be entirely uniform. He concludes, “I shall take for granted that the rule given concerning the event M... is also the rule to be used in relation to any event concerning the probability of which nothing at all is known antecedently to any trials made or observed concerning it.”

If Bayes’s conclusion is correct, we have a basis for the whole of statistical inference. Richard Price, who sent Bayes’s paper to the Royal Society, seems to imply in a covering letter that Bayes was not satisfied with his argument by analogy and, hence, had declined to publish it. Whatever the case with Bayes, Laplace had no qualms about Bayes’s argument; and from 1774 he regularly assumed uniform prior probability distributions. His enormous influence made Bayes’s ideas almost unchallengeable until George Boole protested in his Laws of Thought (1854). Since then, Bayes’s technique has been a constant subject of controversy.

Today there are two kinds of Bayesians. Sir Harold Jeffreys, in his Theory of Probability, maintains that, relative to any body of information, even virtual ignorance, there is an objective distribution of degrees of confidence appropriate to various hypotheses; he often rejects Bayes’s actual postulate, but accepts the need for similar postulates. Leonard J. Savage, in his Foundations of Statistics, rejects objective probabilities, but interprets probability in a personal way, as reflecting a person’s personal degree of belief; hence, a prior probability is a person’s belief before he has made some observations, and his posterior probability is his belief after the observations are made. Many working statisticians who are Bayesians, in the sense of trying to argue from prior probabilities, try to be neutral between Jeffreys and Savage. In this respect they are perhaps close to Bayes himself. He defined the probability of an event as “the ratio between the value at which an expectation depending upon the happening of the event ought to be computed, and the value of the thing expected upon its happening.” This definition can be interpreted in either a subjective or an objective way, but there is no evidence that Bayes had even reflected on which interpretation he might prefer.

BIBLIOGRAPHY

I. Original Works. Bayes’s works published during his lifetime are Divine Benevolence, or an Attempt to Prove That the Principal End of the Divine Providence and Government Is the Happiness of His Creatures (London, 1731); and An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of The Analyst (London, 1736). “An Essay Towards Solving a Problem in the Doctrine of Chances” was published in Philosophical Transactions of the Royal Society of London, 53 (1763), 370–418, with a covering letter written by Richard Price; repr. in Biometrika, 45 (1958), 296–315, with a biographical note by G. A. Barnard. Also of interest is “A Letter on Asymptotic Series from Bayes to John Canton,” in Biometrika, 45 (1958), 269–271; repr. with the paper from the Philosophical Transactions in Facsimiles of Two Papers by Bayes (Washington, D.C., n.d.), with a commentary on the first by Edward C. Molina and on the second by W. Edwards Deming.

II. Secondary Literature. Supplementary information may be found in John Arbuthnot, “An Argument for Divine Providence Taken From the Constant Regularity of the Births of Both Sexes,” in Philosophical Transactions of the Royal Society of London, 23 (1710), 186–190; George Berkeley, The Analyst (London-Dublin, 1734); Jacques Bernoulli, Ars conjectandi (Basel, 1713); George Boole, An Investigation of the Laws of Thought (London, 1854); Harold Jeffreys, Theory of Probability (Oxford, 1939; 3rd ed., 1961); Pierre Simon Laplace, “Mémoire sur la probabilité des causes par les événements,” in Mémoires par divers savants, 6 (1774), 621–656, and Théorie analytique des probabilités (Paris, 1812); Abraham de Moivre, The Doctrine of Changes (London, 1718; 3rd ed., 1756); and Leonard J. Savage, The Foundations of Statistics (New York-London, 1954).

Ian Hacking

More From encyclopedia.com