Bayes, Thomas

BIBLIOGRAPHY

Thomas Bayes (1702–1761) was the eldest son of the Reverend Joshua Bayes, one of the first nonconformist ministers to be publicly ordained in England. The younger Bayes spent the last thirty years of his comfortable, celibate life as Presbyterian minister of the meeting house, Mount Sion, in the fashionable town of Tunbridge Wells, Kent. Little is known about his personal history, and there is no record that he communicated with the well-known scientists of his day. Circumstantial evidence suggests that he was educated in literature, languages, and science at Coward’s dissenting academy in London (Holland 1962). He was elected a fellow of the Royal Society in 1742, presumably on the basis of two metaphysical tracts he published (one of them anonymously) in 1731 and 1736 (Barnard 1958). The only mathematical work from his pen consists of two articles published posthumously in 1764 by his friend Richard Price, one of the pioneers of social security (Ogborn 1962). The first is a short note, written in the form of an undated letter, on the divergence of the Stirling (de Moivre) series ln(z!). It has been suggested that Bayes’ remark that the use of “a proper number of the first terms of the … series” will produce an accurate result constitutes the first recognition of the asymptotic behavior of a series expansion (see Deming’s remarks in Bayes [1764] 1963). The second article is the famous “An Essay Towards Solving a Problem in the Doctrine of Chances,” with Price’s preface, footnotes, and appendix (followed, a year later, by a continuation and further development of some of Bayes’ results).

The “Problem” posed in the Essay is: “Given the number of times in which an unknown event has happened and failed: Required the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named.” A few sentences later Bayes writes: “By chance I mean the same as probability” ([1764] 1963, p. 376).

If the number of successful happenings of the event is p and the failures q, and if the two named “degrees” of probability are b and f, respectively, Proposition 9 of the Essay provides the following answer expressed in terms of areas under the curve x^p(1 - x)^q:

This is based on the assumption (Bayes’ “Postulate 1”) that all values of the unknown probability are equally likely before the observations are made. Bayes indicated the applicability of this postulate in his famous “Scholium”: “that the … rule is the proper one to be used in the case of an event concerning the probability of which we absolutely know nothing antecedently to any trials made concerning it, seems to appear from the following consideration; viz. that concerning such an event I have no reason to think that, in a certain number of trials, it should rather happen any one possible number of times than another” (ibid., pp. 392–393).

The remainder of Bayes’ Essay and the supplement (half of which was written by Price) consists of attempts to evaluate (1) numerically, (a) by expansion of the integrand and (b) by integration by parts. The results are satisfactory for p and q small but the approximations for large p, q are only of historical interest (Wishart 1927).

Opinions about the intellectual and mathematical ability evidenced by the letter and the essay are extraordinarily diverse. Netto (1908), after outlining Bayes’ geometrical proof, agreed with Laplace ([1812] 1820) that it is ein wenig verwickelt (“somewhat involved”). Todhunter (1865) thought that the résumé of probability theory that precedes Proposition 9 was “excessively obscure.” Molina (in Bayes [1764] 1963, p. xi) said that “Bayes and Price … can hardly be classed with the great mathematicians that immediately preceded or followed them,” and Hogben (1957, p. 133) stated that “the ideas commonly identified with the name of Bayes are largely [Laplace’s].”

On the other hand von Wright (1951, p. 292) found Bayes’ Essay “a masterpiece of mathematical elegance and free from … obscure philosophical pretentions.” Barnard (1958, p. 295) wrote that Bayes’ “mathematical work … is of the very highest quality.” Fisher ([1956] 1959, p, 8) concurred with these views when he said Bayes’ “mathematical contributions … show him to have been in the first rank of independent thinkers. …”

The subsequent history of mathematicians’ and philosophers’ extensions and criticisms of Proposition 9—the only statement that can properly be called Bayes’ theorem (or rule)—is entertaining and instructive. In his first published article on probability theory, Laplace (1774), without mentioning Bayes, introduced the principle that if P_j is the probability of an observable event resulting from “cause” j (j = 1, 2, 3, …, n) then the probability that “cause” j is operative to produce the observed event is

This is Principle III of the first (1812) edition of Laplace’s probability text, and it implies that the prior (antecedent, initial) probabilities of each of the “causes” is the same. However, in the second (1814) edition Laplace added a few lines saying that if the “causes” are not equally probable a priori (2) would become

where ?_j is the prior probability of cause j and p_j is now the probability of the event, given that “cause” j is operative. He gave no illustrations of this more general formula.

Laplace (1774) applied his new principle (2) to find the probability of drawing m white and n black tickets in a specified order from an urn containing an infinite number of white and black tickets in an unknown ratio and from which p white and q black tickets have already been drawn. His solution, namely,

was later (1778–1781; 1812, chapter 6) generalized by the bare statement that if all values of x are not equally probable a factor z(x) representing the a priori probability density (facilité) of x must appear in both integrands. However, Laplace’s own views on the applicability of expressions like (4) were stated in 1778 (1778–1781, p. 264) and agree with those of Bayes’ Scholium: “Lorsqu’on n’a aucune donnée a priori sur la possibilité d’un événement, il faut supposer toutes les possibilityés, depuiszéro jusqu’àl’unité, également probables. …” (“When nothing is given a priori as to the prob-ability of an event, one must suppose all probabilities, from zero to one, to be equally likely. …”) Much later Karl Pearson (1924, p. 191) pointed out that Bayes was “considering excess of one variate … over a second … as the determining factor of occurrence” and this led naturally to a generalization of the measure in the integrals of (1). Fisher (1956) has even suggested that Bayes himself had this possibility in mind.

Laplace’s views about prior probability distributions found qualified acceptance on the Continent (von Kries 1886) but were subjected to strong criticism in England (Boole 1854; Venn 1866; Chrystal 1891; Fisher 1922), where a relative frequency definition of probability was proposed and found incompatible with the uniform prior distribution (for example, E. S. Pearson 1925). However, developments in the theory of inference (Keynes 1921; Ramsey 1923–1928; Jeffreys 1931; de Finetti 1937; Savage 1954; Good 1965) suggest that there are advantages to be gained from a “subjective” or a “logical” definition of probability and this approach gives Bayes’ theorem, in its more general form, a central place in inductive procedures (Jeffreys 1939; Raiffa & Schlaifer 1961; Lindley 1965).

Hilary L. Seal

[For the historical context of Eayes’ work, seeStatistics, article onthe history of statistical method; and the biography ofLaplace. For discussion of the subsequent development of his ideas, seeBayesian inference; Probability; and the biographies ofFisher, R. A.; andPearson.]

BIBLIOGRAPHY

Barnard, G. A. 1958 Thomas Bayes: A Biographical Note. Biometrika 45:293–295.

Bayes, Thomas (1764) 1963 Facsimiles of Two Papers by Bayes. New York: Hafner. ? Contains “An Essay Towards Solving a Problem in the Doctrine of Chances, With Richard Price’s Foreword and Discussion,” with a commentary by Edward C. Molina; and “A Letter on Asymptotic Series From Bayes to John Canton,” with a commentary by W. Edwards Deming. Both essays first appeared in Volume 53 of the Philosophical Transactions, Royal Society of London, and retain the original pagination.

Boole, George (1854) 1951 An Investigation of the Laws of Thought, on Which Are Founded the Mathematical Theories of Logic and Probabilities. New York: Dover.

Chrystal, George 1891 On Some Fundamental Principles in the Theory of Probability. Actuarial Society of Edinburgh, Transactions 2:419–439.

de Finetti, Bruno 1937 La prévision: Ses lois logiques, ses sources subjectives. Paris, Université de, Institut Henri Poincaré, Annales 7:1–68.

Fisher, R. A. (1922) 1950 On the Mathematical Foundations of Theoretical Statistics. Pages 10.307a–10.368 in R. A. Fisher, Contributions to Mathematical Statistics. New York: Wiley. ? First published in Volume 222 of the Philosophical Transactions, Series A, Royal Society of London.

Fisher, R. A. (1956) 1959 Statistical Methods and Scientific Inference. 2d ed., rev. New York: Hafner; London: Oliver & Boyd.

Good, Irving J. 1965 The Estimation of Probabilities: An Essay on Modern Bayesian Methods. Cambridge, Mass.: M.I.T. Press.

Hogben, Lancelot T. 1957 Statistical Theory; the Relationship of Probability, Credibility and Error: An Examination of the Contemporary Crisis in Statistical Theory From a Behaviourist Viewpoint. London: Allen & Unwin.

Holland, J. D. 1962 The Reverend Thomas Bayes, F.R.S. (1702–1761). Journal of the Royal Statistical Society Series A 125:451–461.

Jeffreys, Harold (1931) 1957 Scientific Inference. 2d ed. Cambridge Univ. Press.

Jeffreys, Harold (1939) 1961 Theory of Probability. 3d ed. Oxford: Clarendon.

Keynes, J. M. (1921) 1952 A Treatise on Probability. London: Macmillan. ? A paperback edition was published in 1962 by Harper.

Kries, Johannes von (1886) 1927 Die Principien der Wahrscheinlichkeitsrechnung: Eine logische Untersuchung. 2d ed. Tübingen (Germany): Mohr.

Laplace, Pierre S. (1774) 1891 Mémoire sur la probabilité des causes par les événements. Volume 8, pages 27–65 in Pierre S. Laplace, Oeuvres complétes de Laplace. Paris: Gauthier-Villars.

Laplace, Pierre S. (1778–1781) 1893 Mémoire sur les probabilités. Volume 9, pages 383–485 in Pierre S. Laplace, Oeuvres complètes de Laplace. Paris: Gauthier-Villars.

Laplace, Pierre S. (1812) 1820 Théorie analytique des probabilités. 3d ed., rev. Paris: Courcier.

Lindley, Dennis V. 1965 Introduction to Probability and Statistics From a Bayesian Viewpoint. 2 vols. Cambridge Univ. Press.

Netto, E. 1908 Kombinatorik, Wahrscheinlichkeitsrechnung, Reihen-Imaginäres. Volume 4, pages 199–318, in Moritz Cantor (editor), Vorlesungen über Geschichte der Mathematik. Leipzig: Teubner.

Ogborn, Maurice E. 1962 Equitable Assurances: The Story of Life Assurance in the Experience of The Equitable Life Assurance Society, 1762–1962. London: Allen & Unwin.

Pearson, Egon S. 1925 Bayes’ Theorem, Examined in the Light of Experimental Sampling. Biometrika 17: 388–442.

Pearson, Karl 1924 Note on Bayes’ Theorem. Biometrika 16:190–193.

Price, Richard 1765 A Demonstration of the Second Rule in the Essay Towards a Solution of a Problem in the Doctrine of Chances. Royal Society of London, Philosophical Transactions 54:296–325. ? Reprinted by Johnson in 1965.

Raiffa, Howard; and Schlaifer, Robert 1961 Applied Statistical Decision Theory. Harvard University Graduate School of Business Administration, Studies in Managerial Economics. Boston: The School.

Ramsey, Frank P. (1923–1928) 1950 The Foundations of Mathematics and Other Logical Essays. New York: Humanities.

Savage, Leonard J. 1954 The Foundations of Statistics. New York: Wiley.

Todhunter, Isaac (1865) 1949 A History of the Mathematical Theory of Probability From the Time of Pascal to That of Laplace. New York: Chelsea.

Venn, John (1866) 1888 The Logic of Chance: An Essay on the Foundations and Province of the Theory of Probability, With Special Reference to Its Logical Bearings and Its Application to Moral and Social Science. 3d ed. London: Macmillan.

Wishart, John 1927 On the Approximate Quadrature of Certain Skew Curves, With an Account of the Re-searches of Thomas Bayes. Biometrika 19:1–38.

Wright, Georg H. von 1951 A Treatise on Induction and Probability. London: Routledge.