History of Random-ness

In ancient history, the concepts of chance and randomness were intertwined with that of fate. Many ancient peoples threw dice to determine fate, and this later evolved into games of chance. At the same time, most ancient cultures used various methods of divination to attempt to circumvent randomness and fate.[1] [2]

The Chinese were perhaps the earliest people to formalize odds and chance 3,000 years ago. The Greek philosophers discussed randomness at length, but only in non-quantitative forms. It was only in the sixteenth century that Italian mathematicians began to formalize the odds associated with various games of chance. The invention of modern calculus had a positive impact on the formal study of randomness. In the 19th century the concept of entropy was introduced in physics and a proof of the randomness of the digits of the number π was presented.

The early part of the twentieth century saw a rapid growth in the formal analysis of randomness, as various approaches for a mathematical foundation of probability were introduced. Probability theory was axiomatized in 1933 and in the mid to late 20th-century ideas of algorithmic information theory introduced new dimensions to the field via the concept of algorithmic randomness.

Although randomness had often been viewed as an obstacle and a nuisance for many centuries, in the twentieth century computer scientists began to realize that the deliberate introduction of randomness into computations can be an effective tool for designing better algorithms. In some cases such randomized algorithms outperform the best deterministic methods. {| class="toc" id="toc"

Contents
[hide]*1 Antiquity to the Middle Ages
 * 2 17th–19th centuries
 * 3 20th century
 * 4 Notes
 * }

[edit] Antiquity to the Middle Ages
Depiction of Roman Goddess Fortuna who determined fate, by Hans Beham, 1541In ancient history, the concepts of chance and randomness were intertwined with that of fate. Pre-Christian people along the Mediterranean threw dice to determine fate, and this later evolved into games of chance.[3] There is also evidence of games of chance played by ancient Egyptians, Hindus and Chinese, dating back to 2100 BC.[4] The Chinese used dice before the Europeans, and have a long history of playing games of chance.[5]

Over 3,000 years ago, the problems concerned with the tossing of several coins were considered in the I Ching, one of the oldest Chinese mathematical texts, that probably dates to 1150 BC. The two principal elements yin and yang were combined in the I Ching in various forms to produce Heads and Tails permutations of the type HH, TH, HT, etc. and the Chinese seem to have been aware of Pascal's triangle long before the Europeans formalized it in the 17th century.[6] However, Western philosophy focused on the non-mathematical aspects of chance and randomness until the 16th century.

The development of the concept of chance throughout history has been very gradual. Historians have wondered why progress in the field of randomness was so slow, given that humans have encountered chance since antiquity. Deborah Bennett suggests that ordinary people face an inherent difficulty in understanding randomness, although the concept is often taken as being obvious and self-evident. She cites studies by Kahneman and Tversky; these concluded that statistical principles are not learned from everyday experience because people do not attend to the detail necessary to gain such knowledge.[7]

The Greek philosophers were the earliest Western thinkers to address chance and randomness. Around 400 BC, Democritus presented a view of the world as governed by the unambiguous laws of order and considered randomness as a subjective concept that only originated from the inability of humans to understand the nature of events. He used the example of two men who would send their servants to bring water at the same time to cause them to meet. The servants, unaware of the plan, would view the meeting as random.[8]

Aristotle saw chance and necessity as opposite forces. He argued that nature had rich and constant patterns that could not be the result of chance alone, but that these patterns never displayed the machine-like uniformity of necessary determinism. He viewed randomness as a genuine and widespread part of the world, but as subordinate to necessity and order.[9] Aristotle classified events into three types: certain events that happen necessarily; probable events that happen in most cases; and unknowable events that happen by pure chance. He considered the outcome of games of chance as unknowable.[10]

Around 300 BC Epicurus proposed the concept that randomness exists by itself, independent of human knowledge. He believed that in the atomic world, atoms would swerve at random along their paths, bringing about randomness at higher levels.[11] Hotei, the deity of fortune observing a cock fight in a 16th-century Japanese printFor several centuries thereafter, the idea of chance continued to be intertwined with fate. Divination was practiced in many cultures, using diverse methods. The Chinese analyzed the cracks in turtle shells, while the Germans, who according to Tacitus had the highest regards for lots and omens, utilized strips of bark.[12] In the Roman Empire chance was personified by the Goddess Fortuna. The Romans would partake in games of chance to simulate what Fortuna would have decided. In 49 BC, Julius Caesar allegedly decided on his fateful decision to cross the Rubicon after throwing dice.[13]

Aristotle's classification of events into the three classes: certain, probable and unknowable was adopted by Roman philosophers, but they had to reconcile it with deterministic Christian teachings in which even events unknowable to man were considered to be predetermined by God. About 960 Bishop Wibold of Cambrai correctly enumerated the 56 different outcomes (without permutations) of playing with three dice. No reference to playing cards has been found in Europe before 1350. The Church preached against card playing, and card games spread much more slowly than games based on dice.[14] The Christian Church specifically forbade divination and wherever Christianity went divination lost most of its old-time power.[15] [16]

Over the centuries, many Christian scholars wrestled with the conflict between the belief in free will and its implied randomness, and the idea that God knows everything that happens. Saints Augustine and Aquinas tried to reach an accommodation between foreknowledge and free will, but Martin Luther argued against randomness and took the position that God's omniscience renders human actions unavoidable and determined.<sup class="reference" id="cite_ref-16">[17] In the 13th century, Thomas Aquinas viewed randomness not as the result of a single cause, but of several causes coming together by chance. While he believed in the existence of randomness, he rejected it as an explanation of the end-directedness of nature, for he saw too many patterns in nature to have been obtained by chance.<sup class="reference" id="cite_ref-17">[18]

The Greeks and Romans had not noticed the magnitudes of the relative frequencies of the games of chance. For centuries chance was discussed in Europe with no mathematical foundation and it was only in the 16th century that Italian Mathematicians began to discuss the outcomes of games of chance as ratios.<sup class="reference" id="cite_ref-18">[19] <sup class="reference" id="cite_ref-19">[20] <sup class="reference" id="cite_ref-20">[21] In his 1565 Liber de Lude Aleae (a gambler's manual published after his death) Gerolamo Cardano wrote one of the first formal tracts to analyze the odds of winning at various games.<sup class="reference" id="cite_ref-21">[22]

[edit] 17th–19th centuries
Statue of Blaise Pascal, LouvreAround 1620 Galileo wrote a paper called On a discovery concerning dice that used an early probabilistic model to address specific questions.<sup class="reference" id="cite_ref-22">[23] In 1654, prompted by Chevalier de Méré's interest in gambling, Blaise Pascal corresponded with Pierre de Fermat, and much of the groundwork for probability theory was laid. Pascal's Wager was noted for its early use of the concept of infinity, and the first formal use of decision theory. The work of Pascal and Fermat influenced Leibniz's work on the infinitesimal calculus, which in turn provided further momentum for the formal analysis of probability and randomness.

The first known suggestion for viewing randomness in terms of complexity was made by Leibniz in an obscure 17th-century document discovered after his death. Leibniz asked how one could know if a set of points on a piece of paper were selected at random (e.g. by splattering ink) or not. Given that for any set of finite points there is always a mathematical equation that can describe the points, (e.g. by Lagrangian interpolation) the question focuses on the way the points are expressed mathematically. Leibniz viewed the points as random if the function describing them had to be extremely complex. Three centuries later, the same concept was formalized as algorithmic randomness by A. N. Kolmogorov and Gregory Chaitin as the minimal length of a computer program needed to describe a finite string as random.<sup class="reference" id="cite_ref-23">[24]

The Doctrine of Chances, the first textbook on probability theory was published in 1718 and the field continued to grow thereafter.<sup class="reference" id="cite_ref-24">[25] The frequency theory approach to probability was first developed by Robert Ellis and John Venn late in the 19th century. In the 1888 edition of his book The Logic of Chance, John Venn wrote a chapter on "The conception of randomness" and presented a proof of the randomness of the digits of the number Pi by using them to construct a random walk in two dimensions.<sup class="reference" id="cite_ref-25">[26] The Fortune Teller by Vouet, 1617While the mathematical elite was making progress in understanding randomness from the 17th to the 19th century, the public at large continued to rely on practices such as fortune telling in the hope of taming chance. Fortunes were told in a multitude of ways both in the Orient (where fortune-telling was later termed an addiction) and in Europe by gypsies and others.<sup class="reference" id="cite_ref-26">[27] <sup class="reference" id="cite_ref-27">[28] English practices such as the reading of eggs dropped in a glass were exported to Puritan communities in North America.<sup class="reference" id="cite_ref-28">[29]

The term entropy, which is now a key element in the study of randomness, was coined by Rudolf Clausius in 1865 as he studied heat engines in the context of the second law of thermodynamics. Clausius first stated that "entropy always increases".<sup class="reference" id="cite_ref-29">[30] Since the time of Newton to about 1890 it was generally believed that if one knows the initial state of a system with great accuracy, and if all the forces acting on the system can be formulated with equal accuracy, it would be possible, in principle, to make predictions of the state of the universe for an infinitely long time. The limits to such predictions in physical systems became clear as early as 1893 when Henri Poincaré showed that in the three-body problem in astronomy, small changes to the initial state could result in large changes in trajectories during the numerical integration of the equations.<sup class="reference" id="cite_ref-30">[31]

During the 19th century, as probability theory was formalized and better understood, the attitude towards "randomness as nuisance" began to be be questioned. Goethe wrote: The tissue of the world is built from necessities and randomness; the intellect of men places itself between both and can control them; it considers the necessity and the reason of its existence; it knows how randomness can be managed, controlled, and used. The words of Goethe proved prophetic, when in the 20th century randomized algorithms were discovered as powerful tools.<sup class="reference" id="cite_ref-31">[32] By the end of the 19th century Newton's model of a mechanical universe was fading away as the statistical view of the collision of molecules in gases was studied by Maxwell and Boltzmann.<sup class="reference" id="cite_ref-32">[33] Boltzmann's equation S = k loge W (inscribed on his tombstone) first related entropy with logarithms.

[edit] 20th century
Antony Gormley's Quantum Cloud sculpture in London was designed by a computer using a random walk algorithm.During the 20th century, the five main interpretations of probability theory, namely classical, logical, frequency, propensity and subjective became better understood, were discussed, compared and contrasted.<sup class="reference" id="cite_ref-33">[34] A significant number of application areas were developed in this century, from finance to physics. In 1900 Louis Bachelier applied Brownian motion to evaluate stock options, effectively launching the fields of financial mathematics and stochastic processes.

Émile Borel was one of the first mathematicians to formally address randomness in 1909, and introduced normal numbers.<sup class="reference" id="cite_ref-34">[35] In 1919 Richard von Mises gave the first definition of algorithmic randomness via the impossibility of a gambling system. He advanced the frequency theory of randomness in terms of what he called the collective, i.e. a random sequence. Von Mises regarded the randomness of a collective as an empirical law, established by experience. He related the "disorder" or randomness of a collective to the lack of success of attempted gambling systems. This approach led him to suggest a definition of randomness which was later refined and made mathematically rigorous by Alonso Church by using computable functions in 1940.<sup class="reference" id="cite_ref-35">[36] Richard von Mises likened the principle of the impossibility of a gambling system to the principle of the conservation of energy, a law that cannot be proven, but has held true in repeated experiments.<sup class="reference" id="cite_ref-36">[37]

Von Mises never totally formalized his rules for sub-sequence selection, but in his 1940 paper "On the concept of random sequence", Alonzo Church suggested that the functions used for place settings in the formalism of von Mises be recursive functions rather than arbitrary functions of the initial segments of the sequence, appealing to the Church–Turing thesis on effectiveness.<sup class="reference" id="cite_ref-37">[38] <sup class="reference" id="cite_ref-38">[39]

The advent of quantum mechanics in the early 20th century and the formulation of the Heisenberg uncertainty principle in 1927 saw the end to the Newtonian mindset among physicists regarding the determinacy of nature. In quantum mechanics, there is not even a way to consider all observable elements in a system as random variables at once since many observables do not commute.<sup class="reference" id="cite_ref-39">[40] Café Central, one of the early meeting places of the Vienna circleBy the early 1940s the frequency theory approach to probability was well accepted within the Vienna circle, but in the 1950s Karl Popper proposed the propensity theory.<sup class="reference" id="cite_ref-40">[41] <sup class="reference" id="cite_ref-41">[42] Given that the frequency approach can not deal with "a single toss" of a coin, and can only address large ensembles or collectives, the single-case probabilities were treated as propensities or chances. The concept of propensity was also driven by the desire to handle single-case probability settings in quantum mechanics, e.g. the probability of decay of a specific atom at a specific moment. In more general terms, the frequency approach can not deal with the probability of the death of a specific person given that the death can not be repeated multiple times for that person. Karl Popper echoed the same sentiment as Aristotle in viewing randomness as subordinate to order when he wrote that "the concept of chance is not opposed to the concept of law" in nature, provided one considers the laws of chance.<sup class="reference" id="cite_ref-42">[43] <sup class="reference" id="cite_ref-43">[44]

Claude Shannon's development of Information theory in 1948 gave rise to the entropy view of randomness. In this view, randomness is the opposite of determinism in a stochastic process. Hence if a stochastic system has entropy zero it has no randomness and any increase in entropy increases randomness. Shannon's formulation defaults to Boltzmann's entropy in case all probabilities are equal.<sup class="reference" id="cite_ref-44">[45] <sup class="reference" id="cite_ref-45">[46] Entropy is now widely used in diverse fields of science from thermodynamics to quantum chemistry.<sup class="reference" id="cite_ref-46">[47]

Martingales for the study of chance and betting strategies were introduced by Lévy in the 1930s and were formalized by Doob in the 1950s.<sup class="reference" id="cite_ref-47">[48] The application of random walk hypothesis in financial theory was first proposed by Maurice Kendall in 1953.<sup class="reference" id="cite_ref-48">[49] It was later promoted by Eugene Fama and Burton Malkiel. A. N. KolmogorovRandom strings were first studied by in the 1960s by A. N. Kolmogorov (who had provided the first axiomatic definition of probability theory in 1933),<sup class="reference" id="cite_ref-49">[50] Chaitin and Martin-Löf.<sup class="reference" id="cite_ref-50">[51] The algorithmic randomness of a string was defined as the minimum size of a program (e.g. in bits) executed on a universal computer that yields the string. Chaitin's Omega number later related randomness and the halting probability for programs.<sup class="reference" id="cite_ref-51">[52]

Despite mathematical advances, reliance on other methods of dealing with chance, such as fortune telling and astrology continued in the 20th century. The government of Myanmar reportedly shaped 20th century economic policy based on fortune telling and planned the move of the capital of the country based on the advice of astrologers.<sup class="reference" id="cite_ref-52">[53] <sup class="reference" id="cite_ref-53">[54] <sup class="reference" id="cite_ref-54">[55] White House Chief of Staff Donald Regan criticized the involvement of astrologer Joan Quigley in decisions made during Ronald Reagan's presidency in the 1980s.<sup class="reference" id="cite_ref-55">[56] <sup class="reference" id="cite_ref-56">[57] <sup class="reference" id="cite_ref-57">[58] Quigley claims to have been the White House astrologer for seven years.<sup class="reference" id="cite_ref-58">[59]

During the 20th century limits in dealing with randomness were better understood. The best-known example of both theoretical and operational limits on predictability is weather forecasting, simply because models have been used in the field since the 1950s. Predictions of weather and climate are necessarily uncertain. Observations of weather and climate are uncertain and incomplete, and the models into which the data are fed are uncertain.<sup class="reference" id="cite_ref-59">[60] In 1961, Edward Lorenz noticed that a very small change to the initial data submitted to a computer program for weather simulation could result in a completely different weather scenario. This later became known as the butterfly effect, often paraphrased as the question: "Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?".<sup class="reference" id="cite_ref-60">[61] A key example of serious practical limits on predictability is in geology, where the ability to predict earthquakes either on an individual or on a statistical basis remains a remote prospect.<sup class="reference" id="cite_ref-61">[62]

In the late 1970s and early 1980s computer scientists began to realize that the deliberate introduction of randomness into computations can be an effective tool for designing better algorithms. In some cases such randomized algorithms outperform the best deterministic methods.<sup class="reference" id="cite_ref-62">[63]

[edit] Notes

 * 1) ^ Handbook to Life in Ancient Rome, Lesley Adkins, 1998 ISBN 0195123328 p. 279
 * 2) ^ Religions of the Ancient World, Sarah Iles Johnston, 2004 ISBN 0674015177 p. 370
 * 3) ^ What is Random?: Chance and Order in Mathematics and Life, Edward J. Beltrami, 1999, Springer ISBN 0387987371 pp. 2-4
 * 4) ^ Encyclopedia of Leisure and Outdoor Recreation, John Michael Jenkins, 2004 ISBN 0415252261 p. 194
 * 5) ^ Audacious Angles of China, Elise Mccormick, 2007 ISBN 1406753327 p. 158
 * 6) ^ The Nature and Growth of Modern Mathematics, Edna Ernestine Kramer, 1983 ISBN p. 313
 * 7) ^ Randomness, Deborah J. Bennett, Harvard University Press, 1998. ISBN 0-674-10745-4 pp. 8-9 and 24
 * 8) ^ Design and Analysis of Randomized Algorithms, Juraj Hromkovič, 2005 ISBN 3540239499 p. 1
 * 9) ^ Aristotle's Physics: a Guided Study, Joe Sachs, 1995 ISBN 0813521920 p. 70
 * 10) ^ A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 p. 30
 * 11) ^ Epicurus: an Introduction, John M. Rist, 1972 ISBN 0521084261 p. 52
 * 12) ^ The Age of Chance, Gerda Reith, 2000 ISBN 0415179971 p. 15; Tac. Germ. 10
 * 13) ^ What is Random?: Chance and Order in Mathematics and Life, Edward J. Beltrami, 1999, Springer ISBN 0387987371 pp. 3-4
 * 14) ^ A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 pp. 29-36
 * 15) ^ A general history of the Christian church Volume 2 by Joseph Priestley 1804 ASIN B002KW4M6O page 11
 * 16) ^ Catholic encyclopedia [1]
 * 17) ^ The Case for Humanism, Lewis Vaughn, Austin Dacey, 2003 ISBN 0742513939 p. 81
 * 18) ^ The treatise on the divine nature: Summa theologiae I, 1-13, by Saint Thomas Aquinas, Brian J. Shanley, 2006 ISBN 0872208052 p. 198
 * 19) ^ A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 pp. 30-4
 * 20) ^ World of Scientific Discovery‎, Kimberley A. McGrath and Bridget Traverspage, 1999 ISBN 0787627607 p. 893
 * 21) ^ Randomness, Deborah J. Bennett, Harvard University Press, 1998. ISBN 0-674-10745-4 p. 8
 * 22) ^ A Dictionary of Scientists, John Daintith, Derek Gjertsen, 1999 ISBN 0192800868 p. 88
 * 23) ^ A History of Probability and Statistics and Their Applications before 1750, Anders Hald, 2003 ISBN 0471471291 p. 41
 * 24) ^ Thinking about Gödel and Turing, Gregory J. Chaitin, 2007 ISBN 9812708960 p. 242
 * 25) ^ Schneider, Ivo (2005), "Abraham De Moivre, The Doctrine of Chances (1718, 1738, 1756)", in Grattan-Guinness, I., Landmark Writings in Western Mathematics 1640-1940, Amsterdam: Elsevier, p. 105–120, ISBN 0444508716
 * 26) ^ Annotated Readings in the History of Statistics, Herbert Aron David, 2001 ISBN 0387988440 p. 115. NB. The 1866 edition of Venn's book (on Google Books) does not include this chapter.
 * 27) ^ Asia in the Making of Europe, Volume 3, Donald Frederick Lach, Edwin J. Van Kley, 1998 ISBN 0226467694 p. 1660
 * 28) ^ A History of the Gypsies of Eastern Europe and Russia, David M. Crowe, 1996 ISBN 0312129467 p. 36
 * 29) ^ Events that Changed America through the Seventeenth Century, John E. Findling, Frank W. Thackeray, 2000 ISBN 0313290830 p. 168
 * 30) ^ Great physicists by William H. Cropper 2004 ISBN page 93
 * 31) ^ On Limited Predictability, A. Wiin-Nielsen, 1999 ISBN 8773041858 p. 3
 * 32) ^ Design and Analysis of Randomized Algorithms, Juraj Hromkovič, 2005 ISBN 3540239499 p. 4
 * 33) ^ Encyclopedia of science and technology by James S. Trefil 2001 ISBN 0415937248 Page cxxxiii
 * 34) ^ Stanford Encyclopedia of Philosophy
 * 35) ^ E. Borel, Les probabilites denombrables et leurs applications arithmetique Rend. Circ. Mat. Palermo 27 (1909) 247-271
 * 36) ^ Companion Encyclopedia of the History and Philosophy Volume 2, Ivor Grattan-Guinness 0801873975 p. 1412
 * 37) ^ The Philosophy of Karl Popper, Herbert Keuth ISBN 0521548306 p. 171
 * 38) ^ Alonzo Church, "On the concept of random sequence," Bull. Amer. Math. Soc., 46 (1940), 254–260
 * 39) ^ J. Alberto Coffa, "Randomness and knowledge," in PSA 1972: proceedings of the 1972 Biennial Meeting Philosophy of Science Association, Volume 20, Springer, 1974 ISBN 9027704082 p. 106
 * 40) ^ Introduction to random time and quantum randomness by Kai Lai Chung, Jean-Claude Zambrini 2003 ISBN 9812384154 page
 * 41) ^ Karl Popper, 1957, "The propensity interpretation of the calculus of probability and the quantum theory”, in S. Körner (ed.), The Colston Papers, 9: 65–70.
 * 42) ^ Karl Popper, 1959, "The propensity interpretation of probability", British Journal of the Philosophy of Science, 10: 25–42.
 * 43) ^ Karl Popper, The Logic of Scientific Discovery p. 206
 * 44) ^ The Philosophy of Karl Popper, Herbert Keuth ISBN 0521548306 p. 170
 * 45) ^ Single Orbit Dynamics, Benjamin Weiss 1999 ISBN 0821804146 p. 83
 * 46) ^ The mathematical theory of information by Jan Kåhre 2002 ISBN 1402070640 page 218
 * 47) ^ Reviews in Computational Chemistry, Volume 23 by Kenneth B. Lipkowitz ISBN 0470082011 page 279
 * 48) ^ Martingale approximation by Yu. V. Borovskikh, 1997 ISBN 9067642711 page 287
 * 49) ^ Kendall, M. G. (1953). "The analysis of economic time-series-part I: prices", Journal of the Royal Statistical Society. A (General) 116 (1): 11–34. [2]
 * 50) ^ Probability theory: the logic of science by Edwin T. Jaynes, G. Larry Bretthorst 2003 ISBN 0521592712 page 49
 * 51) ^ Information and Randomness: an Algorithmic Perspective, Cristian Calude, 2002 ISBN 3540434666 p. 145
 * 52) ^ Thinking about Gödel and Turing, Gregory J. Chaitin, 2007 ISBN 9812708960 p. 185
 * 53) ^ Myanmar (Burma) since 1962: the failure of development by Peter John Perry 2007 ISBN 0754645347 page 10
 * 54) ^ Asia Times, June 18, 2009 Instant karma in Myanmar [3]
 * 55) ^ NY Times, November 11, 2005 A government on the move to a half-built capital, [4]
 * 56) ^ Time Magazine, May 16, 1988, Good Heavens! [5]
 * 57) ^ Encyclopedia of the Reagan-Bush years 1996 by Peter B. Levy ISBN 0313290180 page 25
 * 58) ^ Exit with honor: the life and presidency of Ronald Reagan by William E. Pemberton 1997 ISBN 0765600951 page 123
 * 59) ^ Quigley, Joan. What Does Joan Say?: My Seven Years as White House Astrologer to Nancy and Ronald Reagan. Carol Publishing Group. New York, NY; 1990
 * 60) ^ Predictability of Weather and Climate, Tim Palmer, Renate Hagedorn, 2006 ISBN 0521848822 p. 1
 * 61) ^ Storm Warning: The Story of a Killer Tornado, Nancy Mathis, 2007 ISBN 0-7432-8053-2 p. x
 * 62) ^ L. Knopoff, "Earthquake prediction: the scientific challenge", Proceedings of the National Academy of Sciences, 1999 ISBN 0309058376 p. 3720
 * 63) ^ Design and Analysis of Randomized Algorithms, Juraj Hromkovič 2005 ISBN 3540239499 p. 4