Editor’s note: We are delighted to present a new series by William Dembski, adapted from his Substack. This is the fourth post. Find the full series so far here, “My Personal History with Information.”
In the early and mid 1980s, A. E. Wilder-Smith’s ideas about information excited me but I could think of no way to contribute to them as a researcher pushing back the frontiers of knowledge. I was a graduate student in mathematics, focused on probability theory. I got my PhD in 1988 with joint doctoral supervisors in math under Patrick Billingsley (1925–2011) and in physics under Leo Kadanoff, who is pictured with me at the top (1937–2015). Both were well-known in their fields, Billingsley more as an expositor, Kadanoff as a star researcher (he received the Wolf Prize and was Nobel caliber). Work on nonlinear dynamics in Kadanoff’s research group inspired my work on probability.
As it is, probabilities are everywhere in information theory, and yet probability theory and information theory have a very different feel. Interestingly my math supervisor Pat Billingsley wrote widely used probability textbooks, such as Probability and Measure (3rd edition 1995); but he also wrote a book titled Ergodic Theory and Information (1965). So he saw the connection. And yet the fields are distinct.
Radically Improbable Events
To vastly oversimplify, probability theory is about calculating probabilities whereas information theory is about moving bit strings across communication channels. Probability theory feels like pure and applied math. Information theory feels like engineering. Engineers are concerned with moving signals reliably, which means their attention is on average performance of communication systems. Hence the ubiquitous “entropy,” a probabilistic average of information measures. As I became interested in the late 1980s in radically improbable events and how they might point to the activity of intelligent causes, I didn’t see much use for information theory as such. I was focused on particular outcomes, less so on ensembles of outcomes, as in information theory.
I had read Thaxton et al.’s The Mystery of Life’s Origin in the late 1980s (I recall borrowing it from the Evanston Public Library). Information played a prominent role in that book. Even the term specified complexity, with which my name later became closely associated, appeared in the book. But I didn’t see in specified complexity a direct connection to mainstream information-theoretic concepts. As with Wilder-Smith, the emerging design community kept talking about information. We were gesturing at it. There clearly was something to it. And yet there was work to be done connecting our emerging understanding of information with the standard formalism of information theory.
A General Account of Uniform Probability
I defended my math dissertation in the spring of 1988. One of the problems I solved in it was providing a general account of uniform probability. I was trying to understand the nature of randomness, and it seemed to me that the most random of all things was sampling from a uniform probability distribution. I wrote up that account of uniform probability and published it in 1990 in the Journal of Theoretical Probability. This work was done at the height of the excitement over nonlinear dynamics and chaos, which commanded a lot of my physics advisor’s (Leo Kadanoff’s) attention back then.
Next, “Randomness by Design: A Derivative Idea, and a Fundamental One.”