Editor’s note: We are delighted to present a new series by William Dembski, adapted from his Substack. This is the 13th post. Find the full series so far here, “My Personal History with Information.”
I wrote up the mathematical details of the insights I shared at the Niels Bohr Institute in a paper that I put up on my blog, designinference.com (I later moved that blog over to billdembski.com). The paper was titled “Searching Large Spaces: Displacement and the No Free Lunch Regress” (available here), which appeared in early 2005, and thus before the Evolutionary Informatics Lab got off the ground and before my collaborations with Bob Marks, Winston Ewert, and George Montañez.
A Proper Information-Theoretic Framework
So really, with this paper, the theoretical basis for why there would be no way around displacement in the evolutionary computing literature was made clear. Still, that paper needed to be situated within a proper information-theoretic framework. Bob Marks, with his extensive background in engineering and theoretical computer science, helped to provide just such a framework. At the same time, because of the mathematical demands the “Searching Large Spaces” paper placed on readers (measure theory, functional analysis, and vector-valued integration), case-by-case deconstructions of inflated claims for the ability of natural selection to create information were, in addressing biologists and non-technical researchers, still needed.
The “Searching Large Spaces” paper elicited from at least some readers a respect that I had not seen before. As happens in academic jockeying, it helps to be able to show off some expertise that others lack. This paper forced me to dust off some of the more advanced mathematics from my math grad student days going back about twenty years. I found writing up its results particularly satisfying. At least one technically proficient critic of intelligent design, Thomas English, who previously had used a pen name and who after this paper publicly identified himself, noted that this was the first paper that he thought showed intelligent design offering theoretical insights not easily dismissed.
Specified Complexity as an Actual Information Measure
Before describing the aftermath of this paper, I need to note another key development in my thinking about information that occurred in 2005. That year, my friend and colleague Jay Richards was guest editing a special double issue of Philosophia Christi, the journal of the Evangelical Philosophical Society. This issue focused on intelligent design. Jay wanted me to address how exactly specified complexity constituted a type of information. Till now, specified complexity was less a form of information and more the basis for a type of statistical inference. The challenge then was to say exactly how specified complexity or complex specified information was information in a conventional information-theoretic sense.
The paper I submitted to this special double issue was titled “Specification: The Pattern that Signifies Intelligence” (available here). Looking back at this paper, I don’t think it’s all that great. It rehashes a lot of what I had already written in my 2002 book No Free Lunch. What it calls specified complexity incorporates not only what has become today specified complexity in its full information-theoretic form but also incorporates what I defined as probabilistic resources, both replicational and specificational. If you will, I overcomplicated specified complexity in this paper.
Replicational and Specificational Resources
Probabilistic resources denote the number of opportunities to bring about an otherwise chance event. The more such opportunities, the more likely the event is to happen regardless of the event’s inherent improbability. Probabilistic resources can be distinguished as replicational and specificational. Think of replicational resources as arrows in your quiver and specificational resources as targets so that hitting any of them counts as a success. Imagine, for instance, shooting an arrow into a forest and hitting someone. If the explanation for hitting someone is chance, the actual chance of hitting someone will depend on how many arrows are in your quiver (replicational resources — the number of shots you can get off) and also how many people are in the forest (specificational resources — the number of targets that can count as success).
As it is, I factored into my definition of specified complexity in the Philosophia Christi paper these probabilistic resources. This was ill-advised because it made specified complexity depend on the relevant probabilistic resources in a given situation, and these could vary. In the lead up to my definition of specified complexity in that paper, however, I defined what I called specificity, denoted by a Greek lower-case sigma:
The first term under the logarithm, the lower-case phi term, turned out to be an exponential of Kolmogorov information. The second term under the logarithm clearly denoted a probability. In fact, this definition proved to be equivalent to the definition of specified complexity that Winston Ewert articulated for a 2012 conference paper:
Ewert’s Definition of Specified Complexity
When Bob Marks shared with me Winston’s definition of specified complexity, I pointed out that my 2005 formulation was equivalent (the phi in my equation was the same as the K in his, albeit raised to the power 2). Winston and Bob, who had written up the latter formulation of specified complexity, graciously added me as a co-author. I say graciously because even though what I had come up with was equivalent, it was not nearly as clean and clear as what they produced.
When Winston and I collaborated on the second edition of The Design Inference in 2023, we provided the latter definition of specified complexity, which is to say a difference between Shannon and Kolmogorov information (subject also to the condition that the underlying language was binary, prefix-free, and Turing complete). This definition gives specified complexity a fully mainstream information-theoretic formulation.
Next, “Breaking into the Peer-Reviewed Engineering Literature.”