Incompressibility and Incompleteness
One exciting feature of Kolmogorov complexity is that it has an incompleteness result as a relatively straightforward consequence.

• Consider for example a string S that is Chaitin random and is of Kolmogorov complexity k.
• Suppose we have a theory T of Kolmogorov complexity n. This basically means that we have made a version of our theory in the form of a compact program that prints theorems. Call this program P; P of course has complexity n.
• Suppose k >> n (that is, k is very much larger than n). Because there are many Chaitin random strings of every length, we know that there is such a string of complexity k where k >> n.
• Can our theory T recognize or produce as a theorem the claim that S is random? Suppose we have a predicate Random, such that Random(S) means that S is Chaitin random (of maximal Kolmogorov complexity for its length, incompressible, etc.). The answer is no. We use a reductio ad absurdum argument. The assumption for reductio is that P (our compact program for T) can print that S is random.
  
  
  1. Then P can print Random(S).
  2. Then P can print S.
  3. But n the Kolmogorov complexity of the theory T (which is the size of program P) is n.
  4. Therefore, the Kolmogorov complexity of S is at most n (since a program P of size n can print S!).
  5. But by supposition the Kolmogorov complexity of S is k, and k >> n. This is absurd, since we've now shown both k is less than or equal than n and that k is very much larger than n.
  6. Hence, T cannot recognize that S is random, or even print S for that matter.

The Strength of Theories

• This incompleteness result is very general and powerful, and can be roughly stated in the following way: every theory T has a complexity n. Therre are then questions about objects whose complexity exceed n that cannot be answered by T.
• Note that there are, in mathematics at least, infinitely many possible objects (for example, relations between numbers) of complexity exceeding that of any particular theory!
• One of the limitations here is that we cannot determine precisely the Kolmogorov complexity of things much more complex than our existing theories. For any arbitrary long string, then, the Kolmogorov complexity of that string is itself typically effectively approximatable, but not effectively computable. (But, please note, the Kolmogorov complexity is still an objective feature!)
• G. Chaitin has a nicely bold way of describing the implications of this (here, you might read "no structure" to mean "no reducible structure"):

  I thought that maybe the problem is larger and Godel and Turing were just the tip of the iceberg. Maybe things are much worse and what we really have here in pure mathematics is randomness. In other words, maybe sometimes the reason you can't prove something is not because you're stupid or you haven't worked on it long enough, the reason you can't prove something is because there's nothing there! Sometimes the reason you can't solve a mathematical problem isn't because you're not smart enough, or you're not determined enough, it's because there is no solution because maybe the mathematical question has no structure, maybe the answer has no pattern, maybe there is no order or structure that you can try to understand in the world of pure mathematics. Maybe sometimes the reason that you don't see a pattern or structure is because there is no pattern or structure! (Exploring Randomness)

An Example of Hard Complexity: Chaitin-Omega

• Gregory Chaitin has describe a very complex number which he calls "Omega" (since we've discussed other omegas, we'll call this Chaitin-Omega).
• Chaitin-Omega is the Halting probability of programs for some particular universal Turing machine (and so, of all programs).
• We need one restriction: we rule out that any working program is the extension of a working program.
• Chaitin-Omega gives the probability that a program generated randomly will halt. (You would sum over the number of programs that halt of any particular size.)
• Chaitin-Omega is completely incompressible! You cannot figure it out except maybe a few bits worth and out of dumb luck.
• Chaitin has described a problem in elementary number theory that is equivalent to knowing Chaitin-Omega.
• Chaitin describes Omega thus:

  Physicists feel comfortable with randomness, but this is the black or white world of pure mathematics --- how is this possible, how can it be? Each of these bits is well-defined, it's a specific 0 or a 1, because [Chaitin- Omega] is a specific real number once I fix the universal Turing machine or the programming language that I'm dealing with. But it turns out that the right way to think about each bit is that it's not black or white, it's not that it's a 0 or a 1, it's so well balanced, it's so delicately balanced, that it's grey!
  
  Here's another way to put it. Let's go back to Leibniz. What's the idea of mathematics? The normal idea is that if something is true, it's true for a reason --- Leibniz! --- if something is true it's true for a reason. Now in pure math, the reason that something is true is called a proof, and the job of the mathematician is to find proofs, to find the reason something is true. But the bits of this number [Chaitin-Omega], whether they're 0 or 1, are mathematical truths that are true for no reason, they're true by accident! And that's why we will never know what these bits are.
  
  In other words, it's not just that Hilbert was a little bit wrong. It's not just that the normal notion of pure mathematics is a little bit wrong, that there are a few small holes, that there are a few degenerate cases like "This statement is unprovable". It's not that way! It's much, much worse than that! There are extreme cases where mathematical truth has no structure at all, where it's maximally unknowable, where it's completely accidental, where you have mathematical truths that are like coin tosses, they're true by accident, they're true for no reason. That's why you can never prove whether individual bits of [Chaitin-Omega] are 0 or are 1, because there is no reason that individual bits are 0 or 1! That's why you can't find a proof. (Exploring Randomness)

Mathematics as a (contingent) Science?

• What does this mean for reason, and for the special case of reason par excellence, mathematics?
• One way we can interpret the limitations that we've seen is that we will have to add information to our theories whenever we are trying to reason about things more complex than our theory.
• If you cannot prove P, but P looks likely or at least very useful, you may just need to assume P and work with it (perhaps abandoning P later if it turns out to lead to contradictions).
• But this blurs the lines now between pure reason and some of the fallible aspects of empirical science! Mathematics so practiced has strong analogies with science: we assume some things and work with them because of their power to explain, but are open to revision in the future! Our theories become pragmatic and fallible!

In principle limits
We have focussed on one kind of reason: formal reason (logic and math). We have had a special interest in effective procedures, which the Church- Turing thesis claims are all and only what a Turing machine can do. We have seen three kinds of in-principle (that is, inescapable) limits:

1. Godel's First Incompleteness Theorem (For theories of sufficient strength, such as arithmetic with multiplication, there are unprovable truths, or the system is inconsistent).
2. The Halting Problem (There is no program that can verify for any arbitrary program whether that program will halt. If we accept the Church-Turing thesis, this means that there is no effective procedure to find all and only the effective procedures).
3. Kolmogorov-Complexity Strength Limits (Each theory has a Kolmogorov-Complexity. For any theory T about natural numbers of complexity N, there are questions (the complexity of the answer to which exceeds N) about the natural numbers that are undecidable.)