NewKindOfScience

• Rare to have a non-fiction book that is over-hyped. Even rarer the non-fiction book that is over-hyped by the author. This book is probably singular in that it is over-hyped by the author DIRECTLY WITHIN THE PAGES OF THE BOOK!!
• Could have benefited from an editor. My guess is the book could have been made to be 1/3 as long and still convey the same information.
• Beginning was very interesting. Actually a fast read. 2nd half seemed largely disconnected from reality, read more like a crackpot email.
• Some good insights:
  • Can get complicated behavior from simple programs and simple initial conditions
  • Good discussion of sources of observed randomness.
  • Universality is a much more widespread phenomena.
  • Interesting take on Godel's Incompleteness theorem
• Like the idea of explicitly categorizing all possibilities and searching through them systematically.
• Studies suggest, if the rules of the system are sufficiently simple, then you can only ever get purely repetitive behavior. If the rules are more complex, nesting will often appear, and to get complexity in the overall behavior of the system need to go beyond some threshold of complexity. Surprising finding is that the threshold is typically extremely low. (of course as is one of the main problems with the book most of this is simply the author's opinion. And things like "sufficiently", "complexity" and threshold are not defined in any meaningful way)
• Adding more dimensions does not ultimately seem to have much effect on the occurrence of behavior of any significant complexity.

• Three basic mechanisms for randomness:

  • randomness explicitly introduced into the underlying rules for the system
  • The initial condition of the system are chosen randomly, but then the subsequent evolution of the system is assumed to follow definite rules with no randomness.
  • randomness produced by simple programs through iteration, this can be done without randomness already being present outside the system one is looking at.

  (Note: When intrinsic randomness is the dominant mechanism it is realistic to expect at least some level of repeatability in the "random behavior one sees in real experiments)

• Discovery that launched chaos theory: there can be systems whose sensitivity to their initial conditions is so great that no machine with fixed tolerances can ever be expected to yield repeatable results. Implies that to have enough information to make complete predictions one must know not only the rules for a system but also its complete initial conditions.
• An assumption of tradition math that the only relevant attribute of any number is its size. This idealization suggests that all numbers which are sufficiently close in size should somehow be equally common. If this were true it would imply that typical initial conditions would inevitable involve random digit sequences. But there is no particular reason to believe that an idealization which happens to be convenient for mathematical analysis should apply to the natural world.
• natural selection should have a hard time selecting for complex behavior: given a particular program, it can be very difficult to see what the behavior of the program will be, let alone go reverse-engineer programs that lead to specific behavior. (counter point: natural selection does this "blindly"). Suggests that natural selection can only efficiently work on simple features of an organism. This is directly opposed to the common view that natural selection is what gives rise to all the complexity seen in nature.
• Practical experiments almost always end up involving only initial conditions that are fairly simple for us to describe and construct.
• The use of memory is what underlies almost every aspect of human thinking. Capabilities like generalization, analogy and intuition immediately seem very closely related to the ability to retrieve data from memory on the basis of similarity. (An argument for slip box!!!)

• A universal system means that by setting up an appropriate initial condition, it is possible to get the system to emulate any type of behavior that can occur in any other system.

  Computers

• In the past it was assumed that universality is a rare and special quality, only possessed by systems that are specifically constructed to have it. Universality is a much more widespread phenomena.
• Nothing fundamental can ever be gained by using rules that are more complicated than those for the universal cellular automaton. Given the universal cellular automaton, more complicated rules can always be emulated just by setting up initial conditions. (JA: suggests a fundamental rules vs initial conditions trade off)
• As one looks at cellular automata with progressively greater computational capabilities, one will eventually pass the threshold of universality. Once past this threshold, the set of computations that can be performed will always be exactly the same. (JA: agree in principle. In practice you need to know how to "program" the cellular automata, and have constraints on time and memory. Not clear all are really qualitatively equal. Are there fundamental physical limitations to these "practical" concerns ? eg: enough bits/time in the universe. undecidable search needed to fine the right ICs ect. )
• In a proof system, as soon as the question of whether a particular string can even be reached is undecidable it immediately follows that there must be either incompleteness or inconsistency. To say that such a question is undecidable is to say that it cannot in general be answered by any procedure guaranteed to finish.

Incompleteness

• There must be integer equations that have no solutions but where this fact cannot be proved from the normal axioms of arithmetic. At some level it is a consequence of the involvement of infinity. To have a finite way to address questions about infinite numbers of possible integers, is the main justification for setting up a system of axioms. Instead of handling objects like integers directly, axiom systems can just give abstract rules for manipulating statements about them. Within such statements one can refer to infinite sets of integers just by a single symbol. (JA: Godels incompleteness as a limit on power to generalize/summarize an infinite system). There has been tremendous success in mathematics that can be attributed to this approach. but the remarkable fact (from Godels incompleteness theorem) is that whatever one does there will always be cases where the approach must ultimately fail.

Incompleteness

• Claim: Simple infinite quantities like 1/0 or the total number of integers can readily be summarized in finite ways in symbols, the same is not true of all infinite processes.
• Human artifacts are biased by the fact that they must be simple enough so we know what they will do when used. Nature operates under no such constraint.
• Claim: very little of current human technology depends on ideas about primes.