Disagreement with ideas that pass for common sense, Part 3: Intelligence

I don’t believe in the naive account of intelligence, a material structure that allows you to magically find answers to questions.

Accurate answers are arrived at through correspondence with the facts. In reality as I understand it, this means:

1. You have to conduct a search to discover facts.
2. You have to compress the facts you found into a smaller representation for storage. This sometimes has an accidental benefit of ditching needlessly complex theories (Occam’s Razor). You also have to decompress them to generate predictions.
3. You need physical energy to conduct searches, compress and decompress the results. Energy is a scarce resource. If you don’t have the resources to do all this, it’s not possible for you to take over the world by being smart. (That’s also a silly idea for other reasons like needing physical strength to put your plans into motion.)
4. Even after doing all that, the context in which a physical system operates remains underspecified in several ways:

a) The number of combinations possible in physical space are currently much larger than what our knowledge systems can predict. (This is an understatement.) It’s always possible a competitor will construct a larger system.

b) At the limits of representation, it remains possible to construct contradictions through self-reference.

c) Ignoring self-reference, it’s unclear whether a mathematical formalism exists that generalizes knowledge representation across all contexts. Neural networks use a piecewise linear representation. Other possibilities exist.

…