I keep returning to a simple intuition: intelligence has something to do with finding a compact account of a complicated world. Not merely remembering more, but discovering what can be reused.

That intuition is the starting point of Bits to Mind. It is also a position I want to examine rather than turn into a slogan. “Intelligence comes from compression” is an appealing provocation. It is not, by itself, an adequate definition of intelligence.

What is being compressed?

A useful abstraction replaces many individual observations with a structure that explains them. A circuit model is not a list of every voltage it has ever produced. A programming abstraction is not a copy of each place where it is used. Both are valuable because a limited description supports many possible situations.

There is a precise connection between probabilistic prediction and coding: a predictor that assigns higher probability to the observations can support shorter codes for those observations. Delétang and colleagues explore this relationship in Language Modeling Is Compression. That connection is more concrete than the loose idea that “smaller is smarter.”

The minimum description length principle adds an important discipline: account for the description of the model as well as the data given the model. A model that simply hides the whole dataset inside itself has not made the cost disappear. Grünwald’s tutorial is a useful introduction to this distinction.

Total descriptionL(M) + L(D | M)A model is not free just because we stop counting it.

Compression depends on the language

Consider an alternating bit sequence. A run-length encoder can make it much longer, even though “repeat 01” describes its pattern very simply. Failure to compress with one representation is not evidence that no structure exists. It may be evidence that the representation is wrong for the task.

The small compression experiment on this site makes this failure visible. It deliberately uses an unsophisticated coding scheme. Its purpose is not to measure intelligence, but to show why every claim about compression needs a model, a cost convention, and a domain.

Useful understanding must travel

The version of this idea that interests me most is not retrospective. A representation should do something on a new case: predict a response, suggest an experiment, identify a constraint, or support a design decision. Otherwise, we may have described yesterday efficiently without becoming more capable tomorrow.

In engineering, there is another demand. The representation must help an agent act in a world where mistakes are not just incorrect tokens. A compact explanation of an amplifier does not yet select a manufacturable geometry, recover from a failed tool call, or establish that a physical implementation meets its specification.

Compression may help explain how useful structure is learned. It does not, on its own, explain how that structure becomes reliable action.

This is why I am interested in the sequence from representations to agents to environments. Models need ways to inspect, modify, test, and revise. An environment needs an independent way to distinguish an attractive explanation from a supported result.

A lens, not a conclusion

I would not judge a design assistant solely by how concise its answers are, how small its weights are, or how few tokens it uses. Those quantities can matter operationally, but none is a substitute for the quality of its decisions. An aggressively compressed summary can discard exactly the exception that matters.

My working question is more specific: what structure does the system preserve, and what new decisions does that structure make possible? That question connects learning, software architecture, circuit models, and scientific explanation without pretending they are the same thing.

Bits to Mind names a direction of inquiry. The aim is not fewer bits at any cost. It is better representations, more capable systems, and evidence that they work.

Reading behind this note

Ideas in progress. Corrections welcome.

Find me online