Information Theory Primer

The Decision Space

Every software project starts as a vast space of possible paths. Each decision — a framework choice, an error strategy, an API design — eliminates alternatives and narrows what remains. Entropy measures how many meaningful choices are still open.

5 OPTIONS 2 REMAIN 1 CHOSEN 2.3 bits 1.0 bits 0 bits
Each decision prunes the tree — entropy drops toward zero
The key question isn’t how much entropy a project has — it’s when that entropy gets resolved. Decisions made at design time cost tokens. Decisions deferred to runtime cost debugging, deployments, and outages.
Press to explore interactively
Remaining Paths
120 / 120
6.9 bits
Shannon Entropy — area = cumulative uncertainty
H = log₂(N) bits
Step 0 / 5
Chosen Available Pruned Space play step