- Shannon entropy (global) quantifies the amount of disorder/uncertainty
- Mutual information quantifies how much the future remembers the past
- Input entropy considers the entropy in the neighborhoods determining state transition
- (we can also look at larger neighbourhoods to quantify larger structures)
- Complex rules tend to have moderate mutual information and moderate entropy with relatively high variance (esp. input entropy)
- Shannon entropy for larger neighbourhoods over time gives us an idea about self-organisation in the system