Working thoughts on self-correcting systems

Last modified: 2025-08-27

These don't feel fully formed, but I wanted to get them down. Maybe I'll come back to them. Maybe I won't.

We need self-correcting systems — especially if we care about alignment. Humans are fallible. We’re not as good at defining clear goals as we think, and even if we were, our goals would need to adapt as the environment changes.

Self-correcting systems take time to adjust. As long as that timescale roughly matches the pace of environmental change, they work well. The catch is that our awareness of change is tied to how quickly information spreads.

Today, information moves far faster than it did even a decade ago. That shift can make self-correcting systems look sluggish by comparison. Still, I think they’re worth striving for.

Consolidating decision-making isn’t the answer — history gives us plenty of reasons why. But I do wonder if society at large is still capable of thinking on the longer timescales these systems demand.