Also remove the old secret. This way we can publish the history.