If Kane had to do it again, he would have left with his wife from the get-go

At the start, his wife thinks the two of them should hurry off and just leave. She argues that Kane is no longer the sheriff, so Miller is not his concern / problem anymore. Forget being brave. Forget protecting people. Forget killing the wicked. Just go. That’s the wife’s opinion

But Kane feels differently. He thinks he has to stay. He just can’t give up the values and duties of a sheriff. He thinks he has to be brave, can’t be considered a coward who ran from Miller. He has to protect this town, he has to kill the evil wrongdoers. The sheriff position means that much to him. He dedicated his whole life to that role. He was willing to die for it. He was willing to let his wife leave on the noon train for it.

But at the end, I believe Kane sees his error. He debases the sheriff star by throwing it in the dirt in front of everyone (Contrast that act to the way he had to be goaded into taking off the sheriff’s star at the beginning). He then leaves with his wife, ignoring the townspeople around him; and that’s exactly what his wife was asking him to do at the start!

Throwing the star in the dirt reasonably symbolizes a renouncement of all the values of a sheriff — He stayed to fight Miller in order to be brave and protect the town, like a sheriff should. But he realizes that, sometimes, a town doesn’t deserve protection, especially when they won’t protect themselves. After all, the town didn’t want to help him face Miller. Hence, Kane should have focused on his wife alone, the only one who helped him. He should have acted like a coward and ran, just the two of them. Instead of focusing the sheriff star and the town, he should have focused entirely on his wife.

All things considered - if Kane has to do it all over again, he would have left at the start.