The time keeper said what he did was much worse.
Yes, they kept hinting at that but never expanded on it.
I've wondered the same thing, Must have been something HUGE.
Father would fight, win, then give away time...and hope.