So was Fisk corrupt or just really mean?

I spent nearly all the movie assuming that Fisk was padding the roster and pocketing their wages. But then towards the end it seemed he was just banishing people from camp, but they were still working and perhaps collecting wages. But in the end he rolled the timekeeper for his money, so now I don't know what to think. What was Fisk all about?