Why Did He Resign? (Possible Spoiler)

He allowed Smiley to pump him for information about the General's call that he was not supposed to give. By speaking to Smiley he was violating policy that said everyone keep the emigre' groups at arm's length.

When Enderby found out the young man had spoken to George his career was over. Enderby canned him; it was a firing, not a resignation.

In the book, Mostyn is described as resigning the service a few months later as part of the 'wastage rate', ie the high turnover of staff. I think Mostyn was taken with Smiley and had almost 'old school thinking' where more should have been done for the general, and more consideration given to the emigres than simply treating them like sponging cattle, instead of heroes as many were.

Stricklen, Lacon, Mostyn's lazy superior, and most the others, were only concerned about covering their own arses or following politically correct protocols to not offend anyone. They were not bothered about the fact someone they were meant to be protecting and supporting was killed partly because he had to run it all with minimal funds and no support. I think that meeting in the safe house was a straw that would break Mostyns romance with the circus and lead him to disillusionment, realising that the decent people like Smiley were now truly only in the past, replaced by bureaucrats.

In the book: grief and guilt over the General's death drives Mostyn to resign. In the film: Enderby knew Mostyn gave Smiley privileged information, and fired him for it.