Why did Hillary and Besser leave the show after Season 1?

Not sure, but I would guess that they were going in a different direction in season 2 with more standard sitcom episodes, so the surrealistic elements of the show were taken out: Lou having a gorgeous blond girlfriend, a man in his 50s playing a little kid, a caricatured Italian with a different business in every episode. Also, Sid Fields dropped all his other roles, he was just the landlord.

Which is why most prefer Season 1.

My edjumacated guess is that there were several factors involved:

Typical second season budget cuts, and for a show that probably had a meager budget to begin with.

Joe Kirk's divorce from Lou's sister by itself may not have been reason enough for his absence but if the rumors of his abuse of Marie are true then one could reasonably conclude that this made him persona non grata in Lou's eyes. A shame on so many levels (if true)