Did Chang and Fai have implied sex?

I think so. You mean the scene he was draped in a red blanket, right?

I think what gave it away was Fai even confesses to Po that he's been having sex with someone else.

"Whether I left society or it me I cannot say. I suppose you could make an argument for both sides."

My take on that scene is how Chang went to great lengths to care of his friend Fai. Chang brought Fai home and put him to bed. Having sex when Fai was so drunk would not have been an honorable thing to do, and I don't believe Chang did that.