Which movie and what is the quote?

I recall Leslie Mann in a sincere scene where she's telling a man, likely her husband, something like,
Just because you don't get mad doesn't mean you're not hurting me.
I thought maybe it was This Is 40, but all I can find are comic quotes.
I've always been intrigued with this idea, how some people avoid announcing their displeasure and think they aren't at fault even though their actions show they really don't care enough, or want to avoid confronting a situation.
When it's obvious you want someone to care, and they just placidly go about their business, it hurts !