Length of time since he last had sex

In the beginning of the film we see Garlin's "girlfriend" break up with him, and he later says it was "just a thing" or a "fling" or something to that effect.

After Silverman sleeps over, he boasts about having sex for the first time in five years.

I find this to be odd if Garlin was having a "thing" with the first girl in the movie. I mean, it sounded like he and the first girl weren't having a relationship but were just having casual sex.