Why didn't have sex with samantha ?

I believe he mentioned on the phone that he sees women, and he can't get himself to do the things he wants with them. He said he'd had only three sexual partners, a number he perceived as quite low. His interactions with Samantha were illustrating how uncomfortable he is with women in real life, as opposed to how comfortable he was on the phone.

[deleted]