The women of Paterson.

The Laura character treats Paterson like a pet, he's reluctant to talk back for the fear she will do something he won't like. Marvin her other pet goes to the "garage" for acting out so that must be it. People, especially women like pets because they can't hurt them with words. And so with the infinitesimal Paterson the house is basically occupied by her alone.

Then there's Marie. Her pet is loyal but must have been talking back so she sent it to the "garage" indefinitely.

Then there's the degrading remark by the girl, about Paterson being a bus driver who's also a poet. Her mother's influence.

But then Paterson has all he needs. He's like the town itself personified, calm, accommodating and nonjudgmental. Everything a woman can truly wish for.