who was having an affair with the bumpuses?

I don't think either of them would be into them.

The old man hated the Bumpus dogs, it would never have worked out.

the lamp!

If you read the short story, Shepherd gets into much greater detail about the Bumpuses. In fact, one of their daughters named Cassie came home from a reform type school and was built like a goddess. She kissed her old man in a way that was not normal. The rest of the Bumpus women were described as homely. Not kidding, the story is hilarious.

So if it was the Old Man, it would have been with the daughter Cassie.

Projection is a helluva drug!