what about them made them look rich?

I think since he saw their car, (which in my opinion doesn't qualify a person as rich), saw the trailer and figured they were probably moving and had their most precious stuff with them in the little trailer. Remember when he says somethings about his worldly possessions? That's what he was talking about. Total plot hole because Flynn never goes outside. So I wonder how he knew what they were driving?