Weather

For most of the heyday of Hollywood film-making, the movies were usually "sited" on the East Coast, where the majority of Americans lived. Obedience to verities like (the impossibility of) swimming in Westchester in late autumn was just not a dealbreaker for audiences east or west, really.

Oh, and there's always a goofs section for every IMDb film page, including this one; it's just that yours will be the first, if you want to add one.

I doubt it was a goof. I think it was intentional. However, if they showed everyone in heavy coats on and sweaters, there would be no way of showing off the lovely mansion, swimming pool and garden.