If this was filmed in Tarrytown, NY...

It's the old Vanderlip mansion. If I can find "The Films of Merchant-Ivory" on my bookshelf I'll add the name.

Edit: Found it. The house is called Beechwood.