Okay, so I get why Martin was wearing women's glasses, but I missed what was up with the purse.
Martin was searching through Daphne's purse for his glasses case.
The glasses made him look feminine. The purse comment was meant to exaggerate that.