Confused!

She says he was there. And he was, earlier in the movie he goes around to the womans house to try and find out what Alex meant by his Maze comment. Joy was drugged in another room while he was there. That's why she said that and then Keller just put two and two together.

Yeah she didn't say he was there taping her mouth. Remember she was still under the influence of the drugs she'd been given so wasn't able to fully form the sentences she was trying to say.

What she said was “you were there". And “we had tape on our mouths". She is trying to tell him that she had seen him come into the house of the kidnappers but they couldn't call out to him because their mouths were taped shut.

He realises the place he's been was the old ladies house and so he runs straight over there as he knows his daughter might be in immediate danger now Joy has escaped, he doesn't want to waste time explaining to everybody and then have the cops questioning him about why he'd been there already etc.