Question of the two times

I was a little puzzled by that, too. He had been telling her that he would’ve liked to have been admitted to her bedroom a lot more often over the years than what he was. She was saying that she would imagine him on his way to her room and on only two occasions did he actually show up at the same time she was imagining it. After thinking about it, I think this was her way of telling him that she had wanted him to visit more often but she was too reserved to come right out and ask him to?