They tried to make it look like Harp was remorseful for his lust, and that he only cheated on his wife one time, whereas his wife had a continuing physical and emotional affair with one man.
However, the screenwriters didn't achieve this goal. The reasons as to why are numerous. They utterly failed to show the passage of time with his wife's affair, so we get no context in this movie as to how long it goes on for. Nor do we see any evidence that their "relationship," if you can even call it that, gets anywhere close to "serious." Those who argue otherwise are, simply put, wrong; we hear Garp's wife outright say that she told Michael they'd have to break up immediately, should they ever be found out. These sound like the words of someone who's using someone else for sex only, not because they're actually in love with the person. In addition, she only performs fellatio on him as a desperate bid to get the persistent pest out of her hair for good. NOT because she feels any real connection with Michael.
We also see Garp womanizing so often in this movie, that by sheer quantity alone, his sins of impropriety far outweigh hers. In fact, his womanizing is clearly part of the reason why she commits adultery in the first place. He's very obviously cheated on her, and there's an obvious unhappiness pervading the marriage (partially due to his flagrant womanizing). In real life, this often drives others to cheat on their spouses, and it's certainly no different in this fictional situation. And yes, it's bad that she cheated on him. But ultimately, Garp put her through MUCH more emotional stress due to his seemingly-constant womanizing. Yet he has the gall to get angry when he realizes she's giving him just desserts. This makes Garp extremely unlikable, imo, and is a major writing failure.
Ultimately, this screenplay is a poor adaptation and/or reimagining of the novel, and it barely develops several crucial aspects of the story, so I forgive you for not noticing this. And I agree with your point wholeheartedly.