Overrated

I suppose it's a matter of what you feel is scary. There are many different types of horror. It's all a matter of taste.
This film was quite shocking at the time of it's release so you do have to take that into consideration. Also.....
This is hardly just about a guy who dresses like a woman.

In the thirties they started enforcing the Hays code which I believe was a self imposed censorship by the motion picture industry where they eliminated most depictions of violence and sex. This lasted until about 1968. So because there was no internet, television was still in its infancy... the only thing violent most people would've seen in 1960 was if they were unlucky enough to experience it first hand at the hands of a parent, were drafted into the military or if they lived in a bad neighborhood and had to fight all the time. Imagine never seeing anything graphic and then out of virtually nowhere you see someone coming at a vulnerable naked woman with a huge knife. Today of course it seems tame but it was the first of its kind... people had similar reactions to zombies when Night of the Living Dead came out, to mass murderers when Texas Chainsaw Massacre came out and to sharks when Jaws came out. Nowadays people are so immune to violence they need to see a Rob Zombie level of horror to feel anything.

Yeah, and its racist too.

This is exactly what I've been saying.

Well, it's terrifying on a psychological level, rather than emphasizing the usual blood and guts. It's also unrelentingly suspenseful.