Horror?!

I just caught this on tv, b/c I'll watch anything w/ the fantastic Peter Lorre. It was pretty good, although I still think his best film was The Beast W/ 5 Fingers. Anyway, while I liked this okay, I don't get why people were so mean & scared of Lorre's character, just b/c he had burn scars! Did people really treat burn victims like monsters back then?! The movie was also listed as "Horror" which I don't think describes the movie at all. Is it b/c his face is disformed, even though we never really see him w/o his mask?

"Are you going to your grave with unlived lives in your veins?" ~ The Good Girl