Where was this supposed to take place?

I don't believe they mention it by name. I know it was shot in Alphabet City, East Village (if that's what you mean to ask). Here's a passage from a John Flynn interview, with a bonus snippet on his work with JMV:

SC: Defiance (1980) was filmed mostly in New York City’s East Village at a time when street gangs did in fact control that area of Lower Manhattan. Did these hoodlums disrupt filming while you were on location in Alphabet City?

JF: Actually, we did have problems with gangs. We were only shooting for a week in New York. Puerto Rican gang members would throw *beep* down on us from rooftops. Somebody shot the windshield out of one of our picture cars. The cops would help us clear the streets, but they wouldn’t venture onto rooftops at night to rout the gangs out - not for a movie company. So we hired a local karate club to protect us! They were young gang members too. We put a dozen of them up on rooftops with walkie-talkies and we had no further problems.

SC: How did you manage to elicit such a fine performance from journeyman actor Jan-Michael Vincent?

JF: It wasn’t easy. Jan was a drinker even then. He had Heinekens for breakfast. There was a night scene where we literally had to prop him up. Poor Jan. He latched onto Danny Aiello. Jan loved Danny and tried to give him more of his own lines in the picture. I told Jan he couldn’t mess with the script like that. But Jan was a sweet guy. He never believed that he was an actor, though. He was embarrassed to be an actor. He always thought he was doing an awful job and that people were laughing at him. You had to keep telling him he was wonderful and he would do whatever you wanted him to do. Jan was like a little kid, but he just didn’t believe in himself. Talk about actors’ egos. He was the opposite. This was an actor with a non-ego.