Charlie Baker's Racist Moment

Anybody notice that moment in CBTD Charlie has walking into the gym where he has this stare-off with these random guys - one of whom happens to be black - when he looks at the black guy he kind off backs away with this look on his face as if to say "stay away." I guess being from the sticks and not dealing with black people on a regular basis does that to you. Was that a performance choice on Tom's part, reflective of his own nature, or just as the character was supposed to be written