Now, let me start off by saying, I am a HUGE fan of Tom Selleck. I've enjoyed practically everything he's done and I enjoy him in the Jesse Stone movies as well.
Having said that, Jesse Stone is roughly 5' 11" who weighs about 180...and is in his mid-30 years. This is supposed to make him a seemingly non-threatening person when first meeting him. Gives him the advantage.
Tom Selleck can't portray that...and there are moments in the book where that advantage comes to the forefront.