What was the deal with the boats just hanging around the mole?

I'm just now watching the movie for the first time and I thought the premise was to get these guys back home. If that's the case, you'd think there'd have been a little more urgency in getting these first two boats away from the dock and headed on their way. The first one that got bombed...it seemed like it could have taken off earlier, but the second one? They boarded during dusk, were all below eating bread and jam, drinking tea, a few scenes later, its full-on night time, the boat FINALLY starts it engines and of course it gets torpedoed the moment it heads off from the dock.
What gives? Maybe its just the way that the scene was shot, but it looked like the boat was just sitting there for...possibly hours. I'd have thought that there'd be a lot more urgency than that.