My theory is...
...that Colm with age over time became worried about his own legacy and that he had done nothing with his life and hence projected that onto Padraic, who represented and reminded him of his own insipid and banal existence. Breaking it off with him was a way of making himself feel better about his own wretched life. Since Padraic did nothing wrong and was somewhat of a simpleton with a good heart I can understand his hurt and confusion at being blindsided like that.
This is a remote island off the coast of Ireland in the 1920's I believe. There are few inhabitants, let alone suitable blokes his own age, and no smartphones, computers, TV or even radio. You've got a small handful of people to choose from aside from the farm animals and family (Padraic's sister). And you constantly run into them even if not friends anymore, so the situation is quite different from someone ghosting or unfriending you in today's world, although it still can be hurtful.