A Catholic priest in the Co Antrim town of Larne has said has had no option but to advise some of his parishioners to leave the town after they have come under repeated sectarian attack.