The governor of Ireland's only women's prison has resigned saying her position became "completely impossible".