The governor of the Central Bank has said the poor of Ireland have suffered the most during the economic crisis, reopening the debate as to who has been worst affected by five years of austerity.