Ed Miliband has hit back at critics of his leadership, insisting he had a “very clear plan” but that restoring public confidence in Labour would take time and that he relished the fight.