Jane Kennedy is to leave the Northern Ireland Office in the cabinet reshuffle caused by the resignation of Immigrations Minister Beverley Hughes.