Almost 60% of asylum seekers in State accommodation have been there for more than three years, a new report has found.