A housing charity is defending moving senior citizens out of their homes and replacing them with foreign students.