Some former residents of Apollo House have returned to the building after claims of drug use at the alternative accommodation offered to them.