Campaigners say the closure of an asylum centre in Galway will be a huge loss to the local community.