Concerns have been raised over the use of a site in Goatstown for social housing and Traveller accommodation.