A former mayor of Kilkenny has said that opposition to the planned mosque and Islamic centre in the city was not an issue of religion, but one of traffic management.