Almost one million EU nationals currently working in the UK are considering leaving the country in a potential "Brexit brain-drain", according to a new survey.