Most City of London workers plan to quit their jobs within five years, mainly because of financial worries and fear of terrorism, according to a new report today.