THE Olympic Games have displaced more than two million people in the last 20 years, mostly minorities such as the homeless and poor, a rights group said yesterday.