American swimmers Jack Conger and Gunnar Bentz were removed from a flight to the US by the Brazilian authorities, the United States Olympic Committee said.