A dozen US Secret Service agents sent to Colombia to provide security for President Barack Obama at an international summit have been relieved of duty over alleged misconduct.