Police wielding canes and backed by water have broken up a rally of about 500 people against Malaysia's longtime leader Mahathir Mohamed.