A well-known Labour councillor has quit the party, saying it has abandoned the very people it was supposed to represent.