THE backlash against Finance Minister Richie Ryan's Wealth Tax proposals led Fine Gael TDs to warn the Taoiseach they would lose the next election.