Turnbull, a multi-millionaire former tech entrepreneur, won a secret party vote by 54 to 44, Liberal Party chief whip Scott Buchholz told reporters after the meeting in Canberra.
Foreign minister Julie Bishop was elected deputy leader of the party which, with junior coalition partner the National Party, won a landslide election in 2013.
“Ultimately, the prime minister has not been capable of providing the economic leadership our nation needs,” Turnbull told reporters at parliament house ahead of the vote.
“We need a different style of leadership.”
Abbott pledged to fight the challenge but was ultimately unsuccessful in overcoming the “destabilisation” that he said had been taking place within the party over the last few months.
He walked stony-faced out of the party room following the vote and did not speak to reporters.
Abbott ousted Turnbull as leader of the Liberal Party in 2009, although Turnbull has consistently been seen as a preferred prime minister.
However, Turnbull’s support for a carbon trading scheme, gay marriage, and an Australian republic have made him unpopular with his party’s right wing.
The challenge came as Australia’s A$1.5trn economy struggles to cope with the end of a once-in-a-century mining boom and just days before a by-election in Western Australia state widely seen as a test of Abbott’s leadership.
Abbott emerged badly weakened from a leadership challenge in February, which came about after weeks of infighting, and pledged a new spirit of conciliation.
He and his government have since consistently lagged the centre-left opposition Labor Party in opinion polls, helping to fuel speculation over how long his party would give him to turn things around.
Peter Chen, a political scientist from the University of Sydney, said Turnbull faced the same problem as Kevin Rudd, a former Labor prime minister toppled by his own party.
“He is popular with the public, but not within his own party, Chen said.
Abbott has continued to defy popular opinion inside and outside his party, despite pledging to be more consultative, blocking his MPs from supporting same-sex marriage, and announcing an emissions reduction target criticised as inadequate by environmental groups.