Alex Rowley has been suspended as a member of the Scottish Labour Parliamentary group after allegations about his conduct emerged.
Mr Rowley had already stepped aside as interim Scottish Labour leader and deputy leader north of the border after a former partner told the Scottish Sun newspaper that their relationship "felt like emotional blackmail and abuse".
Following calls for his suspension, Scottish Labour confirmed Mr Rowley had had the Labour whip withdrawn in the Scottish Parliament while an investigation takes place.
Business Manager James Kelly MSP said: "It is important that the investigation into these allegations is fair and transparent, and the matter will be thoroughly investigated using the Labour Party's internal complaints procedure.
"However, in light of the serious nature of the allegations, Labour at Holyrood has taken the decision to remove the whip from Alex Rowley for the period of this investigation."
Mr Rowley, who was election agent for former prime minister Gordon Brown and is an MSP for Mid Scotland and Fife, denies the allegations.
He took over as interim leader of the party after former Scottish leader Kezia Dugdale dramatically quit the job in August.
Earlier Ms Dugdale called for Mr Rowley to be suspended, as did both contenders vying to replace her in the top job.
In the newspaper article, the anonymous woman claimed he made her life ''hell with his behaviour'' and said she had been inspired to speak out after a number of other women made complaints about misconduct by politicians at Westminster and Holyrood.
Mr Rowley is the most senior person in the Labour Party to be affected by such allegations.
Ms Dugdale described the claims as "serious and deeply concerning" and said she "would not have hesitated to suspend" him if still leader.
"Sexual harassment and abuse is never acceptable," she said. "If the past weeks have taught us anything it is that we need to support victims of abuse and all allegations need to be taken seriously and investigated in a fair and transparent way."
Voting is currently under way in the contest to find the next Scottish Labour leader, with the results of the ballot due to be announced on Saturday.
Left-wing MSP Richard Leonard, a former trade union organiser, is running against the more moderate MSP Anas Sarwar - Labour's health spokesman at Holyrood.
Both men said it would be ''appropriate'' for Mr Rowley's membership to be suspended.
Mr Sarwar said: "These allegations are distressing and deeply concerning. It is right and proper that Alex has referred himself to the party's internal complaints procedure.
"The party has clear procedures, and given the seriousness of these allegations it would be appropriate for Alex to be suspended from the party while a robust, fair and thorough investigation is carried out.
"It is vital that people have confidence in our party-internal procedures. As leader, I will take a zero-tolerance approach to sexual harassment, abuse and discrimination."
Mr Leonard said: "In the light of the news stories about Alex Rowley and his self-referral to the party, my view is that the appropriate action now is that there should be a suspension of Alex Rowley whilst the matter is investigated."
He added: "In the meantime the Labour Party now has to use its internal procedures to investigate these allegations, ensuring that evidence is heard and support is provided for everyone affected - any decisions must be and should be upheld.
"We have to be very clear that there is zero tolerance for a culture that allows abuse of power in politics and across society and that these allegations will be seriously and thoroughly investigated."
Scottish Labour confirmed that Dumbarton MSP Jackie Baillie, the party's economy spokeswoman, would take over as interim leader.