The HSE has threatened legal action against the Health Information and Quality Authority (Hiqa) over its report.
“I don’t believe it is appropriate for Government agencies to sue each other,” said Mr Varadkar.
According to the HSE it considered going to court to get a “a more balanced and fair” report when Hiqa refused to allow it respond to findings.
The HSE said the report contained inaccuracies and lacked balance.
The draft report is understood to place some of the blame for the hospital’s shortcomings on senior managers within the HSE.
The HSE wrote to the Department of Health earlier this month asking it to intervene in the matter.
Following a meeting last week, both bodies are working in accordance with an agreed process to complete the HSE’s input at the report’s final stages.
“I would expect the two bodies, if at all possible, to sort out any dispute they have without going to law,” said Mr Varadkar.
He had not received the report but action had already been taken to strengthen hospital management. Additional consultants had also been appointed.
“We are certainly not waiting for a report to address some of the issues in Portlaoise,” he said. “A lot of work has been done.”
Asked if he was pushing for the report to be published quickly, Mr Varadkar said it was important that the report was accurate and anyone criticised in it had the right of reply before it was published.
“Certainly, I would like to see it published sooner rather than later,” he said.
A spokesperson for Hiqa said the report was still in draft form and had yet to be approved by the board.
The report has to be presented to the minister before publication.
The draft report suggests some of the blame for what occurred was due to shortcomings in senior HSE corporate management.
Fianna Fáil’s spokesman on health, Billy Kelleher, said it was unacceptable that, more than a year since the report was commissioned, attempts were underway to delay its publication.
“It is now up to Minister Varadkar to ensure that no deliberate efforts are made to avoid accountability for what happened to these babies,” said Mr Kelleher.
“There could hardly be a more serious issue than the health and safety of our maternity units, where new life is being brought into this world.”
Independent TD Denis Naughten said the Portlaoise report highlighted the need for the inquiry at Portiuncula Hospital in Ballinasloe, Co Galway, to include a review of management once they became aware of staff concerns regarding the operation of the maternity unit.
The inquiry at Portiuncula Hospital will cover the delivery of seven oxygen- deprived babies at the hospital last year, two of whom died.
Mr Naughton said that the Portiuncula inquiry must find out what actions management took when they were made aware of staffing, supervision, and training concerns as far back as July 2013.