The health services watchdog HIQA said it cannot guarantee that services in Midland Regional Hospital in Portlaoise are safe.
HIQA's report into services there said the HSE failed to take appropriate action to deal with a litany of issues in the hospital, and said the HSE is "not a learning organisation".
It also found that until last year, the hospital's emergency department - which was open 24 hours a day - only had a consultant on site for 24 hours each week.
The report was undertaken after a Prime Time documentary investigating the deaths of five babies at the hospital.
In a statement accompanying the report, HIQA said the hospital "viewed itself as operating as a model-3 hospital" which provides a full range of services for all manner of injuries – but that "services at the hospital were neither governed, resourced nor equipped to provide this level of care."
“Our findings point to failures, over a number of years by the HSE at a national, regional and local level to decisively address numerous clinical governance and management issues," said HIQA’s Chief Executive Phelim Quinn.
"This impacted upon the quality and safety of services provided at Portlaoise Hospital.”
The HSE said the HIQA report was carried out in October last year and improvements have been made at Portlaoise Hospital since then.
The Dublin Midlands Hospital Group said there are 16 new midwives working in the hospital and a Clinical Director from the Coombe Hospital in Dublin has been working in the hospital for the past four weeks.
Other changes included streamlining emergency paediatric services and better training around foetal monitoring.