A Fine Gael backbencher and member of an influential internal party group has insisted Frank Flannery should not return until the Rehab controversy is addressed.