Finance Minister Michael Noonan “did not want” to know about sex abuse allegations involving children while he was health minister in the 1990s, a former Fine Gael colleague has claimed.