A 51-year-old man has been jailed for five years more than two decades after he was first arrested for repeated sex assaults on a six-year-old girl.