A man who received a five-year sentence for harassing his former girlfriend has had an attempt to overturn his conviction rejected by the Court of Criminal Appeal.