A man who says he was too drunk to remember knocking a woman unconscious before sexually assaulting her has been jailed for four years.