A man who claims he was groped by a now retired surgeon has admitted he considered withdrawing his complaint but denies there was no basis to it.