A TV licence inspector has described an incident where he was assaulted on duty as the most frightening situation of his 13-year career.