A victim of modern slavery in England has told of how he was "afraid to go out on the street" when his family was being exploited.