A highly-criticised undercover police unit was so secretive even Scotland Yard bosses were unaware that it existed, it has emerged.