Australian troops are under investigation for allegedly cutting off the hands of at least one dead militant in Afghanistan, it is reported.