THE first groups of the two million Pakistanis who fled a military offensive against the Taliban in Swat Valley began trickling back home yesterday, but many families refused to leave refugee camps, demanding promised government assistance and citing security concerns.