Three of seven Muslims detained over a suspected plot to assassinate a Swedish cartoonist have been released from custody.