A French judge has placed under investigation the photographer and publishing company behind unauthorised topless photos of the Duchess of Cambridge.