Royal Bank of Scotland is to pay $1.1bn (€979m) to a US regulator to settle two claims over mis-sold mortgage bonds in the run-up to the financial crisis.