Gardaí are monitoring a prominent Real IRA family as part of a large-scale operation to prevent retaliation for the murder of terror leader Alan Ryan.