A security operation is continuing along the Dublin-Belfast railway line near Newry amid fears that it may have been the target of a mortar attack.