A right-of-centre conservative group have hit out at "Stalinesque" criticism of Young Fine Gael president Killian Foley-Walsh's attendance at a hard-line US conference amid calls for him to be removed from his party role.