Placards by the group spoke of 3,000 homeless children in Ireland and the need to house local people first.
The protest began directly opposite the Holy Cross Mercy primary school before moving to Linden House, a former guesthouse which has hosted asylum seekers in the past.
A Garda from the nearby garda station was on duty at the entrance.
At the gates of the house local councillor Donal Grady addressed the gathering to rounds of applause, before handing in a petition, stating the concerns, signed by those protesting.
“Firstly to make clear we believe that people from war torn areas like Iraq and Syria should be accepted in our country,” Mr Grady read before outlining concerns about the lack of consultation with the community.
“We have no knowledge of the vetting procedures these people have been through and they are being housed across the road from a local school,” he said.
"The town had a large number of homeless and they should be looked after first," he continued.
Angry voices in the group which gathered outside Linden House referred to the fact some of the men who arrived on Friday were carrying iPhone sixes; and a lot of them smoked.
Others shouted about the Irish people paying for the accommodation.
“What fire officer signed off on safety for that house for 55 people,” a mother of three who identified herself as Karen asked.
Pat Sullivan, who is involved with the jarveys in Killarney, asked why the Department of Justice had hidden the fact they were re-opening what was a vacant and derelict building.
Work took place at night under lights, two local women said. Most people said they objected to the lack of notice to anybody in Killarney by the Department of Justice.
“It’s not healthy having fifty five men with nothing to do all day,” another parent said.
Meanwhile three of the men who arrived at Linden House spoke briefly to the media.
Others remained inside but opened the door to where a Christmas tree stood to accept the letter of protest.
A 25-year-old man from Albania told this remoprters he had been in Ireland with four years, in Galway and the west waiting for his application to be processed.
“Give me work and I don’t stay here,” the man said. The accommodation in Killarney was "very good," he said, when asked about the standard of the building which had been vacant.
It was the first time he had come across such a protest and he asked not to be identified in photographs. A second man also from Albania said he had been in Ireland with two months.
A 48-year-old man from India said he was from the Punjab and was seeking asylum because of the war there, he had been housed first in Donegal and was in Ireland two months.
There was a brief angry confrontation between one woman and the Albanian man when she asked him where his family was; others remarked how the asylum seekers had “brilliant English” however, Cllr Grady moved the woman away.
Meanwhile, Mr Grady, who organised the protest, had previously called for the building, which is privately owned, to be leased by the council and used for local homeless persons.
Killarney already hosted 120 asylum seekers in two State owned properties, he explained.
“We have people coming in and we don’t know the first thing about them. Killarney has its own problems. I can pick out 40 people in this town who won’t have any warmth tonight,” he said.
“We stress it is not that we don’t wish refugees to be accommodated in Killarney but we believe it should be done under consultation with the local community and when we are in a position to help the nationwide plan after we have accommodate local families who have been either homeless or years on the housing list,” Mr Grady said on Saturday.
The Department of Justice said it is expected that the applicants arriving in Killarney will be from different countries.
“There has been an increase in applications for international protection in Ireland. The latest figures (as of end November 2017) suggest that 2,620 persons applied for international protection this year compared with 1,982 in the same period last year."
There has been strong criticism of the Department of Justice by public representatives because of lack of consultation with locals or with the county council and other bodies.
Michael Healy-Rae received confirmation earlier this week of the arrival of the refugees. He said he has been inundated with calls from locals concerned about pressure on the housing lists and about the proximity to local primary schools.
“We have questions that need answering. Surely someone in the Department of Justice could have contacted the county council,” Mr Healy-Rae said.
His brother Danny raised the matter in the Dáil.