Mr Rajoy’s conservative Popular Party won the most votes in an unprecedented repeat election in June but fell short of a majority in parliament and needs other parties’ support to take power.
Pedro Sanchez, leader of the centre-left Socialist Party which came second in the ballot, said after meeting with Mr Rajoy that he cannot endorse Popular Party policies that cut public services and welfare payments.
“We’re an alternative to the Popular Party. We can’t support what we want to change,” Mr Sanchez told reporters after the meeting.
“I think it is possible - necessary - to reach some kind of understanding,” Mr Rajoy told reporters.
He added: “If the Socialists keep saying no, we’ll be holding new elections.”
Spain has never had a coalition government.