Mrs Merkel must bring together the Greens, the liberal Free Democrats (FDP) and her own conservative bloc to secure a majority after a poor performance in the recent election.
If negotiations collapse then a new national vote would be called.
The question of migrants threatens to deepen the deadlock as the parties cannot agree on the issue.
The divide between the parties widens as the debate on migrants continues to focus on setting a formal cap and the conditions for reuniting families.
The Greens, which is soft on refugee and asylum policies, has scheduled a party congress on November 25 to vote on whether to push on with formal coalition talks.
If the preliminary talks don’t result in substantial progress, a vote may not even take place and the coalition will collapse.
Alexander Dobrindt, a negotiator for the Bavarian Christian Social Union (CSU), said on Saturday: “The question of whether it’s going to be possible to form a government remains wide open.”
Some 158,000 people who were denied asylum were still living in Germany, new figures published by the German government revealed.
More than 115,000 asylum seekers had received a delay on deportation notices for reasons such as illness or political instability back home – but 28,000 denied asylum seekers are counted in the new statistics as potential “enforced deportations”.
Members of Mrs Merkel’s party have called for increased deportations, under pressure from their Bavarian sister party the CSU and the far-right Alternative for Germany (AfD).
Peter Altmaier, Mrs Merkel’s chief of staff and acting finance minister, indicated that the Christian Democrat Union of Germany (CDU) intends to “intensify” its efforts to deport those rejected for asylum. The FDP also support a tough line.
In May, a massive truck bomb in Kabul killed more than 150 people, causing Berlin to suspend all flights until October, when a plane carrying 14 denied asylum seekers left Leipzig for Afghanistan. Berlin suspended the flights earlier this year
Critics, led by the Greens, argue Afghanistan is not safe enough for the government to send even the criminal offenders back.
Greens leader Katrin Göring-Eckardt called the practice “irresponsible,” saying Afghanistan “will always be unsafe.”
And in an interview on Friday with German newspaper Bild, the CDU’s Mr Altmaier said it will be difficult for the four would-be coalition partners to find agreement on the issue of deportations.
Mr Altmaier said: “In principle we agree: No one wants to change the fundamental right to asylum. But in the details, it will be difficult.”