The victims had accused the Chilean Bishop, Juan Barros, who the Pope had appointed, of covering up crimes committed by a fellow priest.
The Pope described their accusations as a slander and said there was no evidence, which sparked a national uproar in Chile and infuriated the families of the victims.
The 81-year-old pope from Argentina said: “The day they bring me proof against Bishop Barros, then I’ll speak.
“There is not one shred of proof against him. It’s all calumny. Is that clear?”
In a rare apology the pope said he realised his words had hurt many people, but said he believed that the Chilean Bishop Juan Barros was innocent.
He made the comments when he spoke to journalist on a plane flying back to Rome.
The Pope said: ”I apologise to them if I hurt them without realising it, but it was a wound that I inflicted without meaning to.
“It pains me very much.”
The Archbishop of Boston, Cardinal Sean O’Malley, said the Pope’s words were “a source of great pain for survivors of sexual abuse”.
He said: “Words that convey the message ‘if you cannot prove your claims then you will not be believed’ abandon those who have suffered reprehensible criminal violations of their human dignity and relegate survivors to discredited exile”.
Anne Barrett Doyle of the online abuse database BishopAccountability.org said: “That is the enigma of Pope Francis.
“He is so bold and compassionate on many issues but he is an old school defensive bishop when it comes to the sex abuse crisis.”
The Pope ended his Latin America tour with a giant mass for more than a million people in the Peruvian capital of Lima.
Earlier in his visit to Peru, Pope Francis became the first pope to visit the Amazon basin in more than 30 years.
Speaking to thousands of members of indigenous communities in the Madre de Dios region, he defended their way of life and spoke out against environmental degradation.