That is the view of his former coach and American tennis legend Andre Agassi.
Djokovic moved to third in the all-time list in January when he swept to a record seventh victory at the Australian Open.
In the final of the tournament, the Serbian beat Rafael Nadal 6-3 6-2 6-3 to win a 15th Grand Slam title.
It was his third major victory in a row after success at Wimbledon and the US Open last year.
And in May, Djokovic will begin his campaign to hold all four titles at the same time and achieve a historic ’Novak Slam’.
Djokovic is just two Grand Slam wins behind Nadal on 17 and five away from Federer on 20.
Federer has shown some signs of decline since winning the Australian Open last year.
And his best chance of adding to his tally is likely to come at Wimbledon.
But by deciding to play on the clay this season, Federer has put himself in contention for Roland Garros.
In his 11 month spell as Djokovic’s coach, Agassi was unable to help the Serbian add to his Grand Slam haul, as injuries hampered him.
However, when asked if he could see the world No 1 taking Federer’s record, the American had little doubt.
He told the Times of India: “I mean I am from Vegas, so I have to make a bet right, if I have to make a bet, what is he at? 15?
“So two more for three years? Yes. Now, you also have children, you also have priorities in life, but if he wants to, he will.”
Speaking after his dominant win in Melbourne last month, Djokovic said he was motivated by the pursuit of Federer’s record.
“I’m aware that making history in the sport I truly love is something special,” he said. “Of course it motivates me.
“Playing Grand Slams, the biggest ATP events, is my utmost priority in this season and in seasons to come.
“How many seasons are to come? I don’t know. I’m not trying to think too much in advance.
“I do want to definitely focus myself on continuing to improve my game and maintaining the overall wellbeing that I have – mental, physical, emotional – so I would be able to compete at such a high level for the years to come.”