She was seen wearing the hat on a safari in Nairobi National Park in Kenya.
While at the park, she helped feed elephants and also inspected them.
The white Pith helmet became popular for British officers in Africa during the 19th century.
Mrs Trump was seen wearing the hat whilst she was photographing zebras and giraffes on her iPhone.
The criticism was fierce on Twitter where one Nairobi resident Pauleen Walo wrote: “That pith helmet you have carried was used by colonialists during the dark days.
“Doesn’t sit well with us Africans.”
Kim Yi Donne, a professor of political science at the University of California, Riverside added: “When people think of Africa, they have these standard narratives.
“Her attire is a signal of her understanding of what Africa is in 2018. It’s tired and it’s old and it’s inaccurate.
On her tour of Africa, Mrs Trump has already visited Ghana and Malawi.
This weekend, she will be visiting Egypt on the next part of her journey.
There has been little detail about why the First Lady decided to make the trip.
But she has been promoting her ‘Be Best’ platform that focuses on children.
She also said she hopes to learn more about life in Africa.
On Thursday she was in Malawi taking part in the donation of one million books to help boost education levels.
Despite this, President Trump has repeatedly criticised foreign aid spending.
Mrs Trump’s trip to Africa was reportedly a decision she made herself which comes after President Trump allegedly described some African countries as “sh**holes”.