Not even scientists can tell these birds apart. But now, computers can

By Erik StokstadJul. 28, 2020 , 12:50 PM

It’s a fact of life for birders that some species are fiendishly difficult to tell apart—in particular, the sparrows and drab songbirds dubbed “little brown jobs.” Distinguishing individuals is nearly impossible. Now, a computer program analyzing photos and videos has accomplished that feat. The advance promises to reveal new information on bird behaviors.

“We spend a lot of time with binoculars, hunkered down, staring at birds and their legs,” says Iris Levin, a behavioral ecologist at Kenyon College who was not involved in the new work. The reason: For years, researchers have identified birds by placing colored bands on their legs. They use those bands to identify birds in the wild—and in photographs and videos back in the lab. The task can often be laborious, Levin says.

Specially outfitted tags can make the job easier, by including GPS and proximity sensors that record when animals interact. Passive integrated transponder (PIT) tags, also used to prevent shoplifting and identify pets, ping linked antennas when a bird lands within a few centimeters. Behavioral ecologist Claire Doutrelant of CNRS, the French national research agency, and her colleagues have been adding these small tags to the leg rings of sociable weaver birds (Philetairus socius) since 2017.

Sociable weavers work together to construct large nests in southern Africa, often in acacia trees. The nests can weigh as much as 1 ton and house up to 200 birds in individual chambers. Their cooperative behaviors also include chick rearing and defense against snakes and falcons. To study these behaviors, the researchers identify and track hundreds of individual birds.

Antennas on feeders keep track of which birds are living in the colony. But more granular information—such as which birds contributed the most to communal activities—hasn’t been possible to get that way. And Doutrelant and her colleagues can’t place antennas all over the nest: The birds are wary of them, and their chambers are too close to one another for reliable data collection.

So team member André Ferreira, a Ph.D. student at the University of Montpellier, decided to try a kind of artificial intelligence. The tool, called a convolutional neural network, sifts through thousands of pictures to figure out which visual features can be used to classify a given image; it then uses that information to classify new images. Convolutional neural networks have already been used to identify various plant and animal species in the wild, including 48 kinds of African animals. They have even achieved a more complicated task for elephants and some primates: distinguishing between individuals of the same species.

Ferreira fed the neural network several thousand photos of 30 sociable weavers that had already been tagged. “No one had come up with an efficient method to collect these training data sets,” he says. To take the photos, he set up cameras near bird feeders equipped with radio-frequency antennas. As soon as the birds landed, a small computer recorded their identity using their PIT tag, and a camera snapped pictures of their backs every 2 seconds. (The rear view is the part of the bird seen most often while they are nesting or foraging.)

After just 2 weeks, Ferreira had enough photos to train the neural network. “We were not sure if it would work,” Doutrelant recalls. “We have observed these birds a lot, and we’ve never managed to recognize them without the color rings.” But when given photos it hadn’t seen before, the neural network correctly identified individual birds 90% of the time, they report this week in Methods in Ecology and Evolution. Doutrelant says that’s about the same accuracy as humans trying to spot color rings with binoculars.

Ferreira then tried the approach on two other bird species studied by Damien Farine, a behavioral ecologist at the Max Planck Institute of Animal Behavior. The tool was just as accurate in identifying zebra finches in captivity and great tits in the wild. Both species are widely studied by ecologists.

But Gail Patricelli, a behavioral ecologist at the University of California, Davis, sees some limits to the approach. For example, with species that are difficult to capture and tag, it could be hard to get the thousands of identifiable photographs needed to train the neural network. She studies the greater sage grouse, a species in decline, and she tries to avoid handling them because it stresses the birds. Another potential limitation: When birds molt, the neural network might not recognize them and would need to be retrained. Ferreria is collecting photos of other traits, such as aspect of the head, to improve the tool.

The biggest limitation with the current neural network, Ferreira says, is that it tries to identify every bird as one that it already knows, so it can’t recognize a new individual. Ferreira is now working with Farine to try a different kind of neural network that could do that—it would need to be trained on pictures of many more birds. If the data set were large enough, the tool could be used even by researchers who haven’t tagged their birds. “I think this would be a complete game changer,” Farine says.

Despite those limitations, Patricelli calls the new work “exciting,” and says it opens possibilities for studying many other bird species and behaviors. “The fact that this algorithm was able to tell them apart—when they look very similar to the naked eye—is definitely striking.”