Artificial Intelligence: Right Whale Photo Identification
We apply deep learning algorithms to photographs of North Atlantic right whales to automate the process of matching individuals to the photo identification catalog.
Right Whale Photo Identification
Managing the recovery of endangered species relies on estimating population abundance and monitoring trends over time. It is impossible to simply count wild animals; a common method for estimating abundance is mark-recapture using photo identification. Monitoring wild populations through photo identification allows us to detect changes in abundance that inform effective conservation.
Researchers take photographs from aircraft and vessels and match individuals to the North Atlantic Right Whale Catalog. The long‐term nature of this data set allows for a nuanced understanding of demographics, social structure, reproductive rates, individual movement patterns, genetics, health, and causes of death. However, photo identification is time‐consuming. Recent advances in machine learning, and deep learning in particular, have paved the way to automate image processing using neural networks modeled on the human brain. Harnessing this new technology could revolutionize the speed at which these images can be matched to known individuals.
Artificial Intelligence (AI) is the concept of computers performing tasks that normally require human intelligence. The term “machine learning” was coined in 1959 by Arthur Samuel and is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns, and make decisions with minimal human intervention. Machine learning is a way to achieve AI. Neural networks are an established family of machine‐learning models, inspired by the human brain, that can handle many layers that send and receive inputs and are capable of learning. Several recent breakthroughs and spectacular applications have made neural networks, and in particular convolutional neural networks (CNN), the model of choice for image processing. As the number of layers in the state‐of‐the‐art CNNs increased, the term “deep learning” was coined to denote training a neural network with many layers.
Kaggle Data Science Competition
NOAA hosted a data science challenge on Kaggle to automate identification of North Atlantic right whales from photographs. We were motivated by the recent advances in computer vision, combined with the power of crowdsourcing a wide variety of approaches to tackle this complex problem. Collaboration with the winning team from Deepsense.ai resulted in a publication in Conservation Biology to share lessons learned.
Flukebook for Aerial Photos
NOAA teamed up with Wild Me to deploy the Deepsense algorithm on their Flukebook platform in 2019. They used a model newly trained on the full dataset of North Atlantic right whale photographs provided by the North Atlantic Right Whale Consortium.
Algorithms are Expanded Worldwide
The Flukebook system was expanded to Southern right whales in April 2020, trained with whale ID catalogs from South Africa, New Zealand, Australia, Argentina, Brazil, and Chile. As expected, accuracy for the Southern right whales was considerably lower than for North Atlantic right whales. Future research can improve the algorithms so that the Southern right whale model can more closely approach the accuracy of the North Atlantic right whale model.
Flukebook for Vessel Photos
Many researchers also study right whales from sea-going vessels, where they can obtain information such as genetic samples, fecal samples, and conduct health assessments. To automate the identification of right whales from vessel photographs of the whales' heads, NOAA collaborated with Wild Me to adapt the Pose Invariant Embeddings algorithm developed by doctoral student Olga Moskvyak at Queensland University of Technology. This first attempt to automatically match right whales from vessel-based photos of the callosity pattern on the head has achieved a high degree of accuracy. This innovative technique has been deployed on the Flukebook platform for both North Atlantic and Southern right whales.
Publications and Reports
- Khan, C., Blount, D., Parham, J. et al. 2022. Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration. Mammalian Biology. https://doi.org/10.1007/s42991-022-00253-3
- Blount, D., Gero, S., Van Oast, J. et al. 2022. Flukebook: an open-source AI platform for cetacean photo identification. Mammalian Biology. https://doi.org/10.1007/s42991-021-00221-3
- Bogucki R, Cygan M, Khan CB, Klimek M, Milczek JK, and Mucha M. 2019. Applying deep learning to right whale photo identification. Conservation Biology, 33: 676-684. https://doi.org/10.1111/cobi.13226
- Angliss R, Sweeney K, Moreland E, Hou B, Richmond E, Sanderson B, Robinson S, Lynn M, Martinez T, and Khan C. 2020. 2019 Image Processing Workshop report, Seattle, WA
- Khan C, Holmberg J, Hamilton P, Pettis H, Blount D, Parham J, Mucha M, Charlton C, Rowntree V, Vermeulen E, Rayment W, Dawson S, Johnston D, Groch K. 2020. From Competition to Collaboration: Automated Identification of Right Whales. Report to the International Whaling Commission, 2020.
- Blount D , Minton G, Khan C, Levenson J, Parham J, Holmberg J. 2020. Flukebook – Continuing growth and technical advancement for cetacean photo identification and data archiving, including automated fin, fluke, and body matching. Report to the International Whaling Commission, 2020.
Awards
2020 Gears of Government Award through the Executive Office of the President. This Government-wide program is centered on recognizing outstanding contributions toward mission results, customer service, and accountable stewardship.
2019 NOAA Bronze Medal Award. The highest honor award granted by the Under Secretary of Commerce for Oceans and Atmosphere, Bronze Medals recognize federal employees for superior performance.
2017 Semifinalist in the Innovations in American Government Awards competition by the Ash Center for Democratic Governance and Innovation at the John F. Kennedy School of Government, Harvard University.