Seal Classification using AI
Overview
A project in partnership with the Cornwall Seal Research Trust to classify seals based on their markings through the Microsoft AI for Earth initiative.
The aim was to enable the public and charity volunteers to submit photos of seals seen throughout Cornwall, and use our trained AI models to identify the seal based on the fur patterns.
Beyond the Digital worked closely with Cornwall Seal Research Trust to understand the existing process, propose, and implement a revolutionary solution saving time and effort for the charity on a cloud platform.
How we used AI models to classify the seals
Using Azure AI Custom Vision we were able to build multiple AI models chained together in a pipeline to identify a particular seal.
Based on a given photograph, we built an AI model to determine the orientation of the seal, and further models to identify a particular seal based on that orientation.
A percentage probability score was then returned to the user; with details about the seal and other photos of it for comparison. These details were then saved in the backend system to be verified by a Cornwall Seal Research Trust volunteer.
Technical Implementation
Using a micro-service architecture, numerous APIs were built in Node.js and Python to manage the different parts of the system whilst keeping it loosely coupled and secure.
Front-end web applications (using React) allowed Cornwall Seal Research Trust volunteers to manage the internal image catalogue and verify sightings submitted by the public. The mobile app (using React and Cordova) allowed the public to upload images of seals and get a returned prediction.
Mobile application
The proof-of-concept mobile application allowed the public to upload an image from their camera whilst out and about in Cornwall and get an instant seal prediction based on our pre-trained AI models.
Outcome
Through our collaboration with the Cornwall Seal Research Trust, Beyond the Digital provided a solution that automised the cataloging of over 1500 seal images and realised the vision to use AI to reduce the hours spent by volunteers to cross-reference and validate seal sightings.