The CETI Quest for Whale intelligence

Project CETI is a global research initiative partnering with Google, Amazon, and AI experts from Harvard and Oxford. Its mission is to interpret and understand sperm whale communications.

In support of this mission, vast quantities of field research data (recordings of whale clicks, video, underwater telemetry, and observational metadata) are collected and filed for analysis by machine learning scientists.

The challenge was that this data comes in all shapes and sizes and from an ever-evolving set of devices (tags, cameras, drones, gliders, hydrophones). What CETI needed was an adaptive iPad application that could capture all this data, keep up to date with advances in device technology, and also handle the harsh environment of multi-vessel data collection at sea.

60-Day Results

In two releases over 60 days, the Modu team delivered:

• Full discovery and functional analysis of a complex cross-functional domain.


• An approved App Store iPad app, written in Flutter, to capture data at sea when online or offline.


• The ability to create and extend field research data collection capability.


• A cloud-based API in C# to ensure app consistency and to safely store field trip data for later analysis.
  Exported data sets ready for downstream ingestion into ML models.

• Github repositories, docker images running in AWS ECS behind a load balancer and a postgres database in RDS.

The Pathfinding Approach

• We began with a short targeted discovery exercise to fully understand all the various domains, from the intricacies of data collection on board multiple vessels to the needs of AI scientists and researchers.


• It was from this work the idea of modelling a ‘Universal Timeline’ of whale activity arose and also the deeper understanding of how the needs for device metadata collection changes over time.


• To keep costs low, we deployed a small team to build the app and the cloud infrastructure led by a product manager to identify both the minimum requirements for launch and also innovation opportunities along the way.


• We field-tested the application ourselves on land to prove its robustness and met with CETI marine biologists regularly to get the app onto the boats and gather feedback in real-world use.


• The entire running platform was released and documented so that academic researchers and volunteers might then be able to contribute to its continued success.

Outcomes

• Project CETI, having previously been reliant on third-party technology providers, now has an extensible tool they own to support field trips.


• The data quality and consistency have been vastly improved, and multiple boats can now track whale activity concurrently.


• The hope is that, one day, enough patterns in the clicks and behaviour of these incredible animals emerge that will point the way towards an understanding of their language, making sperm whales the first non-human species we are able to truly understand.