This document introduces DolphinGemma, a specialized large language model from Google DeepMind designed to analyze dolphin audio. Developed in collaboration with the Wild Dolphin Project and the Georgia Institute of Technology, the model's primary goal is to decipher the structure and patterns within dolphin vocalizations. By training on a unique, long-term dataset of audio and video from a specific wild dolphin community, DolphinGemma learns to process sequences of dolphin sounds, identify recurring patterns, and generate novel, dolphin-like audio. The ultimate ambition of the project is to uncover the potential meanings behind dolphin communication and, in the future, establish a foundation for two-way interaction. Although currently in development, DolphinGemma is slated to be an open model, providing a powerful new tool for the broader research community studying intelligent marine life.
Unlocking Dolphin Communication with DolphinGemma
Project Overview: DolphinGemma is a large language model created by Google DeepMind in partnership with the Wild Dolphin Project and the Georgia Institute of Technology. Its purpose is to use AI to study and understand dolphin communication, with the aspirational goal of one day enabling two-way communication.
Core Capabilities: The model functions as an "audio-in, audio-out" system. It processes sequences of natural dolphin sounds to identify recurring patterns, structures, and clusters, aiming to predict subsequent sounds in a sequence much like a text-based LLM predicts the next word.
Unique Training Data: DolphinGemma is trained on a unique dataset gathered over decades by the Wild Dolphin Project. This dataset consists of underwater audio and video of a community of wild Atlantic spotted dolphins, with data cataloged by individual dolphin and their specific behaviors.
Technical Approach: The model learns the underlying structure of dolphin vocalizations to generate new, dolphin-like sound sequences. Progress and output are visualized using spectrograms, which show sound frequencies over time for vocalizations like whistles and burst pulses.
Availability for Research: DolphinGemma is currently in development. Upon its release, it will be made openly available to the research community to serve as a powerful tool for collectively deepening the scientific understanding of intelligent marine life.
Part of the Gemma Ecosystem: DolphinGemma is a specialized variant within Google's broader Gemma family of open models. This family includes other task-specific models such as MedGemma (medicine), CodeGemma (coding), and PaliGemma 2 (vision-language).