Talking to whales: can AI bridge the chasm between our consciousness and other animals?

T om Mustill was kayaking with his friend Charlotte in Monterey Bay, California, when an animal three times the size of the largest Tyrannosaurus Rex hurtled from the water and crashed down on their tiny craft. As the flying humpback whale fell upon them and their kayak was sucked beneath the waves, Mustill assumed he would die. Miraculously he and Charlotte found themselves gasping for breath, clinging to their capsized kayak.

How had they survived a smash with a creature three times the weight of a double-decker bus? I couldn’t figure out how the whale hadn’t smashed us to bits Tom Mustill What happened next was almost as weird. Mustill and Charlotte went viral. Passing whale-watching tourists had videoed the pair’s near-death encounter and stuck it on YouTube .

Mustill, a wildlife filmmaker, became what he calls “a lightning conductor for whale fanatics”. Interviewed by the global media, he was soon quivering with different and extraordinary stories of whale meetings from around the world: a submariner told him about whales singing to his ship; a book publisher reported being apparently scanned by the sonar-like echolocation of a pregnant female dolphin – a few days later, she discovered that she too was pregnant. “It was really addictive finding out all these other stories,” says Mustill, “because each one was like another lens on the animal and our relationship to them.

” These stories alone could fill a book, but Mustill first made a BBC documentary about humpback whales, before writing his book, How to Speak Whale , which is a thrilling exploration of past, present and future scientific endeavours to communicate with animals and better understand cetaceans in particular. What begins with questions about his own brief encounter soon plumbs profound scientific and philosophical depths. ‘We have now identified almost every whale in the Pacific’: a sperm whale dips below the surface.

Photograph: Gerard Soury/Getty Images As Mustill explains when I meet him beside a watery realm – a reservoir close to his home in east London – his wondering about how he survived became a bigger question. Professor Joy Reidenberg, a whale scientist, told him the footage suggested the whale veered away from Mustill’s kayak mid-breach, as if it didn’t want to hit them. “It made sense because I couldn’t figure out how it hadn’t smashed us to bits,” he says.

“More spiritual friends said, ‘Ah well, the whale didn’t want to hurt you. ’ I felt it was more like walking into a cellar at night, hearing a rat squeak and not wanting to tread on it – it’s not necessarily out of compassion. The whale might have thought, ‘Urgh, what’s that?’” Did the whale mean to spare Mustill? “You can’t just ask a whale,” said Reidenberg.

But perhaps we will soon. “This is the beginning of augmented biology,” he says, “where our human deficiencies – what we can’t sense, where we can’t go, what we actually have the time to find patterns in – all seem to be falling down. ” We’re at a moment in time, he argues, comparable to Antonie van Leeuwenhoek’s mid-17th-century invention of the modern microscope and microbiology.

Today, big data and machine learning could probe an impenetrable frontier – the chasm between our consciousness and those of other animals. Can we communicate with whales? If so, what will we say? And what will they say back? The history of human relations with whales is mostly bloody and exploitative, but Mustill argues that science and technology helped change it for the better. One of many scientific heroes in his book is American researcher Roger Payne .

In 1967, when commercial whaling was at its peak, Payne received recordings of whale sounds from the US navy, whose underwater listening stations were eavesdropping on Soviet submarines. Payne was haunted by the beauty of the sounds, and by the fact that they repeated themselves. His 1971 Science paper on whale “song” was a blockbuster; Payne also released albums of humpback whale song, which moved millions of people.

His science – and the power of song – chimed with the nascent environmental movement and Save the Whales became a sound of the 70s. Whale hunting was banned in US waters in 1972 and a decade later came a global moratorium on commercial whaling. Nevertheless, scientific attempts to communicate with animals are also fraught with gimmicks, eccentrics – the researcher who injected LSD into one of his study dolphins discredited the field for years – and heated debates over whether animal communication can ever be “language”.

Mustill believes these old struggles will be ended by new technology. After graduating in natural sciences at Cambridge University, he began his own scientific career by taking a fieldwork post in Mauritius, where he was tasked with monitoring the pink pigeon, working for Carl Jones , an inspirational biologist who defied scientific orthodoxies to captive-breed species on the brink of joining the dodo, saving them from extinction. Jones is a hero, but Mustill’s fieldwork was ill-fated – there was a cyclone and the pigeon pairs he watched failed to rear any young.

Mustill concluded he could do more for conservation by becoming a filmmaker. Today, he’s excited that new technology is vastly improving the efficiency of conservation fieldwork. Tiny audio recorders are used to detect rare birdsong in Hawaiian forests, for instance.

“The machine never gets distracted. It’s much better than me at doing that job, which is a bit galling. ” Fluke shot: Tom Mustill films a tail slap.

Photograph: Ru Mahoney Computers flicking through vast reams of biological data learn to recognise patterns that would take humans centuries to detect. Recognition programmes are now widespread in popular apps that identify plant species or birdsong. Mustill discovered the power of big nature data when he met Ted Cheeseman , founder of the Happywhale website, which collects people’s whale snaps to identify individual animals.

When Cheeseman replaced the laborious human study of each whale tail, or fluke, with an algorithm, they exponentially increased the number of flukes they could identify. “They have now identified almost every whale in the Pacific, which would once have been a pipe dream for any team of biologists,” says Mustill. Cheeseman also helped him discover the individual whale that may have spared his life: it was named Prime Suspect.

Recognising individual whales is one thing, but Mustill then met Aza Raskin and Britt Selvitelle, two Silicon Valley entrepreneurs leading efforts to communicate with animals via the Earth Species Project (ESP), a not-for-profit mission billed as Google Translate for whales. AI successfully translates human languages; ESP’s AI experts backed by a multimillion-dollar budget are working on other species. “ESP is looking at every technological bottleneck across all animal communication and trying to design solutions that everybody can use,” says Mustill.

They are creating tools, not new information, but just after Mustill handed his book into his publisher, Roger Payne – still championing whale research aged 87 – rang him at 11pm on Christmas Eve with some new facts. “He was like, ‘I’m really sorry to say your book’s not finished. ’ I’m so glad he did call because he loops back into the book like a human boomerang.

” Payne led Mustill to the Cetacean Translation Initiative (CETI), a supergroup of scientists with an awesome target: to communicate with a whale well enough to exchange ideas and experiences. By 2026. Led by marine biologist David Gruber , CETI is throwing everything at a well-studied population of sperm whales off the island of Dominica: multiple underwater listening stations; drones carrying hydrophones; whales tagged by drones; soft robotic fish swimming among the whales gathering audio and video.

Will they converse with a whale by 2026? “Everything that David Gruber has done before he has nailed,” says Mustill. “It’s going to be the biggest animal behaviour data set ever recorded. The voyage of [Darwin’s] Beagle didn’t just require loads of specimen cases and somebody who could capture these species, it needed people back home ready to catalogue, compare and preserve these specimens.

The data version of that is data centres, formatting, and they’re making it open source so other people can do it. ” ‘More spiritual friends said, “Ah well, the whale didn’t want to hurt you”’: Tom Mustill. Photograph: Sam Mansfield There’s a long history of scientific breakthroughs used for ill.

If we begin conversing with other animals, it’s easy to imagine them being manipulated: pigeons could carry diseases to enemies or migratory turtles instructed to deliver drugs to a distant shore. But Mustill is heartened by the fact that both ESP and CETI are run on open-source principles – their data and tools are free for others to use. “That’s both a way of fostering collaboration and allowing scrutiny, because one of the only protections against exploitation is being open,” he says.

For all the fears of abuse, when – and if – we learn to communicate with other animals, it seems likely to trigger profound changes in inter-species relationships. Selvitelle, says Mustill, has described ESP as “a machine for making vegans”. Imagine subtitles from footage of abattoirs.

Animal rights will be revolutionised if animals can advocate for themselves. “In the history of people being mean to less-powerful people, who controls the story, whose voice is heard and who is considered to have a voice is one of the key things that allows manipulation,” says Mustill. Of course, if we can hear animals, we might not like what they have to say.

Facial-recognition apps translating what our pets are “saying” is an obvious commercial innovation, but what if they reveal that our pets hold us in contempt? Mustill sees conversations with whales as potentially comparable to missionaries meeting indigenous people. “We unwittingly transfer things aside from good vibes when we make contact with previously separate worlds. If sperm whales talk to each other and transmit information that shapes their culture and actions, and we’re ready to speak to them, are they ready to be spoken to?” Mustill remains convinced that, if possible, conversations with animals will engender new human respect and, potentially, new consciousness.

It would certainly become less comfortable sitting on a sofa made from animal skins if those beasts could speak. But will we listen? Pleas from indigenous Amazonians to halt the destruction of the natural world fall on deaf ears in the west. “Industrialised western society hasn’t listened to them, but some of us have, and ideas from those cultures – such as the idea that a river can be alive – changes how you look at a river,” says Mustill.

Suzanne Simard , the professor who discovered trees’ subterranean exchanges and communications via fungal networks, was recently asked what she would ask a tree if they could talk. “What do you think of us?” she replied. What would Mustill ask a whale? “‘What do you think of us?’ would be really interesting from their perspective because they’d sense us in such a different way, but I’d also be interested in ‘How are you?’ Because the answer to that question would reveal both what is important to them and whether they have a sense of the individual,” he says.

“One of the biggest problems we have is individualism and the feeling that we’re supposed to get as much out of our lives as we can. Perhaps other social animals offer us more collective ways of looking at our lifespan and relationship to the world. ” Will we ever be able to speak whale? In this extract from his book, Tom Mustill reveals how AI will help What if you could design a mission to record a data set of whale communications perfectly optimised for the latest machine-learning and language-processing tools to scan? What if you could capture not just whole conversations but hundreds of thousands of them, from scores of different whales totalling millions, perhaps billions, of vocalisation units? Would you then have a chance at speaking whale? This is the plan of the Cetacean Translation Initiative, or CETI.

CETI is an interdisciplinary A‑team of badass scientists: marine robotics specialists, cetacean biologists, AI wizards, linguistics and cryptography experts and data specialists. They were all brought together at a meeting of academics at Harvard in 2019, which was chaired by David Gruber. Gruber is a marine biologist and inventor, crafting cameras that can capture the glow of sea turtles and soft, robot graspers to gently handle fragile deep-sea animals.

The team is huge, with scholars from Imperial College, MIT, Harvard and other universities and help from among others Twitter, Google and Amazon. Their goal, Gruber told me, was: ‘To learn how to communicate with a whale well enough to exchange ideas and experiences’. CETI’s plan is to throw everything they’ve got at the population of sperm whales off the island of Dominica in the Caribbean.

CETI will rig the seafloor with multiple listening stations. They will cover a 12. 5‑mile radius and form the Core Whale Listening station, recording 24 hours a day.

Alongside will be drones and ‘soft robotic fish’ equipped with audio and video recording equipment, able to move among the whales without disturbing them. CETI hopes to place tags on mothers, grandmothers, teenagers and great bull males from different pods. There will be weather sensors and other contextual data, and they will link vocalisations to behaviour and what they know of each individual whale: was it hungry, fishing, pregnant, or mating? All of these data will be available for the open-source community, so that everyone can get stuck in.

Then the AIs will really be unleashed. They will analyse the coda click patterns that whales use to communicate, distinguishing between those of different clans and individuals. They will seek the building blocks of the communication system.

By listening to baby whales learn to speak, the machines and the humans guiding them will themselves learn to speak whale. All of the machine-learning tools will be part of an attempt to build a working model of the sperm whale communication system. To test this system, they will build sperm whale chatbots.

To gauge if their language models are correct, researchers will test whether they can correctly predict what a whale might say next, based on their knowledge of who the whale is, its conversation history and its behaviours. Researchers will then test these with playback experiments to see whether the whales respond as the scientists expect when played whale-speak. Finally, they will try to speak, back and forth, with the whales.

What do they expect to say? I asked David. ‘The important thing to me,’ he said, ‘is to exhibit that we care and we are listening. To show the other beautiful life-forms that we see them.

’ How To Speak Whale by Tom Mustill is published by William Collins at £20. Buy a copy for £17. 40 from guardianbookshop.

com.