On a screen at the headquarters of the European Centre for Medium-Range Weather Forecasts (ECMWF), in Reading, a series of unassuming pixels move across a map of the United States. First yellow, then green or blue, they form a tadpole as they fly over the southern states of Texas and Louisiana.
These pixels represent Hurricane Beryl, which tore through the southern United States last week – and was a major breakthrough in weather forecasting, a breakthrough that earned the team behind it the “Nobel Prize for engineering,” the MacRobert Award, on the same day Beryl struck.
Hurricane Beryl is one of dozens of extreme weather events accurately predicted by GraphCast, a weather forecasting system developed by British artificial intelligence (AI) company Google DeepMind. The program beats the highest resolution (HRES) system, the current gold standard in weather forecasting, at more than 90 percent of its accuracy measures.
The team behind GraphCast, led by scientist Rémi Lam, were described as “heroes” by Prof Sir Richard Friend when he presented them with the prize last week. Also in the running for the prize were the University of Oxford and AstraZeneca, for designing the manufacturing process that enabled the rapid rollout of vaccines during the pandemic, saving an estimated six million lives.
It’s easy to see why DeepMind’s feat is considered more heroic than even this: As extreme weather events become more common around the world, meteorologists are in a race against time to develop software that can accurately predict disasters before they happen and help governments evacuate people from danger zones. Other well-known tech companies like Nvidia and Huawei have also developed their own models, but GraphCast is considered the most advanced.
Weather forecasting is all about the ‘butterfly effect’: how very small things in the initial situation you observe can compound and make the weather unpredictable in the long term, explains Remi Lam.
Traditional methods involved “physical models” – complex mathematical equations first performed by humans and later, as now, by bus-sized supercomputers. GraphCast, by contrast, can run on a laptop and produce a weather forecast in minutes.
The model works by making a prediction based on data it has been given about weather events over the past decade, and applying it to millions of grid points around the globe, without any math involved. It is being trialled by the ECMWF in Reading, along with other experts around the world, who have approached it with a “healthy dose of scepticism”, says Lam.
“We live in a world that is getting warmer and we are seeing more extreme events, so we really need more accurate measurements,” Lam says. For this reason, DeepMind has made its model open source, allowing government organizations and other companies around the world to use and build upon its work for their own purposes.
The first time GraphCast was used in real life at the ECMWF, to predict the movements of Hurricane Lee over the eastern United States last September, the team behind it was “blown away,” Lam says. Machine learning, the technique used to produce GraphCast, is a “very analytical” and tedious way of working, Lam says. “You’re always looking at decreasing error, and you see that margin going down as you train a model.”
But visualizing it as a cyclone on the road was “the most striking example of what GraphCast can do,” Lam says.
“We’re not weather experts and most of us don’t have a background in forecasting,” he adds. “So the first time we saw a real cyclone, moving in real time, everyone thought, wow.”
But even in June 2022, less than a year after DeepMind’s first attempts at mid-range weather forecasting began, the team knew they were onto something “really promising.” And in 2023, in the case of Hurricane Lee, the model was able to predict “where the hurricane would make landfall, in Nova Scotia.”
“And we were sure it would be locked [to that path] “Six days before it happened, three days earlier than the traditional approach could predict,” Lam says.
In an age where we all have 24-hour access to weather forecasts on our phones, as well as on TV and in newspapers, it’s easy to forget just how complicated it all is. People have been trying to predict rain, sunshine and the movement of clouds for over 2,000 years, ever since 300 B.C., when philosophers in ancient Greece, China and India devised their own ways to track cloud patterns and predict how every tiny change in the weather might unfold.
It’s no wonder then that experts like Carlos Osuna of MeteoSwiss, the Swiss Federal Office of Meteorology and Climatology, are amazed by the rapid new developments in this field. In recent years, there has been an “explosion” in the use of machine learning to predict the weather, Osuna says, after years of stagnation in the capabilities of traditional forecasting.
State agencies like MeteoSwiss have “reached the limit of computational cost,” Osuna says, and are unable to build supercomputers large enough to perform all the calculations needed to make more accurate forecasts. This current system is “expensive and resource-intensive,” he explains. Machine learning models, on the other hand, are cheap to run and can be run on computers like the ones many of us have at home, all made possible by huge advances in AI capabilities.
Currently, MeteoSwiss uses various types of machine learning to predict pollen levels in Switzerland, and to predict “nowcast,” or rainfall within a very short time frame. This has long been a tough nut to crack in weather forecasting, and DeepMind has made progress here as well. In 2021, the company’s technology could predict the likelihood of rain within a two-hour window with a high degree of accuracy, outperforming similar tools 89 percent of the time, according to 50 meteorologists at the U.K. Met Office.
Although the Met Office no longer uses GraphCast to make predictions, the organisation has been using machine learning in various forms for “decades”, says Prof Kirstine Dale, its chief AI officer. She is working with the Alan Turing Institute to develop FastNet, the Met Office’s own machine learning model that will “complement” other available models, such as GraphCast.
Like GraphCast, FastNet will be widely used to generate “medium-range” forecasts up to two weeks in advance, many of which will be used for government briefings in the future. But the Met Office is also working on ways to use AI to help predict long-term weather issues, on a “sub-seasonal” or “sub-tropical” scale – the sort of thing that would help the country prepare for floods, droughts or other disasters long before most people can see the warning signs.
Dale says that such mapping, of long- and short-term weather events, will always go hand in hand with physical models when it comes to official advice. “Any machine learning model is only as good as the data it’s trained on,” she explains. “If there was an event like a volcanic eruption, where there was a lot of aerosol in the air, you would want to rely on a physics-based model because that would be an out-of-sample event that the machine learning model wouldn’t be able to predict,” she says.
Carlos Osuna from Switzerland agrees. “Machine learning models don’t understand physics or mathematics, so they can make stupid mistakes,” he explains. It’s the classic “black box” problem with generative AI, which also makes chatbots susceptible to “hallucinations” or making up facts, a potential pitfall in weather forecasting.
“But it’s fantastic at capturing all the patterns that came before it, so it produces a really good result for things like cyclones or complex thunderstorms, which are traditionally very difficult to predict with just mathematics.”
‘Most chaotic system in the world’
According to Alex Levy, CEO of Atmo, a California-based AI weather forecasting company, the limitations of machine learning predictions are more a problem with human understanding of the natural world than with the machine learning models themselves.
Such models are trained on all the data ever generated by older mathematical systems, a problem because “weather is by far the most chaotic system in the world,” Levy says. “Somewhere along the line in the prediction process, there are always going to be laws of physics that we’re going to run up against, beyond which it’s impossible to know how things are going to turn out.”
This year, Atmo signed a contract with the U.S. Air Force to forecast wind patterns at Cape Canaveral, one of the primary U.S. space launch sites. The company also recently began working with the Philippine national government to forecast weather conditions down to the “city block” level in the capital, Manila.
“The Philippines experiences one of the highest number of typhoons per year of any country in the world, 10 to 20, and at the same time, Manila has a huge population that is larger than many other countries,” Levy said.
Hampered by the limit on physical computing power, “the old way of forecasting was not very accurate and it was difficult to say which parts of Manila would be hit by extreme weather.” Such precision is crucial when it comes to “emergency response, which can be poorly managed without it,” Levy says.
Similarly, old physics-based methods had to be general, following universal laws of physics, while new AI models can be very specific in their predictions. In Manila, typhoons are the extreme weather phenomenon that poses the greatest risk to life, but in other environments the dangers can be radically different.
While we may never be able to say with absolute certainty what the weather will hold for us tomorrow or next weekend, the days when countries can be struck by an unexpected catastrophe – such as in October 1987, when weatherman Michael Fish infamously told the nation not to prepare for a hurricane that actually hit – may be becoming a thing of the past.
After all, machine learning systems are constantly improving themselves, Levy says. So in all but “the most strange and unusual events,” it’s now much easier to prepare for the future in ways that forecasters never thought possible.