When we think of “big data” in our modern world, a few examples come easily to mind: the copious amounts of textual information available on the web, economic indicators and the stock market, or maybe even medical images and records. In a world that is being rapidly reshaped by a revolution driven by Artificial Intelligence, these popular sources of information have often become the focal point of AI innovation. Yet in recent years, an emerging line of AI research, to which the general public has remained largely oblivious, has seen remarkable breakthroughs – weather forecasting.
The Current State of Weather Forecasting
The weather and its forecasting is an element of our lives that many overlook – a negligible backdrop to our day. Light jacket or raincoat? In reality, however, it is a cutting-edge industry fueled by the latest scientific and technological advancements.
Consider, for instance, the sheer logistics needed for a daily forecast in the National Weather Service (NWS): atmospheric data from weather balloons launched daily from over 92 locations, detailed imagery taken by weather satellites, and the collective effort from over 10,000 volunteers to gather additional ground data – just to name a few. This data is then sent to the NWS’ Weather and Climate Operational Supercomputer System (WCOSS), a supercomputer more than six million times as powerful as the average desktop pc. WCOSS then runs the data through the NWS’ numerical weather models in order to generate forecast guidance for specialized meteorologists, who use a combination of skill and experience in order to construct the forecast.
The drawbacks to this current system, however, are plenty. Conventional weather models are extremely energy-intensive and require hours to process, wasting precious time for scientists to formulate and communicate latest forecasts with areas facing severe weather. And the sheer complexity of the task takes a toll on the accuracy of conventional computer models – in fact, current 5-day forecasts are only about 80% accurate, while the accuracy for 10-day forecasts drops to 50%. When it comes to predicting the path of a major hurricane, accuracy and speed can be the difference between life and death.
The industry of weather forecasting, however, may just be on the brink of revolution.
GraphCast: A New Breakthrough
On November 14th 2023, DeepMind, a subsidiary AI laboratory of Google, unveiled a novel AI weather model known as GraphCast, as well as research comparing it with conventional models. And the results are astounding.
GraphCast was able to outperform the state-of-the-art model from the European Centre for Medium-Range Weather Forecasts (ECMWF) on more than 90% of the over 1,300 overall test areas. When only considering the troposphere, which has the largest impact on weather that affects us, it came on top in 99% of weather variables, such as precipitation and temperature. And it was able to accomplish this higher standard while requiring less than a minute to run, a far cry from the hours that traditional supercomputers would need for the task.
How was GraphCast able to achieve such a breakthrough? The answer lies in the nature and design of the model itself. Instead of using conventional hard-coded physics-based equations, GraphCast utilizes a neural network – essentially a digital mapping of intelligent systems such as the human brain – in order to represent millions of different interconnected points on the surface of the earth. Then, through training itself with over four decades of historical weather data, the neural network is able to establish the often unseen connections between variables. Once training is complete, the model is then able to quickly draw conclusions about a new set of data.
GraphCast’s big moment came early this September. With the onset of Hurricane Lee; state-of-the-art weather models were initially unable to accurately predict the exact position of landfall – in fact, they were only able to pinpoint Nova Scotia, which would be proven correct, six days in advance. GraphCast was able to do so nine days in advance. While this may not appear to be a substantial difference, any additional time that can help evacuation efforts and local communities and governments to prepare can save lives and property.
As these recent successes have unfolded, GraphCast is being rapidly deployed in practicality. In fact, the ECMWF has already begun to make use of it.
Peter Dueben, head of Earth System Modeling at ECMWF, commends DeepMind for its creation and believes that it may be a turning point in weather forecasting. “It showed that these models are so good that we cannot avoid them anymore,” he says.
Limitations and the Future of Forecasting
GraphCast, however, is still a far cry from flawless. Perhaps one of its biggest weaknesses lies in predicting short-term precipitation, where it is outmatched by conventional models. And, similar to many other AI models, GraphCast is a “black box” – researchers and meteorologists are unable to fully understand what the model is actually doing, or what the intermediate results mean. This raises questions about their reliability, as it will be difficult for scientists to identify mistakes in the results. Matthew Chantry, a machine-learning coordinator at the ECMWF, predicts that “it will be another two to five years before people can use forecasting from machine-learning approaches to make decisions in the real world.”
Nonetheless, GraphCast has been generally perceived as a big step towards the new future for the industry of weather forecasting. DeepMind is also making GraphCast open source, which will further catalyze novel research and potential breakthroughs in AI forecasting.
Since the early days of gazing at the stars to predict the weather, we’ve doubtlessly come a long way – but in a modern world further complicated by the onset of climate change, AI might just be the next step towards unveiling the secrets in the skies above us.
