3 upvotes, 3 direct replies (showing 3)
View submission: Ask Anything Wednesday - Engineering, Mathematics, Computer Science
with the huge amount of data that we have about the weather in the past several decades, how can't we make an AI that learns from it and gives accurate forecast for the next 2 days at least?
Comment by chilidoggo at 26/06/2024 at 18:11 UTC
8 upvotes, 1 direct replies
To add to what the other guy said, weather is notoriously difficult to predict. It was actually one of the things that led to the development of Chaos Theory, since even a small change in inputs would radically change their prediction models.
Comment by ron_leflore at 26/06/2024 at 17:43 UTC
13 upvotes, 0 direct replies
People are working on that. Here's for just the SF Bay area: https://sf.atmo.ai/[1] by https://www.atmo.ai/[2]
There was a big paper by Google Deepmind about this idea last year, look up GraphCast.
You should see big changes in weather prediction in the next few years as these ideas make their way into products.
Comment by functor7 at 27/06/2024 at 17:47 UTC
4 upvotes, 0 direct replies
I wouldn't put too much stock into machine learning weather predictors. Weather prediction works by having a mathematical model of how the atmosphere works, and then working with current measurements and their errors, runs it forward with slightly with randomized initial conditions. From what meteorologists say, this is relatively reliable on the 1-4 day timescale, a little finicky at the 5-7 day timescale, and 10-day predictions are not reliable.
Now, when making predictions you are using all of the information that you have available now in order to make an assessment about what is happening going forward. So the real question should be: How much information about future weather is contained in the measurements we're able to make today? This would be the theoretical limit for what we can predict.
The important thing here is that we know how weather works mathematically, and so we use the math as a way to evolve the weather. It is able to use what we give to push things forward. The interesting thing about this is that it works with truly novel situations. If there are weather formations that have not been seen before (something we can expect going forward with Climate Change), then the mathematical model does not care that it is new and can make predictions just as well as it can for mundane weather. With math being the rules for how weather evolves, we're able to optimize that information limit.
This is the opposite for how machine learning would work. Machine learning does not learn the "rules" for how weather evolves. It cannot know how weather works. It can only make predictions through pattern recognition which means that it cannot work with novel information well. If the measurements today are novel, it will still try to place it in a pattern it already recognizes, which is bad for prediction. Moreover, weather is chaotic and so the amount of information contained inside the raw data is relatively low - mathematical simulations help keep predictions on track but pattern recognition doesn't have any guard rails to stabilize its accuracy. This is a poor level informational efficiency. And this is a problem for all machine learning, it produces what is expected and what is typical and does not follow any meaningful logical structure by design. AI isn't a magic box that "just works", it can work for certain things and we need to be better at understanding how it works so that we can be better at discerning where and how it can be used. (A bit so that articles proclaiming the magic of AI don't trick us into misplacing our trust with it.)
As for weather predictions, we think that they're bad because we only notice when it's wrong and when it being wrong impacts us. Every day is a datapoint, but you are only going to be collecting your own personal anecdotal datapoints when it is off. Which biases our opinion about it. Weather predicting is much better than we think, and the best way to improve it even more would be to make more and better measurements, and large/faster computers that can compute these models with more fidelity.