Yandex.Meteum - a new development or marketing ploy?
In late November, Yandex announced its updated weather service called Yandex.Meteum. Allegedly, the new software development by its algorithms is able to calculate the forecast with accuracy to the house. As an amateur meteorologist, I could not but be interested in a new product. I have always respected Yandex, despite the unsuccessful restart of Kinopoisk, but having studied in detail the announcement published on the Habrahabr website , I found a number of inconsistencies and logical errors in it. Then I decided to conduct my research on the accuracy of the new service, relative to other weather resources, namely, my website Weather 45 (Weather Forecast for Kurgan) and Foreca (the basic resource from which Yandex takes data).
In the future, I will rely on an extended announcement published on Habrahabr. Let us examine these inconsistencies and logical inaccuracies that I found in this announcement.
')
Is there a need for such a forecast? Existing computer models use a grid of 20 by 20 kilometers or 15 by 15 kilometers. What does this mean? This means that each calculated node from the previous one is at a distance of 20 kilometers. As a rule, for flat terrain with small elevation differences this is sufficient. But in the conditions of the mountainous terrain, the help of mesoscale models is required, where the computational nodes are at a closer distance. Most often it is 5, 2 or 1 kilometer. But there are exceptions. For example, a mesoscale COSMO model with a 100-meter grid was developed specifically for the 2014 Sochi Olympics! Of course, such a model well calculates local phenomena inherent in a given area: fog, orographic precipitations, and sharp temperature drops. But as you can see, even here the grid is used with an accuracy of 100 meters. And it is justified. After all, an increase in the number of nodes leads to an increase in the load on the supercomputer on which the model is launched. But is home accuracy needed? In some cases, the air mass is so homogeneous that the temperature changes every 10, 20 or 50 kilometers by one degree! The forecast with accuracy to the house is an advertising duck, no more.
The minimum temperature in the Central Federal District at night on January 2
“More detailed information about the global state of the atmosphere comes to us from the American Global Forecast System, which is considered one of the most accurate global models in the world and has a resolution of 0.25 degrees.”
Another statement requiring clarification. The American computer model GFS is really good, but it is not the most accurate. The European ECMWF model is the most accurate model. On the US website weatherbell.com, graphs of the predicted baric field and actual field are shown, using two models for up to 5 days. For 2015, the European model has a compliance ratio of 0.910, while the American model has 0.885. The prediction of the pressure field is a basic parameter. If the model calculates it well, it means that it can adequately calculate the temperature, wind, cloudiness and precipitation.
Analysis of the quality of prediction of the pressure field of the American and European models.
So the moment of truth has come. Now a few words about the conditions of the experiment. I am not familiar with the criteria for assessing the accuracy of the forecast. On my website "Weather 45", I have been making my own forecasts for my city of Kurgan for three years. I use a multi-model ensemble method, the forecast is based on a dozen different models that are “included” at different times of the year and in different synoptic situations. In 2014, the forecast for one-day temperature was justified by 95%, and in 2015 it was possible to increase the accuracy by 1%! My forecast is 10% more accurate than our local weather forecasters, for whom 85% is the upper limit. Not a single service, not a single independent model has been able to make a forecast more precisely than mine. The multiple approach shows greater efficiency than the prediction of single models.
Every day throughout December I recorded the forecast of Yandex, Foreca, and my own. Prediction of maximum temperature during the day and minimum at night. This is a worldwide practice, there is nothing new here. Analyzed the percentage and average absolute error in degrees. To assess the accuracy, I used the hydrometeorological instruction. It is here .
The results of the experiment were expected for me. The forecast from Yandex was 2% more accurate than Foreca’s, and the average absolute error turned out to be almost equal. This means that these 2% of the ordinary user will not even feel. And, of course, Yandex could not surpass my forecasts published on the website "Weather 45". In addition, during my observation, I noticed that Yandex does not know how to predict night cooling, and this is the main problem in the winter season. Another problem point concerned the site itself and the application. From mid-December, Yandex began issuing the most inadequate forecasts that can be attributed to technical problems. For example, on New Year's Eve, I expected a cold snap to -30 on most models. And Yandex showed only -21. This is an unforgivable mistake! And 35% of the excellence, I will not even comment.
Glitch on the site Yandex. No cold snap. New Year's Eve was -33. Yandex shows -21.
Ultimately, we can say that Yandex did not create anything supernatural, an objective analysis of the forecast showed superiority over the Foreca site forecasts only within the margin of error, which means that ordinary users will not be able to feel this difference. And if there is no difference, why pay more?
In the future, I will rely on an extended announcement published on Habrahabr. Let us examine these inconsistencies and logical inaccuracies that I found in this announcement.
Debriefing
"The forecast is accurate to the house"')
Is there a need for such a forecast? Existing computer models use a grid of 20 by 20 kilometers or 15 by 15 kilometers. What does this mean? This means that each calculated node from the previous one is at a distance of 20 kilometers. As a rule, for flat terrain with small elevation differences this is sufficient. But in the conditions of the mountainous terrain, the help of mesoscale models is required, where the computational nodes are at a closer distance. Most often it is 5, 2 or 1 kilometer. But there are exceptions. For example, a mesoscale COSMO model with a 100-meter grid was developed specifically for the 2014 Sochi Olympics! Of course, such a model well calculates local phenomena inherent in a given area: fog, orographic precipitations, and sharp temperature drops. But as you can see, even here the grid is used with an accuracy of 100 meters. And it is justified. After all, an increase in the number of nodes leads to an increase in the load on the supercomputer on which the model is launched. But is home accuracy needed? In some cases, the air mass is so homogeneous that the temperature changes every 10, 20 or 50 kilometers by one degree! The forecast with accuracy to the house is an advertising duck, no more.
The minimum temperature in the Central Federal District at night on January 2
“More detailed information about the global state of the atmosphere comes to us from the American Global Forecast System, which is considered one of the most accurate global models in the world and has a resolution of 0.25 degrees.”
Another statement requiring clarification. The American computer model GFS is really good, but it is not the most accurate. The European ECMWF model is the most accurate model. On the US website weatherbell.com, graphs of the predicted baric field and actual field are shown, using two models for up to 5 days. For 2015, the European model has a compliance ratio of 0.910, while the American model has 0.885. The prediction of the pressure field is a basic parameter. If the model calculates it well, it means that it can adequately calculate the temperature, wind, cloudiness and precipitation.
Analysis of the quality of prediction of the pressure field of the American and European models.
Accuracy Assessment
So the moment of truth has come. Now a few words about the conditions of the experiment. I am not familiar with the criteria for assessing the accuracy of the forecast. On my website "Weather 45", I have been making my own forecasts for my city of Kurgan for three years. I use a multi-model ensemble method, the forecast is based on a dozen different models that are “included” at different times of the year and in different synoptic situations. In 2014, the forecast for one-day temperature was justified by 95%, and in 2015 it was possible to increase the accuracy by 1%! My forecast is 10% more accurate than our local weather forecasters, for whom 85% is the upper limit. Not a single service, not a single independent model has been able to make a forecast more precisely than mine. The multiple approach shows greater efficiency than the prediction of single models.
Every day throughout December I recorded the forecast of Yandex, Foreca, and my own. Prediction of maximum temperature during the day and minimum at night. This is a worldwide practice, there is nothing new here. Analyzed the percentage and average absolute error in degrees. To assess the accuracy, I used the hydrometeorological instruction. It is here .
Conclusion
The results of the experiment were expected for me. The forecast from Yandex was 2% more accurate than Foreca’s, and the average absolute error turned out to be almost equal. This means that these 2% of the ordinary user will not even feel. And, of course, Yandex could not surpass my forecasts published on the website "Weather 45". In addition, during my observation, I noticed that Yandex does not know how to predict night cooling, and this is the main problem in the winter season. Another problem point concerned the site itself and the application. From mid-December, Yandex began issuing the most inadequate forecasts that can be attributed to technical problems. For example, on New Year's Eve, I expected a cold snap to -30 on most models. And Yandex showed only -21. This is an unforgivable mistake! And 35% of the excellence, I will not even comment.
Glitch on the site Yandex. No cold snap. New Year's Eve was -33. Yandex shows -21.
Ultimately, we can say that Yandex did not create anything supernatural, an objective analysis of the forecast showed superiority over the Foreca site forecasts only within the margin of error, which means that ordinary users will not be able to feel this difference. And if there is no difference, why pay more?
Source: https://habr.com/ru/post/274641/
All Articles