Whose weather forecast is more accurate?
Many people think from time to time - how accurate are the weather forecasts? Whose weather forecast is more accurate? In which direction are most often wrong predictions for my city?
Mathematics Adventure Time!
The last few years, Yandex has been actively developing its weather service. A couple of years ago, Metium technology and precipitation forecast were presented to the nearest minute.
')
In the discussion of these publications stand out comments fantast8 . The amateur meteorologist, in a rather aggressive form, pounced on criticizing Yandex technologies and, if not paying attention to the communication style, the gist of the complaints was reduced to the lack of texture and secrecy of statistics.
And then I wondered: does anyone even consider the accuracy of forecasts?
A quick googling project was issued by ForecastAdvisor , however, forecasts were evaluated there exclusively for US cities, and, of course, there were no Russian-speaking providers (Yandex, Hydromet, etc.).
The project MyCli.me, now closed, existed in runet for a while. However, he had persistent problems with data collection and the rating system (choosing the best forecast) was not based on the scientific method.
In addition, I found many undertakings, for example, from OLS and lucius , but all of them were limited either by time, or by city, or by provider. For example, one of the most amazing projects from slimyhorror analyzes the predictability of forecasts for more than 5000 days in a row! But only forecasts from the BBC and only for cities in the UK are evaluated.
Thus, it was decided to make your project. Things are easy - to collect forecasts, to collect the actual weather and to evaluate the sales on the basis of a scientifically based formula. Let's start with it.
In 2009, the Hydrometeorological Center of Russia issued "Instructions for short-term forecasts." According to this guide, the forecast is considered fulfilled if it differs from the actual observed temperature by no more than 3 ° C in any direction. If the forecast differs from the actual weather by more than 3 °, then this forecast is considered unfulfilled (instruction point 7.3.1.4).
Weather forecasts should be assessed separately for day and night (7.2.3). The justification of the daily forecast is determined as the average of the values of the prediction of the night and the day. The assessment of the success of weather forecasts for a calendar period (month, season, year, etc.) consists in calculating their average accuracy for the corresponding period (9.1.1).
Okay, it’s not difficult - we take the forecast for tomorrow (for the day after tomorrow, etc.), collect data on the actual weather, compare it separately for day and night. If a
then the forecast did not come true (Pt = 0%), if <= 3 then Pt = 100%. We consider the average accuracy for the corresponding period and obtain the necessary data.
From parsing sites with the weather immediately refused, because HTML parsing is not stable and this would inevitably increase the number of errors in the final statistics. Fortunately, many providers provide an API to access their predictions. In the first stage, I added forecasts from accuweather.com, darksky.net, openweathermap.org and wunderground.com. All of them provide JSON-API. At the second stage, the Norwegian yr.no with XML-API, the Russian Hydrometeorological Center (RSS had to be parsed) and Yandex.Pogoda were added.
Today there is an agreement with foreca.com and negotiations are underway with gismeteo.ru, intellicast.com and English metoffice. If you have friends in these projects who are ready to help with the API, write!
In theory, there shouldn't have been any problems with collecting actual weather. Every major city has a weather station, usually in the center of the city, for example in Moscow it is the Exhibition of Economic Achievements . National Hydrometeorological Services transmit data from these stations to the World Weather Watch , where it is processed and stored. Observations are transmitted in the form of a SYNOP code every 3 or 6 hours.
In addition, every major airport has a weather station, which every hour (or half an hour) transmits an aviation weather report in METAR format.
Raw SYNOP and METAR data can be found on the NOAA website; processed data on the site ogimet.com. The stations themselves are conveniently searched on weatherobs.com.
However, there are several problems. First, the quality of the transmitted data - there are often typos in reports (for example, the letter O instead of the number 0). Secondly, data stability. In 2017, Aviamettelecom stopped transferring weather reports from more than fifty Russian airfields to the international exchange for a whole month. Thirdly, there are cities in which I simply could not find a working station. For example, in Rostov-on-Don there is no data on either station 34730 (URRR), or station 34731.
In such situations, I use information from the private network of The Weather Company stations (now owned by IBM) to collect actual weather.
The result was the site Meteotrust.com
Currently, data is collected for 10 cities of Russia from 7 providers. In the near future I plan to add more cities, more providers and more periods (long-term forecasts from hydrometcenter and accuweather, forecasts for 10 days). In addition, a collection of precipitation statistics is being prepared. It is slightly more difficult to assess the accuracy of precipitation forecasts than temperature. This will be a separate post.
Openweathermap - bottom.
Accuweather, yr.no, darksky - strong middling.
Yandex and Wunderground have excellent predictions.
The weather center often fights with Yandex for first place in the quality of short-term forecasts. However, there is an assumption that with an increase in the number of cities, especially foreign ones, the overall success of the forecasts of the Hydrometeorological Center will fall.
Mathematics Adventure Time!
The last few years, Yandex has been actively developing its weather service. A couple of years ago, Metium technology and precipitation forecast were presented to the nearest minute.
')
In the discussion of these publications stand out comments fantast8 . The amateur meteorologist, in a rather aggressive form, pounced on criticizing Yandex technologies and, if not paying attention to the communication style, the gist of the complaints was reduced to the lack of texture and secrecy of statistics.
And then I wondered: does anyone even consider the accuracy of forecasts?
Fire kingdom
A quick googling project was issued by ForecastAdvisor , however, forecasts were evaluated there exclusively for US cities, and, of course, there were no Russian-speaking providers (Yandex, Hydromet, etc.).
The project MyCli.me, now closed, existed in runet for a while. However, he had persistent problems with data collection and the rating system (choosing the best forecast) was not based on the scientific method.
In addition, I found many undertakings, for example, from OLS and lucius , but all of them were limited either by time, or by city, or by provider. For example, one of the most amazing projects from slimyhorror analyzes the predictability of forecasts for more than 5000 days in a row! But only forecasts from the BBC and only for cities in the UK are evaluated.
Thus, it was decided to make your project. Things are easy - to collect forecasts, to collect the actual weather and to evaluate the sales on the basis of a scientifically based formula. Let's start with it.
Banana guard
In 2009, the Hydrometeorological Center of Russia issued "Instructions for short-term forecasts." According to this guide, the forecast is considered fulfilled if it differs from the actual observed temperature by no more than 3 ° C in any direction. If the forecast differs from the actual weather by more than 3 °, then this forecast is considered unfulfilled (instruction point 7.3.1.4).
Read more about the gradation.
Theoretically, the provider could predict the weather from -60 to +60 every day and thus get one hundred percent sales. However, the correct gradation of the forecast is considered to be a deviation of no more than 1 degree, for example, “Daytime temperature from 2 ° to 4 °”. Thus, a deviation of 2 degrees from the interval or 3 degrees from the value is allowed.
Weather forecasts should be assessed separately for day and night (7.2.3). The justification of the daily forecast is determined as the average of the values of the prediction of the night and the day. The assessment of the success of weather forecasts for a calendar period (month, season, year, etc.) consists in calculating their average accuracy for the corresponding period (9.1.1).
Okay, it’s not difficult - we take the forecast for tomorrow (for the day after tomorrow, etc.), collect data on the actual weather, compare it separately for day and night. If a
Math.abs(predict - actual) > 3 then the forecast did not come true (Pt = 0%), if <= 3 then Pt = 100%. We consider the average accuracy for the corresponding period and obtain the necessary data.
Dark cloud
From parsing sites with the weather immediately refused, because HTML parsing is not stable and this would inevitably increase the number of errors in the final statistics. Fortunately, many providers provide an API to access their predictions. In the first stage, I added forecasts from accuweather.com, darksky.net, openweathermap.org and wunderground.com. All of them provide JSON-API. At the second stage, the Norwegian yr.no with XML-API, the Russian Hydrometeorological Center (RSS had to be parsed) and Yandex.Pogoda were added.
Today there is an agreement with foreca.com and negotiations are underway with gismeteo.ru, intellicast.com and English metoffice. If you have friends in these projects who are ready to help with the API, write!
Increased oddity
In theory, there shouldn't have been any problems with collecting actual weather. Every major city has a weather station, usually in the center of the city, for example in Moscow it is the Exhibition of Economic Achievements . National Hydrometeorological Services transmit data from these stations to the World Weather Watch , where it is processed and stored. Observations are transmitted in the form of a SYNOP code every 3 or 6 hours.
In addition, every major airport has a weather station, which every hour (or half an hour) transmits an aviation weather report in METAR format.
Raw SYNOP and METAR data can be found on the NOAA website; processed data on the site ogimet.com. The stations themselves are conveniently searched on weatherobs.com.
However, there are several problems. First, the quality of the transmitted data - there are often typos in reports (for example, the letter O instead of the number 0). Secondly, data stability. In 2017, Aviamettelecom stopped transferring weather reports from more than fifty Russian airfields to the international exchange for a whole month. Thirdly, there are cities in which I simply could not find a working station. For example, in Rostov-on-Don there is no data on either station 34730 (URRR), or station 34731.
In such situations, I use information from the private network of The Weather Company stations (now owned by IBM) to collect actual weather.
Science taxis
The result was the site Meteotrust.com
Currently, data is collected for 10 cities of Russia from 7 providers. In the near future I plan to add more cities, more providers and more periods (long-term forecasts from hydrometcenter and accuweather, forecasts for 10 days). In addition, a collection of precipitation statistics is being prepared. It is slightly more difficult to assess the accuracy of precipitation forecasts than temperature. This will be a separate post.
What are the conclusions?
Openweathermap - bottom.
Accuweather, yr.no, darksky - strong middling.
Yandex and Wunderground have excellent predictions.
The weather center often fights with Yandex for first place in the quality of short-term forecasts. However, there is an assumption that with an increase in the number of cities, especially foreign ones, the overall success of the forecasts of the Hydrometeorological Center will fall.
Source: https://habr.com/ru/post/354160/
All Articles