OpenWeatherMap 2017: Results of the Year

So the, new year, 2018 has come, and it's time to sum up the results of the past year for our company, OpenWeatherMap (UK, US and Latvia), developers of one of the best weather APIs in the world. 
A lot happened and much has changed. 
During the year, our number of users grew from 600,000 to 1 million. We participated in the Startup Grind Global Conference in Silicon Valley where the team from OpenWeatherMap was named in the top 50 Startup Exhibitions of 2017. Our mature team was filled up with excellent professionals and wonderful people. 
We did a lot of new things and qualitatively improved our current developments. 
In 2017, we were pleased to present to you:

Weather data: API and Weather Maps

• The Open Dashboard for Agricultural Monitoring 
   can help give you an idea of the possible use for meteorological and satellite data in your agricultural applications.
• Specially for the agriculture sector, we launched an API for accumulated temperature data and an API for accumulated precipitation data.
• The Weather Historical Bulk service. 
  Now you can simply choose a city (or several cities) and download an archive, which contains bulk file with the weather history up to 5 years - any day, week or even several years.
• A new and improved version of an API for UV-index
    
  Throughout the entire year, we worked constantly on our history weather API. For this year, the amount of data supplied and speed to process that data increased significantly. Also, we made it possible to quickly upload data in a format that doesn’t require additional processing and can be understood by any user. 
  Our Weather Maps app changed qualitatively. 
  This year, we added the ability to switch the layers of weather and satellite maps, create various combinations with them, and connect them to mobile and web apps.

Satellite data: Vane platform

This year, we significantly improved our satellite platform. Our team undertook a huge amount of work and in June were able to present  a new version  of the satellite image processing  platform VANE.

Based on the Vane platform, we developed a new product called  Global Satellite Base Map, which uses visual tools and query language to generate a map from satellite images. The uniqueness of the product is that all data processing is done on the fly, and there are no presets or pre-made calculations. The user defines the parameters for a calculation and image processing and immediately receives a result for any territory. This capability was only possible thanks to Vane, our super powerful data processing platform.

We recently presented you with Query Builder, our new interface for the Vane platform. Now you can use this simple tool to create your own map in just a few seconds, and with just one click receive a completed link for display on your site or app using a web map library like Leaflet, Open Layer, Mapbox and Google Map.

We are grateful to everyone that worked with us for all this time. We thank you all your feedback and for not getting bored by our tech support. 
We have a ton of plans for the coming year. Stay tuned, and you will see a lot of the new and interesting things to come. Subscribe to our Telegram Channel https://t.me/openweathermap and get news first about our updates and new products!

Cloudless: global cloudless composite coverage based on the VANE Platform

In the drawing is global coverage obtained between 06.01.2017 and 09.01.2017 using data from the MODIS spectroradiometer aboard KA Terra and Aqua.

The current cloudless coverage of the Earth by medium and low-resolution satellite images are an important element in the regional and global systems that monitor the territorial changes caused by natural and man-made factors. For example, assessing the damage inflicted by forest fires caused by deforestation, volcanic eruptions, flooding,etc. Also, such types of coverage are popular as the base layer for cartographic web services.

The main stages of creating such coverage are: the selection of images, expert as a rule, the masking of clouded areas, tonal adjustment of images taken at different times of the year, and pasting them into single coverage using so-called “cutlines” which enable, to a certain extent, the joins between the pasted images to be hidden. Such operations, as a rule, are carried out in semi-automatic mode and require specialized software and highly qualified experts, which substantially increases both the time taken to create such a product and its cost.

We have designed and delivered a completely different approach to producing such coverage. At the heart of this approach lie the principles used in processing Big Data and machine-learning algorithms. The technology we have implemented allows us to obtain, in a fully automatic mode, the very best coverage, in terms of cloud cover, of the Earth for a specified time frame. Not only this but the technology we use for data preprocessing means that a composite product can be obtained using either one or several different data sources. To obtain a composite image, we carry out a pixel by pixel analysis of all images stored in our database for a specified time frame. As a result, a single value, the best in terms of cloud cover, is selected. By assembling such values from there, complete coverage is produced. 
At the present time, the most popular, due to its high performance and general accessibility, is the data received from images taken by the Landsat-8 satellite and the Sentinel-2 pair of satellites. It is also worth noting that images from satellite data are taken in near spectral bands which, in turn, simplifies its simultaneous analysis.

In the drawings below are fragments of composite coverage collected from data from Landsat-8 and Sentinel-2, 2016-2017.