The EUXDAT services are deployed on top of a DIAS platform. That is a fact, but what is a DIAS and why do we need it? DIAS has been a hot topic the last few years for developers and service providers in all domains more or less related with earth observation. But the key concept and the reason why this program was carried out by the European public authorities is not always fully understood. Let’s take a leap in the past and try to understand how all of this began.
It all started in the early 2000s with a decision from European Commission to develop their own means for monitoring and observation of our environment. It led among others things with the launching of the Sentinel earth observation satellites (first one Sentinel-1A was launched on 2014, followed by Sentinel-1B in 2016, 2A and 2B in 2015 and 201, Sentinel-3A and 3B in 2016 and 2018 and finally Sentinel 5P in 2017). These satellites holding efficient and modern remote sensing technologies can examine our planet in all aspects, not only in the visible spectrum but also in larger spectral bands (infrared for instance) or through radar wavelengths.
The data generated by this observation means is of course a goldmine for scientists and environment research labs, but the European Commission strongly desires that the private sector takes also possession of it to develop new use cases and new businesses. Hence, it was necessary to make it accessible to everyone. And the task was not as trivial as you might think. A constellation like Sentinel 2 generates around 1 petabytes of raw data per year, 3 terabytes per day… And we will not even talk here about the different processing required to get ready-to-use data (orthorectification, atmospheric correction…), which multiplies even more the amount of data to store.
ESA has set up the “Sentinel Data Hub” portal (now called Copernicus Open Data Hub) since 2014 to allow all those who wish to freely download Sentinel product. The technical limitations of such a system were quickly reached. To work on an image, a user must download a whole Sentinel product through internet (for example with Sentinel 2, from 600 to 800 megabytes for a square of 100km per 100km), must store the product on its proper means, and finally must process it to extract the valuable information. Setting up kind of “real time” processing chain was not possible due to the limited bandwidth of internet, and the cost to develop ambitious use cases on extended areas and periods was prohibitive due to the required on-site storage and processing infrastructure.
To mitigate these limitations, the commission and ESA decided around 2017 to follow a last decade trend: cloud and edge computing. The basic idea is to run processing close to the data instead of moving the data to the processing. To do so, Copernicus data should be made available on a European cloud open to everyone. Users, instead of downloading data, will deploy their applications on this cloud and run their processing close to the data. No need any more to transfer petabytes of data on internet, no need to duplicate data all over the world, everything is centralized and optimized on this cloud. As you might have already guessed, this move to cloud program of Copernicus data has a name: DIAS, for Data and Information Access Service.
Rather than allocate the setup and operation of this platform to a single contractor, the commission and ESA decided to fund for 4 years the ramp up of 5 platforms, with the idea that free competition and market reality will do their work so that in the end only sustainable solutions will remain. These DIAS platforms have been in service since 2018.
Thus, Europe has now at its disposal a modern and efficient system of Copernicus data dissemination. But DIAS platforms are only the foundations on which should be built a global ecosystem of applications exploiting Copernicus data.
Thematic platforms focused on domains as diverse as environmental protection, agriculture monitoring or natural hazards and disaster management are emerging. As we said earlier, EUXDAT is part of this trend. The EUXDAT consortium chose Mundi, the DIAS platform operated by Atos, to access Sentinel 2 images and host its applications.
On top of these DIAS platforms, processing solutions are also being developed to provide users with a higher level of abstraction of the manipulated data and thus make easier to develop use cases without technical expertise of the earth observation domain. We can mention here the Sentinel Hub initiative, or the Data Cube Facility Service program launched by ESA. Everything suggests that this story is just the beginning…
Technical Leader at Atos (Toulouse, France) and Technical Coordinator of the EUXDAT project