RS Python

RS Python features an innovative workflow orchestration solution built around the python Prefect orchestration framework and the adoption of the SpatioTemporal Asset Catalog (STAC) concepts for internal data management. RS Python can be installed locally or scaled up on a cloud cluster as a multi-user data processing platform. It is designed to be securely integrated with the CSC Ground Segment Operational context.

With the RS Python solution, one can set up a platform to support Copernicus Ground Segment operation related activities such as processor validation and benchmarking, implementation and fine-tuning of data processing workflows, re-processing and production services, data quality investigations, integration of new processors and missions.

Reference System Python will support at first processing chains for Sentinel-1, Sentinel-2, and Sentinel-3 data. Looking ahead, we’re excited to expand our capabilities to include Sentinel-5P and other upcoming Copernicus missions.


Sentinel-1	Sentinel-2	Sentinel-3	Sentinel-5P

RS-Server Catalog

The rs-server offers a fine access control per Collection to the Catalog. The catalog provided by rs-server adheres to the SpatioTemporal Asset Catalog (STAC) format. By adopting a standardized format like STAC, rs-server aims to streamline the organization and querying of geospatial asset metadata. The term “spatiotemporal asset” encompasses any file that conveys information about Earth captured at a specific location and time.

Here is an example of STAC collection retrieved from RS catalog.

 {
    "stac_version": "1.0.0",
    "type": "Collection",
    "license": "XXX",
    "id": "20201211_223832_XXX",
    "description": "A simple collection example",
    "links": [],
    "extent": {},
    "summaries": {}
}

RS-Server staging

The rs-server offers staging functionality with access control for retrieving products from external data sources. The rs-server grants access to auxiliary data from the AUXIP station, as well as telemetry raw data from CADIP stations. It will also provide access to Sentinel level-0 products from LTA and Level-1 and Level-2 products from PRIP.

Processing workflow as a code

RS Python leverages the Prefect orchestrator for managing and automating workflows. Prefect simplifies workflow management, reduces operational costs, and provides a smoother experience for data orchestration compared to other tools. Its flexibility, quick adoption, and supportive community make it an excellent choice for RS’s workflow needs.

Virtual environment

RS Python provides access to JupyterLab to end-user. The end-user can build or start already made Prefect worflows from rs-client libraries. JupyterLab proposes a flexible workspace organization to users making it easier to work with multiple tabs simultaneously. Additionally, JupyterLab provides a comprehensive Markdown editor, enhancing the writing and documentation experience within notebooks. JupyterLab offers a richer and more efficient data programming experience, making it a wise choice for RS.

RS deployment

The rs-server can be deployed on several environments. On a Kubernetes cluster or on a local machine. From the local machine it is also possible to access both local Catalog and remote rs-server services.

Cluster mode

Here are CSC RS Python in a Nutshell features from the cluster.

Local mode

And here are CSC RS Python in a Nutshell features from the laptop.