System architecture¶

This page describes the architecture of the RS-Python system as a whole. The architecture of each component is described in its own repository, e.g. rs-server or rs-dpr-service.

Main components¶

Main components

Layer	Components	Repositories
RS-Python	Web services (frontend, catalog, external data sources, staging, OSAM), STAC browsers	rs-server, rs-helm, rs-server-deployment
	Python libraries and Prefect flows	rs-client-libraries, rs-workflow-deployment
	DPR service and processors	rs-dpr-service, rs-workflow-env (Docker images of the processors)
	Virtual environment: JupyterHub, Prefect, Dask Gateway, API Key Manager	rs-workflow-env
Infrastructure	Core: Kubernetes, Envoy Gateway, cert-manager, Keycloak, CloudNativePG, Velero...	rs-infra-core
	Monitoring: Grafana, Prometheus, Loki, Tempo, Alloy	rs-infra-monitoring
	Security: Wazuh, NeuVector	non-public repository
Physical infrastructure	Cloud provider (OpenStack virtual machines, S3 object storage), provisioned with Terraform and Ansible	rs-infra-core

The detailed list of the third-party components and their licenses is given in the Third-party components annexes.

Collaboration between the components¶

Functional overview

The users interact with the system through JupyterHub, the Prefect dashboard, the STAC browsers and the Swagger page. They are all authenticated by Keycloak.
The Prefect flows implement the processing chains. They call the web services with the rs-client-libraries, authenticated with the API key of the user.
All the web services are exposed by Envoy Gateway behind a single domain name, and check the rights of the user for each request.
The external data sources services translate the STAC requests into OData requests to the Copernicus stations.
The staging and the processing services run their tasks on Dask clusters, and write the products in the object storage.
The catalog stores the metadata of the products in a PostgreSQL database (pgstac).

The Python modules depend on each other as follows:

Python modules

Each kind of data follows a lifecycle, from its staging or generation to its deletion, described in Catalog data model and lifecycle.

Deployment¶

The deployment on a Kubernetes cluster, in several layers and node pools, is described in Deploy RS-Python. The local mode, used for the development, is described in Local mode.

Local mode and cluster mode

Actors¶

Actors

All the actors are RS users, with specific access rights: end users, operators, administrators and security officers. The external data sources provide the raw data and auxiliary data needed by the processing chains. The DPR processors and the settings of the Dask clusters that run them are delivered by the EOPF project. Two targets are supported to deploy RS-Python: a local machine and a Kubernetes cluster.