Skip to content

rs_workflows/pi_db_models.md

<< Back to index

Tables for the performance indicator database

Base

Bases: DeclarativeBase

Set Base as a type, thus mypy understands it. Otherwise, error from mypy

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
31
32
33
34
35
36
class Base(DeclarativeBase):  # pylint: disable=too-few-public-methods
    """
    Set Base as a type, thus mypy understands it. Otherwise, error from mypy
    """

    pass  # pylint: disable=unnecessary-pass

FlowRun

Bases: Base

The table flow_run brings together Prefect flow general information.

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
class FlowRun(Base):  # pylint: disable=too-few-public-methods
    """
    The table flow_run brings together Prefect flow general information.
    """

    __tablename__ = "flow_run"

    id = Column(BigInteger, primary_key=True, autoincrement=True)
    flow_type = Column(Text)
    mission = Column(Text)
    prefect_flow_id = Column(UUID(as_uuid=True))
    prefect_flow_parent_id = Column(UUID(as_uuid=True))
    dask_version = Column(Text)
    python_version = Column(Text)
    dpr_processor_name = Column(Text)
    dpr_processor_version = Column(Text)
    dpr_processor_unit = Column(Text)
    dpr_processing_input_stac_items = Column(JSONB)
    dpr_processing_start_datetime = Column(TIMESTAMP)
    dpr_processing_stop_datetime = Column(TIMESTAMP)
    dpr_processing_status = Column(Text)
    excluded_from_pi = Column(Boolean, default=False)

    products_expected = relationship("ProductExpected", back_populates="flow_run", cascade="all, delete")
    products_realised = relationship("ProductRealised", back_populates="flow_run", cascade="all, delete")
    products_missing = relationship("ProductMissing", back_populates="flow_run", cascade="all, delete")

PiCategory

Bases: Base

The table pi_category provides the list of PI per mission.

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
class PiCategory(Base):  # pylint: disable=too-few-public-methods
    """
    The table pi_category provides the list of PI per mission.
    """

    __tablename__ = "pi_category"

    id = Column(Integer, primary_key=True, autoincrement=True)
    mission = Column(Text)
    name = Column(Text)
    description = Column(Text)
    max_delay_seconds = Column(BigInteger)

    products_expected = relationship("ProductExpected", back_populates="pi_category")
    products_realised = relationship("ProductRealised", back_populates="pi_category")
    products_missing = relationship("ProductMissing", back_populates="pi_category")

ProductExpected

Bases: Base

This table provides all expected product from a single flow_run. This is the difficult part of the PI computing: estimate the production. The computing depends on mission, level and more generally on every STAC ITEM provided as input to the DPR processor.

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
class ProductExpected(Base):  # pylint: disable=too-few-public-methods
    """
    This table provides all expected product from a single flow_run.
    This is the difficult part of the PI computing: estimate the production.
    The computing depends on mission, level and more generally on every STAC ITEM
    provided as input to the DPR processor.
    """

    __tablename__ = "product_expected"

    id = Column(BigInteger, primary_key=True, autoincrement=True)
    flow_run_id = Column(BigInteger, ForeignKey("flow_run.id", ondelete="CASCADE", onupdate="CASCADE"))
    pi_category_id = Column(Integer, ForeignKey("pi_category.id"))
    eopf_type = Column(Text)
    sensing_start_datetime = Column(TIMESTAMP)
    min_count = Column(Integer)
    max_count = Column(Integer)

    __table_args__ = (
        CheckConstraint("min_count >= 0"),
        CheckConstraint("max_count >= min_count"),
    )

    flow_run = relationship("FlowRun", back_populates="products_expected")
    pi_category = relationship("PiCategory", back_populates="products_expected")

ProductMissing

Bases: Base

The table product_missing provides the list of missing_products. The list is based on what has been estimated during the construction of the table product_estimated.

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
class ProductMissing(Base):  # pylint: disable=too-few-public-methods
    """
    The table product_missing provides the list of missing_products.
    The list is based on what has been estimated during the construction of the table product_estimated.
    """

    __tablename__ = "product_missing"

    id = Column(BigInteger, primary_key=True, autoincrement=True)
    flow_run_id = Column(BigInteger, ForeignKey("flow_run.id", ondelete="CASCADE", onupdate="CASCADE"))
    pi_category_id = Column(Integer, ForeignKey("pi_category.id"))
    eopf_type = Column(Text)
    sensing_start_datetime = Column(TIMESTAMP)
    count = Column(Integer)

    __table_args__ = (CheckConstraint("count >= 0"),)

    flow_run = relationship("FlowRun", back_populates="products_missing")
    pi_category = relationship("PiCategory", back_populates="products_missing")

ProductRealised

Bases: Base

The table product_realised provides the list of products generated by the DPR processor.

Source code in docs/rs-client-libraries/rs_workflows/pi_db_models.py 
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
class ProductRealised(Base):  # pylint: disable=too-few-public-methods
    """
    The table product_realised provides the list of products generated by the DPR processor.
    """

    __tablename__ = "product_realised"

    id = Column(BigInteger, primary_key=True, autoincrement=True)
    flow_run_id = Column(BigInteger, ForeignKey("flow_run.id", ondelete="CASCADE", onupdate="CASCADE"))
    pi_category_id = Column(Integer, ForeignKey("pi_category.id"))
    eopf_type = Column(Text)
    stac_item = Column(JSONB)
    sensing_start_datetime = Column(TIMESTAMP)
    origin_date = Column(TIMESTAMP)
    catalog_stored_datetime = Column(TIMESTAMP)
    unexpected = Column(Boolean, default=False)
    on_time_0_day = Column(Boolean, default=False)
    on_time_1_day = Column(Boolean, default=False)
    on_time_2_day = Column(Boolean, default=False)
    on_time_3_day = Column(Boolean, default=False)
    on_time_7_day = Column(Boolean, default=False)

    flow_run = relationship("FlowRun", back_populates="products_realised")
    pi_category = relationship("PiCategory", back_populates="products_realised")