Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions _freeze/docs/force/guide/level2/execute-results/html.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions _freeze/docs/force/guide/processes/execute-results/html.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions _freeze/docs/force/guide/query/execute-results/html.json

Large diffs are not rendered by default.

Binary file modified _freeze/docs/force/guide/query/figure-html/cell-3-output-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions _freeze/docs/force/guide/tsa/execute-results/html.json

Large diffs are not rendered by default.

2 changes: 0 additions & 2 deletions docs/force/.gitignore

This file was deleted.

4 changes: 2 additions & 2 deletions docs/force/SOURCE.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
repository: bcdev/apex-force-openeo
release: manual
commit: e0a10dd5385cd1c0798b1847e3e738aa961e5974
Generated: Fri Jul 24 14:19:48 UTC 2026
commit: d6917a4828a21ac94a563f6efd603949b7b83fc7
Generated: Thu Aug 20 08:30:08 UTC 2026
5 changes: 2 additions & 3 deletions docs/force/feature_overview.qmd
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
---
title: Features and Limitations

---


The FORCE integration into to CDSE is in a proof-of-concept stage. It is possible to use the FORCE level 2 and Time Series Analysis (TSA) modules on a small scale. Most parameters supported by FORCE are available through the openEO interface.
The FORCE integration into to CDSE is in a proof-of-concept stage. It is possible to use the FORCE level 2 and Time Series Analysis (TSA) modules on a small scale. Most parameters supported by FORCE are available through the openEO interface.

Unsupported parameters are usually either managed by the implementation (parallelization options, input/output directores) or reuqire supplementary files which are not avaiable on the backend.
The `aoi` (openEO interface) / `FILE_AOI` (FORCE parameter file) parameter is good example of a parameter that is exposed in a different way in openEO to avoid the file interface. Instead of a shape file, a GEOJSON string is passed.
Expand Down Expand Up @@ -37,6 +36,6 @@ If you encounter limitations not mentioned here, please raise an issue in the [G
### Parallelizing large Time Series Analysis jobs

Server-side parallelization of the TSA module is not currently enabled. It is possible to parallelize TSA by creating one openEO job per tile by setting the `x_tile_range` and `y_tile_range` arguments to a single tile per job.
Furthermore, consider computing spatio-temporal metrics (STM) in separate jobs to reduce memory usage.
Furthermore, consider computing spatio-temporal metrics (STM) in separate jobs to reduce memory usage.

Finally, you may reduce the `chunk_size` which will reduce memory usage at the cost of a longer processing time.
24 changes: 16 additions & 8 deletions docs/force/guide/intro.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,45 @@ Instead of installing FORCE locally, manually downloading data and annotating yo

## Prerequisites

This guide assumes basic familiarity with the FORCE processing engine.
This guide assumes basic familiarity with the FORCE processing engine and Python.
You do not need to be an expert FORCE user to make use of the CDSE deployment.

All examples use the [openEO Python client](https://open-eo.github.io/openeo-python-client/). Due to openEO's
language-agnostic architecture, you should be able to replicate all behavior with other client libraries as well
(e.g., [R](https://openeo.org/documentation/1.0/r/), [Javascript](https://openeo.org/documentation/1.0/javascript/)),
although this has not been tested.

::: {.callout-tip title="What to read if you are unfamiliar with FORCE"}
If you are an openEO user who wants to get familiar with FORCE, we recommend to have a look at the FORCE documentation pages for
If you are an openEO user who wants to get familiar with FORCE, we recommend to first have a look at the FORCE documentation pages for:

- **The [data cube](https://force-eo.readthedocs.io/en/latest/howto/datacube.html) concept and organization**: This is the file structure you will receive when running FORCE processing modules
- **The [level 2 processing system](https://force-eo.readthedocs.io/en/latest/howto/l2-ard.html#level-2-ard)**: the basic FORCE module to generate analysis ready data cubes from raw Satellite input products
- **The [level 2 processing system](https://force-eo.readthedocs.io/en/latest/howto/l2-ard.html#level-2-ard)**: the basic FORCE module to generate Analysis Ready Data Cubes from raw Satellite input products
- (Optional) **The [Time Series Analysis (TSA)](https://force-eo.readthedocs.io/en/latest/components/higher-level/tsa/index.html#time-series-analysis) module**: This is the higher level processing module made available through openEO
:::

::: {.callout-tip title="What to read if you are unfamiliar with openEO"}
If you are a FORCE user curious how to process in the cloud and the openEO interface, it will be helpful to be familiar with
If you are a FORCE user curious how to process in the cloud and the openEO interface, it will be helpful to be familiar with:

- The [openEO vocabulary](https://openeo.org/documentation/1.0/glossary.html#processes)
- **The [openEO Python client](https://open-eo.github.io/openeo-python-client/api.html)**: The main interface used to interact with the openEO backend. You may want to use the [R client](https://open-eo.github.io/openeo-r-client/) or [Javascript client](https://open-eo.github.io/openeo-js-client/latest/) instead, if you prefer. However, in this guide, examples will be given with the Python client.
- **The [openEO Python client](https://open-eo.github.io/openeo-python-client/api.html)**: The main interface used to interact with the openEO backend.
You may want to use the [R client](https://open-eo.github.io/openeo-r-client/) or [Javascript client](https://open-eo.github.io/openeo-js-client/latest/) instead, if you prefer.
However, in this guide, examples will be given exclusively with the Python client.
:::

::: {.callout-important title="Data cube representation"}
Please note that the FORCE integration does not make use of [openEO's data cube concept](https://openeo.org/documentation/1.0/datacubes.html), processing is done on FORCE's native data cubes.
Please note that the FORCE integration does not make use of [openEO's data cube concept](https://openeo.org/documentation/1.0/datacubes.html),
processing is done on FORCE's native data cubes. Therefore, other openEO processes cannot be applied to FORCE data cubes.
:::

## Scope

In this guide, you will learn how to create a full processing pipeline using FORCE and openEO:

1. Discover inputs: Query a STAC catalog to determine input products
2. Generate an Analysis Ready (ARD) data cube with the FORCE level 2 processing system
2. Generate an Analysis Ready data cube with the FORCE level 2 processing system
3. Analyze time series with FORCE's higher level processing system

On the way, we will cover how to
On the way, we will cover how to:

- Download data cubes for local processing / visualization / permanent storage
- Apply higher level processing without having to download intermediate results
59 changes: 25 additions & 34 deletions docs/force/guide/level2.qmd
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
---
execute:
freeze: true
title: FORCE level 2
order: 4
---

# FORCE level 2

## Recap

:::{.callout-tip collapse="true" title="Selecting input products"}
Expand All @@ -14,7 +11,6 @@ order: 4

L1C_COLLECTION_URL = "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l1c"
STAC_ROOT_URL = "https://stac.dataspace.copernicus.eu/v1"
#w,s,e,n = 10.386, 44.437, 11.423, 44.973
w, s, e, n = 11.0, 44.5, 11.1, 44.6
spatial_extent = { "west": w, "south": s, "east": e, "north": n}
# ATTENTION: Inclusion of the second date in the search depends on the query method.
Expand Down Expand Up @@ -70,7 +66,7 @@ The integrated FORCE uses (lowercase) [`snake_case`](https://en.wikipedia.org/wi

The parameters are documented in the process description. We can inspect it using the Python client's [`describe_processes()`](https://open-eo.github.io/openeo-python-client/api.html#openeo.rest.connection.Connection.describe_process).

<!--
<!--

FIXME:
It is necessary to connect to the staging backend, until the custom processes are publicly
Expand All @@ -84,8 +80,9 @@ The parameters are documented in the process description. We can inspect it usin
connection = openeo.connect("openeo.dataspace.copernicus.eu").authenticate_oidc()
```

<!-- TODO
Temporary until we can use the production environment directly
<!-- FIXME
Temporarily we need to use openeo-staging for `describe_process` until the process is discoverable on the production
environment directly
-->

```{python}
Expand All @@ -100,10 +97,15 @@ connection.describe_process("force_level2")
```
:::

```{python}
#| echo: false
connection = openeo.connect("openeo.dataspace.copernicus.eu").authenticate_oidc()
```

### Building the Process Graph

The FORCE process graph is based on the `force_level2` process, as described in [openEO processes](processes.qmd). Because FORCE does not use the standard openEO version of the "data cube" concept, but
uses its own data cube representation, we need to wrap the process so that we can interact with the results in a convenient way as a StacResource and export the processing results to a workspace.
uses its own data cube representation, we need to wrap the process so that we can interact with the results in a convenient way as a `StacResource` and export the processing results to a workspace.


Concept | Qualified name (Python client) | Purpose
Expand All @@ -123,15 +125,17 @@ When running FORCE level2 on its own, we do not need to use a workspace. Simply

We can instruct openEO to store our results in a workspace using the [`export_workspace`](https://openeo.org/documentation/1.0/processes.html#export_workspace) openEO process.

When using `export_workspace`, we need to pass a `merge` string to identify your result. The merge will be a key prefix on object storage. The `merge` is something similar to a path in a file system.
When using `export_workspace`, we need to pass a `merge` string to identify your result. The merge will be a key prefix on object storage. The `merge` is something similar to a path in a file system.
It is very important to set a good `merge` parameter and remember the result, because results can only be accessed when the `merge` is known. Furthermore, all users of the workspace (in the case of the `apex-force-results-workspace`, anyone) may write data, so make sure to use a unique `merge`, to reduce the likelihood of accidentally modifying others' data and have others overwrite your data by accident.


:::{.callout-tip}
# FORCE workspace

The dedicated workspace for FORCE results is accessible to all users:
`apex-force-results-workspace` (<https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me>)

- id: `apex-force-results-workspace`
- url: https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me
:::


Expand Down Expand Up @@ -367,7 +371,7 @@ While the cube will stay in place for multiple days, eventually the storage will
<!--

We use a succeeded job instead of the one created above to avoid having to run and
wait for an entire job every time the documentation is re-generated
wait for an entire job every time the documentation is re-generated
(happens only when the freeze cache is invalidated:
https://quarto.org/docs/projects/code-execution.html#freeze).

Expand All @@ -385,8 +389,7 @@ While the cube will stay in place for multiple days, eventually the storage will

```{python}
#| echo: false
#l2_job = connection.job("j-260527114522498d959f20eff3fc9b78")
l2_job = connection.job("j-260702093443422db1ddead799544cff")
l2_job = connection.job("j-26081912081541d78d85f046c726a647")
```

#### Inspecting STAC metadata {#sec-inspecting-stac}
Expand All @@ -395,10 +398,11 @@ l2_job = connection.job("j-260702093443422db1ddead799544cff")
This step is optional
:::

The basic URL of the FORCE workspace `apex-force-results-workspace` is
<https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me>. Together with the `merge` path defined for our processing, we can determine the path to the `catalog.json` for our FORCE level results:
The basic URL of the FORCE workspace `apex-force-results-workspace` is
<https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me>.
Together with the `merge` path defined for our processing, we can determine the path to the `catalog.json` for our FORCE level results:

```{.python}
```{python}
from pystac.catalog import Catalog

WORKSPACE_URL = "https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me"
Expand All @@ -410,35 +414,22 @@ l2_catalog.set_self_href(l2_catalog_url)
l2_catalog
```

<!-- FIXME can be removed once there is a result present at the merge -->
```{python}
#| echo: false
WORKSPACE_URL = "https://s3.waw4-1.cloudferro.com/apex-force-results-waw4-1-exotc5yuexi2c5tvwqhoivj62fz8v0uupy0me"
```


#### Downloading files

:::{.callout-tip}
This step is optional if you plan to continue processing with FORCE on CDSE
:::

Assets can be downloaded using the standard openEO mechanism to access result assets.
The STAC metadata specifies the directory structure of the resulting cube. This structure is respected by openEO's [`download_files`](https://open-eo.github.io/openeo-python-client/api.html#openeo.rest.job.JobResults.download_files).
The STAC metadata specifies the directory structure of the resulting cube. This structure is respected by openEO's [`download_files`](https://open-eo.github.io/openeo-python-client/api.html#openeo.rest.job.JobResults.download_files).

To inspect a particular asset, we may use [`download_file`](https://open-eo.github.io/openeo-python-client/api.html#openeo.rest.job.JobResults.download_file) to download singular files instead.

```{.python}
```{python}
l2_results = l2_job.get_results()
l2_results
```

```{.python}
```{python}
l2_results.download_files("force-level2-results")
```

<!--
```{.bash}
tree force-level2-results
```
-->
```
31 changes: 16 additions & 15 deletions docs/force/guide/parametrization.qmd
Original file line number Diff line number Diff line change
@@ -1,17 +1,13 @@
---
execute:
freeze: true
title: Parametrization
order: 6
---

# Parametrization

The [FORCE openEO processes](processes.qmd) (`force_level2` and `force_tsa`) accept most of the parameters that can be passed using FORCE's native parametrization files ([level 2](https://force-eo.readthedocs.io/en/latest/components/lower-level/level2/param.html), [TSA](https://force-eo.readthedocs.io/en/latest/components/higher-level/tsa/param.html)). The exceptions to this rule are explained below.

There are 4 types of parameters that do not correspond exactly to the FORCE parameter files
There are 4 types of parameters that do not correspond exactly to the FORCE parameter files

1. Special parameters that require translation (e.g., the area of interest)
1. Additional parameters of the openEO integration (e.g., the processing name)
1. Additional parameters of the openEO integration (e.g., the processing `name`)
1. FORCE parameters that are not exposed through the openEO API (e.g., the number of processes used for processing)
1. Parameters that support only a subset of the possible values

Expand All @@ -20,22 +16,27 @@ It is possible to inspect the processes (and see their parameters) using openEO'

Parameter values may be strings (the same as in FORCE parameter files) or native python values which can be serialized to JSON. Values are converted to UPPERCASE before being passed into the parameter file.

Parameter values are passed as a dictionary to the `arguments` parameter of `openeo.processes.process`:
Parameter values are passed as a dictionary to the `arguments` parameter of `openeo.internal.graph_building.PGNode`:


```{.python}
graph = openeo.processes.process(
from openeo.internal.graph_building import PGNode

graph = PGNode(
process_id="force_level2",
arguments=dict(
stac_document=stac_catalog,
name="Modena",
do_brdf=True,
# ... <--- more parameters
arguments={
"stac_document": stac_catalog,
"name": "Modena",
# FORCE parameters
"do_brdf": True,
# ... <--- more FORCE parameters
}
)

```

This process must be wrapped as a `StacResource` as described in [FORCE level 2](level2.qmd).

## Examples

| FORCE parameter file | openEO |
Expand Down Expand Up @@ -77,7 +78,7 @@ Input/Output directories (managed by the integration)

Parallel processing parameters (managed by the integration)

- `NPROC`
- `NPROC`
- `NTHREAD`
- `PARALLEL_READS`
- `DELAY`
Expand Down
Loading
Loading