.. # define a hard line break for HTML .. |br| raw:: html
Scenarios --------- .. contents:: Contents: :local: :depth: 3 .. _scenario.S1Processor: Orthorectify pairs of Sentinel-1 images on Sentinel-2 grid ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ In this scenario pairs of Sentinel-1 images are: - calibrated according to β\ :sup:`0`, γ\ :sup:`0` or σ\ :sup:`0` calibration - then orthorectified onto MGRS Sentinel-2 grid, - to be finally concatenated. The unique elements in this scenario are: - the :ref:`calibration option ` that must be either one of ``beta``, ``sigma`` or ``gamma`` - the main executable which is :ref:`S1Processor`. All options go in a :ref:`request configuration file ` (e.g. ``MyS1ToS2.cfg`` in ``workingdir``). Important options will be: - the time range (:ref:`first_date ` and :ref:`last_date `), - the :ref:`Sentinel-2 tiles `, - the orthorectification options (in :ref:`[Processing] `), - the directories where images are downloaded, produced, etc. - the download credentials for the chosen data provider -- see :ref:`eodag_config `. Then running S1Tiling is as simple as: .. code:: bash cd workingdir S1Processor MyS1ToS2.cfg Eventually, - The S1 products will be downloaded in :ref:`s1_images `. - The orthorectified tiles will be generated in :ref:`output `. - Temporary files will be produced in :ref:`tmp `. .. note:: S1 Tiling never cleans the :ref:`tmp directory ` as its files are :ref:`cached ` in between runs. This means you will have to watch this directory and eventually clean it. .. _scenario.S1ProcessorLIA: Orthorectify pairs of Sentinel-1 images on Sentinel-2 grid with σ\ :sup:`0`\ :sub:`RTC` NORMLIM calibration +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ This scenario is a variation of the :ref:`previous one `. The difference lies in the calibration applied: it is the :math:`σ^0_{RTC}` NORMLIM calibration described in [Small2011]_. .. [Small2011] D. Small, "Flattening Gamma: Radiometric Terrain Correction for SAR Imagery," in IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 8, pp. 3081-3093, Aug. 2011, doi: 10.1109/TGRS.2011.2120616. In S1Tiling, we have chosen to precompute Local Incidence Angle (LIA) maps on MGRS Sentinel-2 grid. Given a precise orbit file, a relative orbit and a MGRS tile, we directly compute the correction map on the selected Sentinel-2 tile. That map will then be used for all series of pairs of Sentinel-1 images, of compatible orbit, β° calibrated and projected to the associated S2 tile. Regarding options, the only difference with previous scenario are: - the :ref:`calibration option ` that needs to be ``normlim``, - the :ref:`directory ` where EOF files will be searched for, or downloaded to. - the :ref:`directory ` where LIA maps will be searched for, or produced in. - a single pair of :ref:`platform ` + :ref:`relative orbit ` to which the Local Incidence Angles will be calculated, S1Tiling will then automatically take care of: - obtaining the precise orbit files (EOF), if none match the request parameters, - producing, or using existing, maps of sin(LIA) for each Sentinel-2 tiles -- given an orbit and its direction, - producing intermediary products calibrated with β\ :sup:`0` LUT. .. list-table:: :widths: auto :header-rows: 0 :stub-columns: 0 * - .. figure:: _static/sin_LIA_s1a_33NWB_DES_007.jpeg :alt: sine(LIA) :scale: 50% Map of sine(LIA) on 33NWB descending orbit 007 - .. carousel:: :show_controls: :show_indicators: :show_fade: :show_shadows: :show_dark: :show_captions_below: :data-bs-interval: false :data-bs-pause: false .. figure:: _static/s1a_33NWB_vh_DES_007_20200108txxxxxx_beta.jpeg :alt: 33NWB β° calibrated :scale: 50% 33NWB β° calibrated -- 20200108 .. figure:: _static/s1a_33NWB_vh_DES_007_20200108txxxxxx_NormLim.jpeg :alt: 33NWB NORMLIM σ° RTC calibrated :scale: 50% 33NWB NORMLIM σ° RTC calibrated -- 20200108 .. figure:: _static/s1a_33NWB_vh_DES_007_20200108txxxxxx_Normlim_filtered_lee.jpeg :alt: 33NWB NORMLIM σ° RTC calibrated and filtered :scale: 50% 33NWB σ° RTC calibrated and despeckled (Lee) -- 20200108 .. warning:: If you wish to parallelize this scenario and dedicate a different cluster node to each date -- as recommended in “:ref:`scenario.parallelize_date`” scenario, you will **NEED** to produce all the LIA maps beforehand. Otherwise, a same file may be concurrently written to from different nodes, and it will likely end up corrupted. .. note:: This scenario requires `NORMLIM σ0 `_ binaries. At the moment, NORMLIM σ\ :sup:`0` binaries need to be compiled manually. Unless you use either S1Tiling docker images, or S1Tiling on CNES TREX cluster. .. note:: This scenario requires to configure either ``cop_dataspace`` data provider in :ref:`eodag configuration file `, or to enter valid EarthData credentials in your :file:`~/.netrc` file (can be overridden with :envvar:`$NETRC`). .. _scenario.S1LIAMap: Pre-produce maps of Local Incidence Angles for σ\ :sup:`0`\ :sub:`RTC` NORMLIM calibration ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ While :ref:`S1Processor` is able to produce the necessary LIA maps on the fly, it is not able to do so when parallelization is done manually over time ranges -- as described in “:ref:`scenario.parallelize_date`” scenario. A dedicated program is provided to compute the LIA maps beforehand: :ref:`S1LIAMap`. It takes the exact same parameter files as :ref:`S1Processor`. A few options will be ignored though: calibration type, masking… But the following (non-obvious) options are mandatory: - :ref:`[DataSource].platform_list ` -- but only a single value shall be used - :ref:`[DataSource].relative_orbit_list ` -- but only a single value shall be used - :ref:`[DataSource].first_date ` and :ref:`[DataSource].last_date ` if :ref:`[DataSource].download ` is ``True`` and EOF files are missing. .. code:: bash cd workingdir # Yes, the same file works! S1LIAMap MyS1ToS2.cfg .. note:: LIA maps are perfect products to be stored and reused. .. note:: This scenario requires `NORMLIM σ0 `_ binaries. At the moment, NORMLIM σ\ :sup:`0` binaries need to be compiled manually. Unless you use either S1Tiling docker images, or S1Tiling on CNES TREX cluster. .. note:: To run :ref:`S1LIAMap` from the official S1Tiling docker, use ``--lia`` as the first parameter to the docker execution (just before the request configuration file and other S1LIAMap related parameters). See :ref:`docker.S1LIAMap`. .. _scenario.S1IAMap: Produce maps of Ellipsoid Incidence Angles ++++++++++++++++++++++++++++++++++++++++++ S1Tiling permits producing :ref:`maps of cosine, sine and/or tangent of the incidence angle over the WGS84 ellipsoid `, thanks to :ref:`S1IAMap program `. See :ref:`dataflow-eia` for more detailed information on the internal operation sequencing. The typical use case is the following: 1. Sine and cosine maps have been generated (with :ref:`S1IAMap`), and cached, for all MGRS Sentinel-2 tiles of interest. 2. Series of calibrated and ortho-rectified Sentinel-1 data have been generated for a given calibration (typically :ref:`σ° `), and possibly made available on data providers like `CNES's Geodes `_. 3. You can obtain the same product in other calibrations very quickly by applying the corrective sine/cosine map on the Sentinel-2 tiles product. When input product has been :ref:`σ° calibrated `, products in other calibrations can be obtained thanks to :download:`apply-calibration-map.sh <../s1tiling/resources/apply-calibration-map.sh>`. To convert a σ° calibrated product into: - a β° calibrated product, the image is divided by the :ref:`sine map ` .. code:: bash # By hand, with OTB, wrong CALIBRATION metadata otbcli_BandMath \ -il s1a_tile_polar_dir_087_time_sigma.tif sin_IA_s1a_tile_087.tif \ -exp 'im1b1/im2b1' \ -out s1a_tile_polar_dir_087_time_beta.tif # Fix the incorrect metadata gdal_edit.py -mo CALIBRATION=beta s1a_tile_polar_dir_087_time_beta.tif # By hand, with gdal, all metadata are lost gdal_calc.py \ -A s1a_tile_polar_dir_087_time_sigma.tif \ -B sin_IA_s1a_tile_087.tif \ --calc "A/B" --out s1a_tile_polar_dir_087_time_beta.tif # Wrapped for batch application, with OTB, correct metadata apply-calibration-map.sh -c beta --dirmap path/to_sinIA_files path/to/S1Tiling/products - a γ° calibrated product, the image is divided by the :ref:`cosine map ` .. code:: bash # By hand, with OTB, wrong CALIBRATION metadata otbcli_BandMath \ -il s1a_tile_polar_dir_087_time_sigma.tif cos_IA_s1a_tile_087.tif \ -exp 'im1b1/im2b1' \ -out s1a_tile_polar_dir_087_time_gamma.tif # Fix the incorrect metadata gdal_edit.py -mo CALIBRATION=gamma s1a_tile_polar_dir_087_time_beta.tif # By hand, with gdal, all metadata are lost gdal_calc.py \ -A s1a_tile_polar_dir_087_time_sigma.tif \ -B cos_IA_s1a_tile_087.tif \ --calc "A/B" --out s1a_tile_polar_dir_087_time_gamma.tif # Wrapped for batch application, with OTB, correct metadata apply-calibration-map.sh -c gamma --dirmap path/to_cosIA_files path/to/S1Tiling/products .. note:: Given the calibration is applied on the Sentinel-2 tile geometry, and not in the original Sentinel-1 image geometry, small precision differences may be observed between this approach and :ref:`the one where the desired calibration is applied at the beginning of the processing `. Relevant parameters ^^^^^^^^^^^^^^^^^^^ It takes a very similar parameter files as :ref:`S1Processor`. Actually the same file can be used: only relevant parameters will be taken in account: - :ref:`[Paths].output ` or :ref:`[Paths].ia ` - :ref:`[Paths].eof_dir ` - :ref:`[Paths].tmp ` - :ref:`[DataSource].eodag_config ` - :ref:`[Processing].tiles ` - :ref:`[DataSource].platform_list ` -- but only a single value shall be used - :ref:`[DataSource].relative_orbit_list ` -- but only a single value shall be used - :ref:`[DataSource].first_date ` and :ref:`[DataSource].last_date ` if :ref:`[DataSource].download ` is ``True`` and EOF files are missing. - :ref:`[Processing].output_spatial_resolution `, - :ref:`[Processing].ia_maps_to_produce `, - plus filename name format options, parallelization options… .. code:: bash cd workingdir # Yes, the same file works! S1IAMap MyS1ToS2.cfg .. note:: This scenario requires `NORMLIM σ0 `_ binaries. At the moment, NORMLIM σ\ :sup:`0` binaries need to be compiled manually. Unless you use either S1Tiling docker images, or S1Tiling on CNES TREX cluster. .. note:: To run :ref:`S1IAMap` from the official S1Tiling docker, use ``--ia`` as the first parameter to the docker execution (just before the request configuration file and other S1IAMap related parameters). See :ref:`docker.S1IAMap`. .. _scenario.masks: Generate masks on final products ++++++++++++++++++++++++++++++++ Pixel masks of valid data can be produced in all :ref:`S1Processor` scenarios when the option :ref:`generate_border_mask ` is ``True``. .. _scenario.parallelize_date: Process huge quantities of data +++++++++++++++++++++++++++++++ This use case concerns people that: - have a lot of images to process over many tiles and over a consequent time-range, - and have access to computing resources like HPC clusters In that case, S1Tiling will be much more efficient if the parallelization is done time-wise. We recommend cutting the full time range in smaller subranges, and to distribute each subrange (with all S2 tiles) to a different node -- with jobarrays for instances. .. warning:: This scenario is not compatible with ``normlim`` calibration where the LIA maps would be computed on-the-fly. For ``normlim`` calibration, it's imperative to precompute (and store LIA maps) before going massively parallel. .. _scenario.choose_dem: Use any other set of DEM inputs +++++++++++++++++++++++++++++++ By default, S1Tiling comes with a GPKG database that associates SRTM30 geometries to the SRTM tile filename. In order to use other DEM inputs, we need: 1. DEM files stored in :ref:`[PATHS].dem_dir ` directory. |br| The format of these DEM files needs to be supported by OTB/GDAL. 2. A DEM (GPKG) database that holds a key (or set of keys) that enable(s) to locate/name DEM files associated to a DEM geometry. |br| Set the :ref:`[PATHS].dem_database ` key accordingly. |br| For instance, `eotile `_ provides a couple of DEM databases for various types of DEM files. 3. A naming scheme that will associate an identifier key from the :ref:`DEM database ` to a DEM filename (located in :ref:`[PATHS].dem_dir ` directory). |br| Set the :ref:`[PATHS].dem_format ` key accordingly. |br| The default :file:`{{id}}.hgt` associates the ``id`` key to STRM 30 m DEM files. |br| Using `eotile `_ :file:`DEM_Union.gpkg` as DEM database, we could instead use: - :file:`{{Product10}}.tif` for Copernicus 30 m DEM files, using ``Product10`` key from the GPKG file. - :file:`{{Product30}}.tif` for Copernicus 90 m DEM files, using ``Product30`` key from the GPKG file. 4. Make sure to use a Geoid file compatible with the chosen DEM. For instance S1Tiling is shipped with EGM96 Geoid with is compatible with SRTM. On the other hand, Copernicus DEM is related to EGM2008 (a.k.a. EGM08)