Configuration

Request Configuration file

The request configuration file passed to S1Processor is in .ini format. It is expected to contain the following entries.

Note

S1LIAMap and S1IAMap work with a subset of the following configuration keys. Unsupported keys will simply be ignored.

You can use this this template, as a starting point.

[PATHS] section

Option	Description
s1_images	Where S1 images are downloaded to, thanks to EODAG. S1Tiling will automatically take care to keep at most 1000 products in that directory – the 1000 last products that have been downloaded. This enables to cache downloaded S1 images in between runs.
output	Where products are generated.
ia	Where (Ellipsoid) Incidence Maps and cos(IA)/sin(IA) products are generated. Its default value is {output}/_IA.
lia	Where Local Incidence Maps and sin(LIA) products are generated. Its default value is {output}/_LIA.
gamma_area	Where γ Area products are generated. Its default value is {output}/_GAMMA_AREA.
tmp	Where intermediary files are produced, and sometimes cached for longer periods.
geoid_file	Path to Geoid model. If left unspecified, it’ll point automatically to the geoid resource shipped with S1 Tiling. Warning Make sure to use an EGM2008 model for Copernicus DEM files.
dem_database	Path to DEM (.gpkg) database. By default points to the internal shapefile/srtm_tiles.gpkg file which knows the geometry of SRTM 30 DEM files.
dem_dir	Path to DEM files.
dem_format	Filename format string to locate the DEM file associated to an identifier within the [PATHS].dem_dir directory. By default associates the id key of tiles found in the DEM database to {id}.hgt. One may want to use the keys from eotile DEM database like for instance {Product10}.tif for Copernicus 30m DEM.
dem_info	DEM identifier to inject in the products GeoTIFF metadata under DEM_INFO key. If not defined, the last part (basename) of [Paths].dem_dir will be used. See § Use any other set of DEM inputs entry for more detailled information.
srtm	(deprecated) Use [PATHS].dem_dir. Path to SRTM files.
eof_dir	Where precise orbit files (EOF) are expected to be found, or where they would be downloaded to on the fly. Default value is {output}/_EOF. See also Q: How can I configure precise orbit files retrieval?.

[DataSource] section

Option	Description
download	When True, enables the downloading: of Sentinel-1 images, that intersects the ROI, from the specified data provider – only local images already in s1_images will be processed otherwise. and of missing EOF products, in scenarios that require them (currently for the production of LIA and IA maps).
eodag_config	Designates where the EODAG configuration file is expected to be found. Default value is fetched in order: ${EODAG_CFG_FILE} > ${EODAG_CFG_DIR}/eodag.yml > %(HOME)s/.config/eodag/eodag.yml. From S1Tiling point of view, EODAG configuration file will list the authentification credentials for the know providers and their respective priorities. See EODAG § on Configure EODAG For instance, given a PEPS account, $HOME/.config/eodag/eodag.yml could contain peps: auth: credentials: username: THEUSERNAME password: THEPASSWORD
nb_parallel_downloads	Number of parallel downloads (+ unzip) of source products. Warning Don’t abuse this setting as the data provider may not support too many parallel requests.
roi_by_tiles	The Region of Interest (ROI) for downloading is specified in roi_by_tiles which will contain a list of MGRS tile names. If ALL is specified, the software will download all images needed for the processing (see [Processing] section) [DataSource] roi_by_tiles : 33NWB
platform_list	Filter to restrict the list of Sentinel-1 platforms from where the products, to download and process, come from. Valid values are comma separated lists of S1A, S1B, and S1C. By default (when left unspecified), no filter is applied. Warning Only one single value is expected in NORMLIM and Ellipsoid Incidence Angle scenarios.
polarisation	Filter on the polarisation mode of the Sentinel-1 products to download and process. Only six values are valid: HH-HV, VV-VH, VV, VH, HV, and HH.
orbit_direction	Filter on the orbit direction of the Sentinel-1 products to download and process. Only two values are valid: ASC (ascending mode order) and DSC (descending mode order). By default (when left unspecified), no filter is applied. Warning Each relative orbit is exclusive to one orbit direction, orbit_direction and relative_orbit_list shall be considered as exclusive.
relative_orbit_list	Filter to download and process only the Sentinel-1 products from the specified relative orbits. Valid values are comma separated list of relative orbit numbers (∈ [1..175]). By default (when left unspecified), no filter is applied. Warning Each relative orbit is exclusive to one orbit direction, orbit_direction and relative_orbit_list shall be considered as exclusive. Warning One and exactly one single value is expected in NORMLIM and Ellipsoid Incidence Angle scenarios.
first_date	Initial date in YYYY-MM-DD format.
last_date	Final date in YYYY-MM-DD format.
tile_to_product_overlap_ratio	Percentage of tile area to be covered for a single or a pair of Sentinel-1 products to be retained. The number is expected as an integer in the [1..100] range.

[Mask] section

Option	Description
generate_border_mask	Enable the generation of border masks for the S2-aligned image files produced. Valid values are True or False.

[Processing] section

Option	Description
cache_dem_by	Tells whether DEM and Geoid files are copied in a temporary directory, or if symbolic links are to be created. For performance reasons with OTB, it’s better to regroup the minimal subset of the DEM files required for processing. Symbolic links work fine most of the time, however if the files are on a remote shared filesystem (GPFS, NAS…), performances will be degraded. In those cases, it’s better to copy the required DEM files on a local filesystem. Geoid file will be also copied (or symlinked), but in {tmp}/geoid/. It won’t be removed automatically. You can also do it manually before running S1Tiling. Two values are supported for this option: copy and symlink. (default: symlink).
calibration	Defines the calibration type to apply: gamma, beta, sigma, normlim or gamma_naught_rtc.
remove_thermal_noise	Activate the thermal noise removal in the images. Values are True or False.
lower_signal_value	Noise removal may set some pixel values to 0. However, 0, is currently reserved by S1Tiling chain as a “no-data” value introduced by Margin Cutting and Orthorectification. This parameter defines which value to use instead of 0 when noise is removed. By default: 1e-7 will be used.
nodata.*	Option Description nodata.IA No-data value to use in IA files nodata.LIA No-data value to use in LIA files nodata.RTC No-data value to use when applying Gamma Area map
output_spatial_resolution	Pixel size (in meters) of the output images
tiles_shapefile	Path and filename of the tile shape definition (ESRI Shapefile). If left unspecified, it’ll point automatically to the Features.shp shapefile resource shipped with S1 Tiling.
orthorectification_gridspacing	Grid spacing (in meters) for the interpolator in the orthorectification process. For more information, please consult OTB OrthoRectification application documentation. A nice value is 4 x output_spatial_resolution
orthorectification_interpolation_method	Interpolation method used in the orthorectification process. For more information, please consult OTB OrthoRectification application documentation. Default value is set to nearest neighbor interpolation (nn) to keep compatibilty with previous results ; Linear method could be more interesting. Note that the bco method is not currently supported.
tiles, tiles_list_in_file	Tiles to be processed. The tiles can be given as a list: tiles: list of tiles (comma separated). Ex: tiles: 33NWB, 33NWC tiles_list_in_file: tile list in a ASCII file. Ex: tiles_list_in_file : ~/MyListOfTiles.txt
mode	Running mode: Normal: prints normal, warning and errors on screen debug: also prints debug messages, and forces $OTB_LOGGER_LEVEL=DEBUG logging: saves logs to files Ex.: mode : debug logging
nb_parallel_processes	Number of processes to run in parallel This number defines the number of Dask Tasks (and indirectly of OTB applications) that will be executed in parallel. Note For optimal performances, nb_parallel_processes*nb_otb_threads should be ≤ to the number of cores on the machine.
ram_per_process	RAM allowed per OTB application pipeline, in MB.
nb_otb_threads	Numbers of threads used by each OTB application. Note For optimal performances, nb_parallel_processes*nb_otb_threads should be ≤ to the number of cores on the machine.
produce_lia_map	When LIA sine map are produced, we may also desire the angle values in degrees (x100). Possible values are: True: Do generate the angle map in degrees x 100. False: Don’t generate the angle map in degrees x 100. Note This option will be ignored when no LIA sine map is required. The LIA sine map is produced by S1LIAMap program, or when calibration mode is "normlim".
ia_maps_to_produce	By default, S1IAMap program produce a map of the incidence angle to the WGS84 ellipsoid in degrees x 100. This option permits to select which of the 4 IA maps will be generated. deg: map in degrees x 100 cos: cosine map sin: sine map tan: tangent map
produce_gamma_area_map	When GAMMA_AREA map is produced. Possible values are: True: Do generate the GAMMA_AREA map. False: Don’t generate the GAMMA_AREA map. Note This option will be ignored when no GAMMA_AREA map is required. The GAMMA_AREA map is produced by S1GammaAreaMap program , or when calibration mode is "gamma_naught_rtc".
dem_warp_resampling_method	DEM files projected on S2 tiles are required to produce LIA maps and GAMMA_AREA maps. This parameters permits to select the resampling method that gdalwarp will use. The possible values are: near, bilinear, cubic, cubicspline, lanczos, average, rms, mode, max, min, med, q1, q3 and qum.
override_azimuth_cut_threshold_to	Permits to override the analysis on whether top/bottom lines shall be forced to 0 in cutting step. Possible values are: True: Force cutting at the 1600th upper and the 1600th lower lines. False: Force to keep every line. not set/None: Default analysis heuristic is used. Warning This option is not meant to be used. It only makes sense in some very specific scenarios like tests.
fname_fmt.*	Set of filename format templates that permits to override the default filename formats used to generate filenames. The filename formats can be overridden for both intermediary and final products. Only the final products are documented here. Filename formats for intermediary products are best left alone. If you change any, make sure to not introduce ambiguity by removing a field that would be used to distinguish two unrelated products. Available fields come from internal metadata. The main ones of interest are: Field Content Applies to geometry flying_unit_code s1a, s1b, s1c S1/S2 tile_name ex: 33NWB S2 polarisation hh, hv, vh, vv S1/S2 orbit_direction ASC/DES S1/S2 orbit 5-digits number that identifies the S1 orbit S1/S2 acquisition_time the full timestamp (yymmddthhmmss) S1/S2 acquisition_day only the day (yymmddtxxxxxx) S1/S2 acquisition_stamp either the full timestamp (yymmddthhmmss), or the day (yymmddtxxxxxx) S1/S2 IA_kind IA/cos_IA/sin_IA/tan_IA S2 LIA_kind LIA/sin_LIA S2 basename Filename of initial S1 image. S1 rootname basename without the file extension. S1 calibration_type beta/gamma/sigma/dn/Normlim/gamma_naught_rtc S1/S2 polarless_basename Same as basename (with file extension), but without polarisation field. Used when the product only depends on the S1 image geometry and not its content. S1 polarless_rootname Same as rootname (without file extension), but without polarisation field. Used when the product only depends on the S1 image geometry and not its content. S1
fname_fmt.concatenation	File format pattern for concatenation products, for β°, σ° and γ° calibrations. Default value: {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}.tif
fname_fmt.s2_lia_corrected	File format pattern for concatenation products when NORMLIM calibrated. Default value: {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_NormLim.tif
fname_fmt.ia_product	File format pattern for IA cos(IA), sin(IA) and tan(IA) files Default value: {IA_kind}_{flying_unit_code}_{tile_name}_{orbit}.tif
fname_fmt.lia_product	File format pattern for LIA and sin(LIA) files Default value: {LIA_kind}_{flying_unit_code}_{tile_name}_{orbit}.tif
fname_fmt.s2_gamma_area_corrected	File format pattern for concatenation products when GammaNaughtRTC calibrated. Default value: {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_GammaNaughtRTC.tif
fname_fmt.gamma_area_product	File format pattern for GAMMA_AREA files Default value: GAMMA_AREA_{flying_unit_code}_{tile_name}_{orbit_direction}_{orbit}.tif
fname_fmt.filtered	File format pattern for filtered files Default value: {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_filtered.tif for β°, σ° and γ° calibrations, Default value: {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_NormLim_filtered.tif when NORMLIM calibrated. {flying_unit_code}_{tile_name}_{polarisation}_{orbit_direction}_{orbit}_{acquisition_stamp}_GammaNaughtRTC_filtered.tif when GammaNaughtRTC calibrated.
dname_fmt.*	Set of directory format templates that permits to override the default directories where products are generated. The directory formats can only be overridden for final products. The only fields available are: Field Reference to {tile_name} Name of the related tile. {out_dir} [PATHS].output {tmp_dir} [PATHS].tmp {ia_dir} [PATHS].ia {lia_dir} [PATHS].lia {gamma_area_dir} [PATHS].gamma_area Products from Option dname_fmt.? Default value (β°/σ°/γ°/NORMLIM/γ°RTC) Final tiled product .tiled {out_dir}/{tile_name} Masks .mask {out_dir}/{tile_name} degree(IA), cos(IA), sin(IA) and tan(IA) .ia_product {ia_dir} degree(LIA) and sin(LIA) .lia_product {lia_dir} GAMMA_AREA .gamma_area_product {gamma_area_dir} Filtering .filtered {out_dir}/filtered/{tile_name}
creation_options.*	Set of extra options to create certain products. Creation options take a first and optional pixel type (uint8, float64…) and a list of GDAL creation options. Products from Option creation_options.? Default value Orthorectification, (β°/σ°/γ°/NORMLIM/γ°RTC) Concatenation… .tiled COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 Filtering .filtered COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 Masks .mask uint8 COMPRESS=DEFLATE IA (in degrees * 100) .ia_deg uint16 COMPRESS=DEFLATE cos(IA) .ia_cos COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 sin(IA) .ia_sin COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 tan(IA) .ia_tan COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 LIA (in degrees * 100) .lia_deg uint16 COMPRESS=DEFLATE sin(LIA) .lia_sin COMPRESS=DEFLATE&gdal:co:PREDICTOR=3 S1 image information projected on DEM .s1_on_dem float32 COMPRESS=DEFLATE, BIGTIFF=YES, PREDICTOR=3, TILED=YES, BLOCKXSIZE=1024, BLOCKYSIZE=1024 Warning This default float32 setting is insufficient for a good precision (ECEF coordinates < 1 meter). Yet it has been chosen for people working on machines that don’t have more than 60 GB of memory. See the note in γ area map procuction scenario. GAMMA_AREA in meters square .gamma_area float32 COMPRESS=DEFLATE
disable_streaming.*	Disables OTB streaming in some applications. Option Step Default value Remarks .normals_on_s2 ComputeNormalsOnS2 True Work around OTB issue #2442. .gamma_area SARGammaAreaImageEstimation False Eliminates artefacts in between streaming tile when True, which requires a lot of memory see γ area production scenario. .apply_gamma_area ApplyGammaNaughtRTCCalibration False
\(γ^0_{T}\) RTC specific options	The following options are exclusively used for the preparation and the application of γ area maps. Option Purpose Default value use_resampled_dem Tell to work on resampled DEM before computing γ area maps. The resampling factors are specified in resample_dem_factor_x and resample_dem_factor_y True resample_dem_factor_x Resampling factor on X axis used on DEM when use_resampled_dem is set. 2.0 resample_dem_factor_y Resampling factor on Y axis used on DEM when use_resampled_dem is set. 2.0 distribute_area Distribute area on pixel’s neighbours (corners) in output geometry. Used in SARGammaAreaImageEstimation False inner_margin_ratio Ratio of largest direction for DEM ‘s tile margin for inner tiles. Set to 0 to ignore. Used in SARGammaAreaImageEstimation 0.01 outer_margin_ratio Ratio of largest direction for DEM ‘s tile margin for outer tiles. Set to 0 to ignore. Used in SARGammaAreaImageEstimation 0.04 min_gamma_area Minimum area before entering shadow. Used in ApplyGammaNaughtRTCCalibration 1.0 calibration_factor Scalar calibration factor value. Used in ApplyGammaNaughtRTCCalibration 1.0

[Filtering] section

Note

Multitemporal filtering is not yet integrated in S1Tiling.

Option	Description
filter	The following spatial speckling filter methods from OTB Despeckle application are supported: Lee, Frost, Gammamap, Kuan. If none or empty, then no filtering is done.
window_radius	Sets the window radius for the spatial filtering. Be cautious: this does expect a radius, i.e. radius=1 means the filter does an 3x3 pixels averaging.
deramp	Deramp factor – for Frost filter only. Factor used to control the exponential function used to weight effect of the distance between the central pixel and its neighbourhood. Increasing the deramp parameter will lead to take into account more pixels farther from the centre and therefore increase the smoothing effects.
nblooks	Number of looks – for all but Frost => Lee, Gammamap and Kuan
keep_non_filtered_products	If not caring for non-filtered product (and if filter method is specified), then the orthorectified and concatenated products won’t be considered as mandatory and they will not be kept at the end of the processing. This (exclusion) feature cannot be used alongside [Mask].generate_border_mask (i.e. keep_non_filtered_products cannot be False if generate_border_mask is True) Warning Note: This feature is only supported after LIA calibration as of V1.0 of S1Tiling. See Issue #118.

[Metadata] section

You can place in this section any extra key : value information that you want written in the GeoTIFF metadata of S1Tiling products.

Example:

[Metadata]
# Extra geotiff metadata to write in products
Contact : My Self <some.one@somewhe.re>
Something Important: you need to known this!

Log configuration

Default logging configuration is provided in S1Tiling installing directory.

It can be overridden by dropping a file similar to ../s1tiling/logging.conf.yaml in the same directory as the one where the request configuration file is. The file is expected to follow logging configuration file syntax.

Warning

This software expects the specification of:

• s1tiling, s1tiling.OTB loggers;

• and file and important handlers.

When mode contains logging, we make sure that file and important handlers are added to the handlers of root and distributed.worker loggers. Note that this is the default configuration.

When mode contains debug the DEBUG logging level is forced into root logger, and $OTB_LOGGER_LEVEL environment variable is set to DEBUG.

Working on clusters

Todo

By default, S1Tiling works on single machines. Internally it relies on distributed.LocalCluster a small adaptation would be required to work on a multi-nodes cluster.

Warning

When executing multiple instances of S1Tiling simultaneously, make sure to use different directories for:

• logs – running S1Tiling in different directories, like $TMPDIR/ on TREX, should be enough

• storing input files, like for instance $TMPDIR/data_raw/ on TREX for instance.