Testing

Contributors will want to test their changes against a baseline to ensure no regression appear.

S1-Tiling tests are not part of an integrated continous workflow. They are meant to be run on an on-demand basis.

At this moment we only have a single end-to-end test on S2 33NWB tile on S1 images acquired in January 2020.

The baseline

There are two ways to obtain the baseline:

  • Either we have given you an authentication token to the S3 server where we have stored the current baseline.

    In that case, thanks to MinIO client, you can:

    • first and once: register your machine to the S3 server we use

      mc config host add minio-otb https://s3.orfeo-toolbox.org/ <access-key> <secret-key> --api S3v4

    • then retrieve the baseline data thanks to

      mc cp --recursive minio-otb/s1-tiling/baseline /some/local/path

    Instead of mc, you can also use rclone – which is for instance already installed on CNES clusters.

  • Or you’ll need to first establish the baseline from a version of S1Tiliing known to work correctly, before introducing any change.

    Organise the S1 images downloaded into a directory named inputs and the results into a directory named expected.

Running the tests

S1 Tiling tests depend on pytest.

I usually execute the tests with:

SRTM_DIR=/path/to/MNT/SRTM_30_hgt pytest --cov=s1tiling --baselinedir=/path/to/tests/20200306-NR/baseline/ \
     -k 'not execute_OTB' -vvv --log-cli-level=DEBUG -o log_cli=true --capture=no --junitxml=report.xml \
     --ram 2048 2>&1 | less -R

You can see all the supported options with:

pytest --help

In particular they depend on the following options:

--baselinedir=BASELINEDIR

Directory where the baseline is.

--outputdir=OUTPUTDIR

Directory where the S2 products will be generated.

Warning

Don’t forget to clean it eventually.

--tmpdir=TMPDIR

Directory where the temporary files will be generated.

Warning

Don’t forget to clean it eventually.

--demdir=SRTMDIR

Directory where DEM (like SRTM) files are – default value: $SRTM_DIR.

--download

Download the input files with eodag instead of using the compressed ones from the baseline. If true, raw S1 products will be downloaded into tmpdir/inputs.