Testing

Contributors will want to test their changes against a baseline to ensure no regression appear.

S1-Tiling tests are not part of an integrated continuous workflow. They are meant to be run on an on-demand basis.

At this moment we only have a single end-to-end test on S2 33NWB tile on S1 images acquired in January 2020.

Installation

In order to install all the packets required to execute tests, the installation shall request the dev extra dependencies:

cd s1tiling-sources-directory
pip install -e .[dev]

Also, the tests depend on VCR cassettes to replay network IO. Currently, this Python project has a bug and requires patching: [VCRpy#956]. To do so, once S1Tiling environment has been installed and activated, the patch can be applied with:

(cd /tmp && wget https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling-dockers/-/raw/master/scripts/vcrpy-956.patch)
cd "$(python3 -c "import sysconfig; print(sysconfig.get_path('purelib')+'/..')")"
patch -p1 < /tmp/vcrpy-956.patch
cd -

The baseline

There are two ways to obtain the baseline:

• Either we have given you an authentication token to the S3 server where we have stored the current baseline.

  In that case, thanks to MinIO client, you can:

  • first and once: register your machine to the S3 server we use

    mc config host add minio-otb https://s3.orfeo-toolbox.org/ <access-key> <secret-key> --api S3v4
    

  • then retrieve the baseline data thanks to

    mc cp --recursive minio-otb/s1-tiling/baseline /some/local/path
    

  Instead of mc, you can also use rclone – which is for instance already installed on CNES clusters.

• Or you’ll need to first establish the baseline from a version of S1Tiliing known to work correctly, before introducing any change.

  Organize the S1 images downloaded into a directory named inputs and the results into a directory named expected.

Running the tests

S1 Tiling tests depend on Pytest.

I usually execute the tests with:

$ SRTM_DIR=/path/to/MNT/SRTM_30_hgt pytest --cov=s1tiling --baselinedir=/path/to/tests/20200306-NR/baseline/ \
-k 'not slow' -vvv --log-cli-level=DEBUG -o log_cli=true --capture=no --junitxml=report.xml \
--ram 2048 --durations=0 -s --record-mode=once 2>&1 | less -R

You can see all the supported options with:

pytest --help

In particular, they depend on the following options:

--baselinedir=BASELINEDIR

Directory where the baseline is.

--outputdir=OUTPUTDIR

Directory where the S2 products will be generated.

Warning

Don’t forget to clean it eventually.

--tmpdir=TMPDIR

Directory where the temporary files will be generated.

Warning

Don’t forget to clean it eventually.

--demdir=SRTMDIR

Directory where DEM (like SRTM) files are – default value: $SRTM_DIR.

--download

Download the input files with eodag instead of using the compressed ones from the baseline. If true, raw S1 products will be downloaded into tmpdir/inputs.