s1tiling.libs.steps.StepFactory

class s1tiling.libs.steps.StepFactory(name: str, *unused_argv, **kwargs)

Bases: ABC

Abstract factory for AbstractStep

Meant to be inherited for each possible OTB application, external application… used in a pipeline.

Sometimes we may also want to add some artificial steps that analyse products, filenames…, or step that help filter products for following pipelines.

When steps are analysed, their output filename(s) are deduced. This information is stored in the meta dictionary under the key out_filename (and it’s meant to be extracted through out_filename()). It can be a single filename or a list of filenames. Internally it will be used to commit_execution() – i.e. to rename tempory files with their final exact filenames. out_filename computation is supposed to be automatically done by the OutputFilenameGenerator passed to the constructor of some step factories.

Also step results need to be precisely identified. This identifier is extracted with get_task_name(). By default its value is the same as out_filename. In some cases, we need to override this task name. This is meant to be done in _update_filename_meta_post_hook() exclusively. A typical use case is when a steps produced several files. It’s better in that case to have a single task name.

At last, sometimes an (OTB) application produces several files, but it only takes a single ouput parameter which acts as a kind of filename pattern/format. For these situations we need an output parameter which is not the list of out_filename``s. This can be done through the *metadata* key ``output_parameter this is retrieved by output_parameter() helper function – if no output_parameter information is set, this accessor function falls back to out_filename value. This metadata is also meant to be set exclusively in _update_filename_meta_post_hook().

See: Existing processings

New methods & Specialized methods

__init__
_do_create_actual_step	Generic variation point for the exact step creation.
_get_canonical_input	Helper function to retrieve the canonical input associated to a list of inputs.
_get_inputs	Extract the last inputs to use at the current level from all previous products seen in the pipeline.
_update_filename_meta_post_hook	Hook meant to be overridden to fix product metadata by overriding their default definition.
_update_filename_meta_pre_hook	Hook meant to be overridden to complete product metadata before they are used to produce filenames or tasknames.
build_step_output_filename	Filename of the step output.
build_step_output_tmp_filename	Returns a filename to a temporary file to use in output of the current application.
check_requirements	Abstract method used to test whether a StepFactory has all its external requirements fulfilled.
complete_meta	Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.
create_step	Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.
has_several_outputs	Tells whether this step produces several files.
update_filename_meta	Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.
update_image_metadata	Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.

Attributes and properties

image_description	Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION
name	Step Name property.

_do_create_actual_step(execution_parameters: Dict, input_step: AbstractStep, meta: Dict) → AbstractStep

Generic variation point for the exact step creation. The default implementation returns a new AbstractStep.

_update_filename_meta_post_hook(meta: Dict) → None

Hook meant to be overridden to fix product metadata by overriding their default definition.

Called from update_filename_meta()

_update_filename_meta_pre_hook(meta: Dict) → Dict

Hook meant to be overridden to complete product metadata before they are used to produce filenames or tasknames.

Called from update_filename_meta()

abstract build_step_output_filename(meta: Dict) → str | List[str]

Filename of the step output.

See also build_step_output_tmp_filename() regarding the actual processing.

abstract build_step_output_tmp_filename(meta: Dict) → str | List[str]

Returns a filename to a temporary file to use in output of the current application.

When an OTB (/External) application is harshly interrupted (crash or user interruption), it leaves behind an incomplete (and thus invalid) file. In order to ignore those files when a pipeline is restarted, an temporary filename is used by the application. Once the application exits with success, the file will be renamed into build_step_output_filename(), and possibly moved into _FileProducingStepFactory.output_directory() if this is a final product.

check_requirements() → Tuple[str, str] | None

Abstract method used to test whether a StepFactory has all its external requirements fulfilled. For instance, OTBStepFactory’s will check their related OTB application can be executed.

Returns:

None if requirements are fulfilled.

Returns:

A message indicating what is missing otherwise, and some context how to fix it.

complete_meta(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) → Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.

create_step(execution_parameters: Dict, previous_steps: List[List[Dict[str, AbstractStep]]]) → AbstractStep

Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.

1. This methods starts by updating metadata information through: complete_meta() on the input metadatas.

2. Then it updates the GDAL image metadata information that will need to be written in the pipeline output image through update_image_metadata().

3. Eventually the actual step creation method is executed according to the exact kind of step factory (ExecutableStepFactory, AnyProducerStepFactory, OTBStepFactory) through the variation point _do_create_actual_step().

While this method is not meant to be overridden, for simplity it will be in Store factory.

Note: it’s possible to override this method to return no step (None). In that case, no OTB Application would be registered in the actual Pipeline.

has_several_outputs() → bool

Tells whether this step produces several files. This method is meant to be overridden in _FileProducingStepFactory.

Returns:

False by default.

property image_description: str | List[str]

Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION

property name: str

Step Name property.

update_filename_meta(meta: Dict) → Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.

This method is used:

• while analysing the dependencies to build the task graph – in this use case the relevant information are the file names and paths.

• and indirectly before instanciating a new Step

Other metadata not filled here:

• get_task_name() which is deduced from out_filename by default

• out_extended_filename_complement()

It’s possible to inject some other metadata (that could be used from _get_canonical_input() for instance) thanks to _update_filename_meta_pre_hook().

This method is not meant to be overridden. Instead it implements the template method design pattern, and expects the customization to be done through the specialization of the hooks:

• _update_filename_meta_pre_hook(),

• _update_filename_meta_post_hook().

update_image_metadata(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) → None

Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.