s1tiling.libs.steps.AnyProducerStepFactory

class s1tiling.libs.steps.AnyProducerStepFactory(cfg: Configuration, action: Callable, gen_tmp_dir: str, gen_output_dir: str | None, gen_output_filename: OutputFilenameGenerator, *argv, **kwargs)

Bases: _FileProducingStepFactory

Abstract StepFactory for executing any Python made step.

All step factories that wrap calls to Python code are meant to inherit from AnyProducerStepFactory.

New methods & Specialized methods

__init__	Constructor
_do_create_actual_step	This Step creation method does more than just creating the step.

Methods inherited from parent

tmp_directory	Directory used to store temporary files before they are renamed into their final version.
parameters	Most steps that produce files will expect parameters.
update_filename_meta	Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.
build_step_output_tmp_filename	This specialization of StepFactory.build_step_output_tmp_filename() will automatically insert .tmp before the filename extension.
check_requirements	Abstract method used to test whether a StepFactory has all its external requirements fulfilled.
create_step	Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.
build_step_output_filename	Returns the names of typical result files in case their production is required (i.e. not in-memory processing).
output_directory	Accessor to where output files will be stored in case their production is required (i.e. not in-memory processing).
update_image_metadata	Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.
complete_meta	Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.

Attributes and properties

image_description	Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION
name	Step Name property.
ram_per_process	Property ram_per_process

_do_create_actual_step(execution_parameters: Dict, input_step: AbstractStep, meta: Dict) → AnyProducerStep

This Step creation method does more than just creating the step. It also executes immediately the external process.

_update_filename_meta_post_hook(meta: Dict) → None

Hook meant to be overridden to fix product metadata by overriding their default definition.

Called from update_filename_meta()

_update_filename_meta_pre_hook(meta: Dict) → Dict

Hook meant to be overridden to complete product metadata before they are used to produce filenames or tasknames.

Called from update_filename_meta()

build_step_output_filename(meta: Dict) → str | List[str]

Returns the names of typical result files in case their production is required (i.e. not in-memory processing).

This specialization uses gen_output_filename naming policy parameter to build the output filename. See the Available naming policies.

build_step_output_tmp_filename(meta: Dict) → str | List[str]

This specialization of StepFactory.build_step_output_tmp_filename() will automatically insert .tmp before the filename extension.

check_requirements() → Tuple[str, str] | None

Abstract method used to test whether a StepFactory has all its external requirements fulfilled. For instance, OTBStepFactory’s will check their related OTB application can be executed.

Returns:

None if requirements are fulfilled.

Returns:

A message indicating what is missing otherwise, and some context how to fix it.

complete_meta(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) → Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.

create_step(execution_parameters: Dict, previous_steps: List[List[Dict[str, AbstractStep]]]) → AbstractStep

Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.

1. This methods starts by updating metadata information through: complete_meta() on the input metadatas.

2. Then it updates the GDAL image metadata information that will need to be written in the pipeline output image through update_image_metadata().

3. Eventually the actual step creation method is executed according to the exact kind of step factory (ExecutableStepFactory, AnyProducerStepFactory, OTBStepFactory) through the variation point _do_create_actual_step().

While this method is not meant to be overridden, for simplity it will be in Store factory.

Note: it’s possible to override this method to return no step (None). In that case, no OTB Application would be registered in the actual Pipeline.

property image_description: str | List[str]

Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION

property name: str

Step Name property.

output_directory(meta: Dict) → str

Accessor to where output files will be stored in case their production is required (i.e. not in-memory processing)

This property is built from gen_output_dir construction parameter. Typical values for the parameter are:

• os.path.join(cfg.output_preprocess, '{tile_name}'), where tile_name is looked into meta parameter

• None, in that case the result will be the same as tmp_directory(). This case will make sense for steps that don’t produce required products

parameters(meta: Dict) → List[str] | Dict[str, str | int | float | bool | List[str]]

Most steps that produce files will expect parameters.

Warning: parameters that designate output filenames are expected to use tmp_filename() and not out_filename(). Indeed products are meant to be first produced with temporary names before being renamed with their final names, once the operation producing them has succeeded.

Note: This method is kind-of abstract – SelectBestCoverage is a _FileProducingStepFactory but, it doesn’t actualy consume parameters.

property ram_per_process

Property ram_per_process

tmp_directory(meta) → str

Directory used to store temporary files before they are renamed into their final version.

This property is built from gen_tmp_dir construction parameter. Typical values for the parameter are:

• os.path.join(cfg.tmpdir, 'S1')

• os.path.join(cfg.tmpdir, 'S2', '{tile_name}') where tile_name is looked into meta parameter

update_filename_meta(meta: Dict) → Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.

This method is used:

• while analysing the dependencies to build the task graph – in this use case the relevant information are the file names and paths.

• and indirectly before instanciating a new Step

Other metadata not filled here:

• get_task_name() which is deduced from out_filename by default

• out_extended_filename_complement()

It’s possible to inject some other metadata (that could be used from _get_canonical_input() for instance) thanks to _update_filename_meta_pre_hook().

This method is not meant to be overridden. Instead it implements the template method design pattern, and expects the customization to be done through the specialization of the hooks:

• _update_filename_meta_pre_hook(),

• _update_filename_meta_post_hook().

update_image_metadata(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) → None

Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.