s1tiling.libs.steps._FileProducingStepFactory

class s1tiling.libs.steps._FileProducingStepFactory(cfg: Configuration, gen_tmp_dir: str, gen_output_dir: str | None, gen_output_filename: OutputFilenameGenerator, *argv, **kwargs)[source]

Bases: StepFactory

Abstract class that factorizes filename transformations and parameter handling for Steps that produce files, either with OTB or through external calls.

create_step() is kind of abstract at this point.

New methods & Specialized methods

__init__

Constructor

_get_nominal_output_basename

Returns the pathless basename of the produced file (internal).

build_step_output_filename

Returns the names of typical result files in case their production is required (i.e. not in-memory processing).

build_step_output_tmp_filename

This specialization of StepFactory.build_step_output_tmp_filename() will automatically insert .tmp before the filename extension.

output_directory

Accessor to where output files will be stored in case their production is required (i.e. not in-memory processing).

parameters

Most steps that produce files will expect parameters.

tmp_directory

Directory used to store temporary files before they are renamed into their final version.

Methods inherited from parent

complete_meta

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.

create_step

Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.

update_filename_meta

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.

update_image_metadata

Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.

check_requirements

Abstract method used to test whether a StepFactory has all its external requirements fulfilled.

Attributes and properties

image_description

Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION

name

Step Name property.

ram_per_process

Property ram_per_process

_do_create_actual_step(execution_parameters: Dict, input_step: AbstractStep, meta: Dict) AbstractStep

Generic variation point for the exact step creation. The default implementation returns a new AbstractStep.

_update_filename_meta_post_hook(meta: Dict) None

Hook meant to be overridden to fix product metadata by overriding their default definition.

Called from update_filename_meta()

_update_filename_meta_pre_hook(meta: Dict) Dict

Hook meant to be overridden to complete product metadata before they are used to produce filenames or tasknames.

Called from update_filename_meta()

build_step_output_filename(meta: Dict) str | List[str][source]

Returns the names of typical result files in case their production is required (i.e. not in-memory processing).

This specialization uses gen_output_filename naming policy parameter to build the output filename. See the Available naming policies.

build_step_output_tmp_filename(meta: Dict) str | List[str][source]

This specialization of StepFactory.build_step_output_tmp_filename() will automatically insert .tmp before the filename extension.

check_requirements() Tuple[str, str] | None

Abstract method used to test whether a StepFactory has all its external requirements fulfilled. For instance, OTBStepFactory’s will check their related OTB application can be executed.

Returns:

None if requirements are fulfilled.

Returns:

A message indicating what is missing otherwise, and some context how to fix it.

complete_meta(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding Step instanciation.

create_step(execution_parameters: Dict, previous_steps: List[List[Dict[str, AbstractStep]]]) AbstractStep

Instanciates the step related to the current StepFactory, that consumes results from the previous input steps.

1. This methods starts by updating metadata information through: complete_meta() on the input metadatas.

2. Then it updates the GDAL image metadata information that will need to be written in the pipeline output image through update_image_metadata().

3. Eventually the actual step creation method is executed according to the exact kind of step factory (ExecutableStepFactory, AnyProducerStepFactory, OTBStepFactory) through the variation point _do_create_actual_step().

While this method is not meant to be overridden, for simplity it will be in Store factory.

Note: it’s possible to override this method to return no step (None). In that case, no OTB Application would be registered in the actual Pipeline.

property image_description: str

Property image_description, used to fill TIFFTAG_IMAGEDESCRIPTION

property name: str

Step Name property.

output_directory(meta: Dict) str[source]

Accessor to where output files will be stored in case their production is required (i.e. not in-memory processing)

This property is built from gen_output_dir construction parameter. Typical values for the parameter are:

  • os.path.join(cfg.output_preprocess, '{tile_name}'), where tile_name is looked into meta parameter

  • None, in that case the result will be the same as tmp_directory(). This case will make sense for steps that don’t produce required products

parameters(meta: Dict) List[str] | Dict[str, str | int | float | bool | List[str]][source]

Most steps that produce files will expect parameters.

Warning: parameters that designate output filenames are expected to use tmp_filename() and not out_filename(). Indeed products are meant to be first produced with temporary names before being renamed with their final names, once the operation producing them has succeeded.

Note: This method is kind-of abstract – SelectBestCoverage is a _FileProducingStepFactory but, it doesn’t actualy consume parameters.

property ram_per_process

Property ram_per_process

tmp_directory(meta) str[source]

Directory used to store temporary files before they are renamed into their final version.

This property is built from gen_tmp_dir construction parameter. Typical values for the parameter are:

  • os.path.join(cfg.tmpdir, 'S1')

  • os.path.join(cfg.tmpdir, 'S2', '{tile_name}') where tile_name is looked into meta parameter

update_filename_meta(meta: Dict) Dict

Duplicates, completes, and returns, the meta dictionary with specific information for the current factory regarding tasks analysis.

This method is used:

  • while analysing the dependencies to build the task graph – in this use case the relevant information are the file names and paths.

  • and indirectly before instanciating a new Step

Other metadata not filled here:

  • get_task_name() which is deduced from out_filename by default

  • out_extended_filename_complement()

It’s possible to inject some other metadata (that could be used from _get_canonical_input() for instance) thanks to _update_filename_meta_pre_hook().

This method is not meant to be overridden. Instead it implements the template method design pattern, and expects the customization to be done through the specialization of the hooks:

update_image_metadata(meta: Dict, all_inputs: List[Dict[str, AbstractStep]]) None

Root implementation of update_image_metadata() that shall be specialized in every file producing Step Factory.