mapchete_eo package

Subpackages

Submodules

mapchete_eo.base module

class mapchete_eo.base.BaseDriverConfig(*, format: str, source: Sequence[Source], time: TimeRange | List[TimeRange] | None = None, cat_baseurl: str | None = None, cache: Any | None = None, footprint_buffer: float = 0, area: str | PathLike | dict | type[BaseGeometry] | None = None, preprocessing_tasks: bool = False, search_kwargs: Dict[str, Any] | None = None)[source]

Bases: BaseModel

Configuration for mapchete-eo drivers.

area: MPathLike | dict | type[BaseGeometry] | None
cache: Any | None
cat_baseurl: str | None
classmethod deprecate_cat_baseurl(values: Dict[str, Any]) → Dict[str, Any][source]
footprint_buffer: float
format: str
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

preprocessing_tasks: bool
search_kwargs: Dict[str, Any] | None
source: Sequence[Source]
time: TimeRange | List[TimeRange] | None
classmethod to_list(values: Dict[str, Any]) → Dict[str, Any][source]

Expands source to list.

class mapchete_eo.base.EODataCube(tile: BufferedTile, products: List[EOProductProtocol] | None, eo_bands: dict, time: List[TimeRange] | None = None, input_key: str | None = None, area: BaseGeometry | None = None, **kwargs)[source]

Bases: InputTile

Target Tile representation of input data.

area: BaseGeometry
area_pixelbuffer: int = 0
default_read_merge_method: MergeMethod = 'first'
default_read_merge_products_by: str | None = None
default_read_nodataval: NodataVal = None
default_read_resampling: Resampling = 0
default_read_values(resampling: Resampling | str | None = None, nodatavals: List[float | None] | float | None = None, merge_products_by: str | None = None, merge_method: MergeMethod | None = None) → dict[source]

Provide proper read values depending on user input and defaults.

eo_bands: dict
filter_products(start_time: str | date | datetime | None = None, end_time: str | date | datetime | None = None, timestamps: List[str | date | datetime] | None = None, time_pattern: str | None = None)[source]

Return a filtered list of input products.

get_read_mask() → ndarray[source]

Determine read mask according to input area.

This will generate a numpy array where pixel overlapping the input area are set True and thus will get filled by the read function. Pixel outside of the area are not considered for reading.

On staged reading, i.e. first checking the product masks to assess valid pixels, this will avoid reading product bands in cases the product only covers pixels outside of the intended reading area.

is_empty() → bool[source]

Check if there is data within this tile.

Returns:

is empty

Return type:

bool

property products: IndexedFeatures[EOProductProtocol][source]

Indexed products.

read(assets: List[str] | None = None, eo_bands: List[str] | None = None, start_time: str | date | datetime | None = None, end_time: str | date | datetime | None = None, timestamps: List[str | date | datetime] | None = None, time_pattern: str | None = None, resampling: Resampling | str | None = None, merge_products_by: str | None = None, merge_method: MergeMethod | None = None, sort: SortMethodConfig | None = None, nodatavals: List[float | None] | float | None = None, raise_empty: bool = True, **kwargs) → Dataset[source]

Read input data into an xarray.Dataset.

read_levelled(target_height: int, assets: ~typing.List[str] | None = None, eo_bands: ~typing.List[str] | None = None, start_time: str | ~datetime.date | ~datetime.datetime | None = None, end_time: str | ~datetime.date | ~datetime.datetime | None = None, timestamps: ~typing.List[str | ~datetime.date | ~datetime.datetime] | None = None, time_pattern: str | None = None, resampling: ~rasterio.enums.Resampling | str | None = None, nodatavals: ~typing.List[float | None] | float | None = None, merge_products_by: str | None = None, merge_method: ~mapchete_eo.types.MergeMethod | None = None, sort: ~mapchete_eo.sort.SortMethodConfig = TargetDateSort(func=<function sort_objects_by_target_date>, target_date=None, reverse=False), raise_empty: bool = True, slice_axis_name: str = 'layers', band_axis_name: str = 'bands', x_axis_name: str = 'x', y_axis_name: str = 'y', **kwargs) → Dataset[source]
read_levelled_np_array(target_height: int, assets: ~typing.List[str] | None = None, eo_bands: ~typing.List[str] | None = None, start_time: str | ~datetime.date | ~datetime.datetime | None = None, end_time: str | ~datetime.date | ~datetime.datetime | None = None, timestamps: ~typing.List[str | ~datetime.date | ~datetime.datetime] | None = None, time_pattern: str | None = None, resampling: ~rasterio.enums.Resampling | str | None = None, nodatavals: ~typing.List[float | None] | float | None = None, merge_products_by: str | None = None, merge_method: ~mapchete_eo.types.MergeMethod | None = None, sort: ~mapchete_eo.sort.SortMethodConfig = TargetDateSort(func=<function sort_objects_by_target_date>, target_date=None, reverse=False), raise_empty: bool = True, **kwargs) → MaskedArray[source]

Read levelled data (cubes with depth) as a MaskedArray.

Parameters:

  • target_height – Target stack height.

  • assets – List of asset names.

  • eo_bands – List of EO bands.

  • start_time – Start time.

  • end_time – End time.

  • timestamps – List of timestamps.

  • time_pattern – Time pattern.

  • resampling – Resampling method.

  • nodatavals – Nodata values.

  • merge_products_by – Property to merge by.

  • merge_method – Merge method.

  • sort – Sorting configuration.

  • raise_empty – Raise error if no data found.

Returns:

Output data array.

Return type:

ma.MaskedArray

read_masks(start_time: str | date | datetime | None = None, end_time: str | date | datetime | None = None, timestamps: List[str | date | datetime] | None = None, time_pattern: str | None = None, nodatavals: List[float | None] | float | None = None, **kwargs)[source]

Read product masks.

Parameters:

  • start_time – Start time.

  • end_time – End time.

  • timestamps – List of timestamps.

  • time_pattern – Time pattern.

  • nodatavals – Nodata values.

Returns:

Mask data.

Return type:

ma.MaskedArray

read_np_array(assets: List[str] | None = None, eo_bands: List[str] | None = None, start_time: str | date | datetime | None = None, end_time: str | date | datetime | None = None, timestamps: List[str | date | datetime] | None = None, time_pattern: str | None = None, resampling: Resampling | str | None = None, merge_products_by: str | None = None, merge_method: MergeMethod | None = None, sort: SortMethodConfig | None = None, nodatavals: List[float | None] | float | None = None, raise_empty: bool = True, **kwargs) → MaskedArray[source]

Read input data as a MaskedArray.

tile: BufferedTile
time: List[TimeRange] | None
class mapchete_eo.base.InputData(input_params: dict, readonly: bool = False, input_key: str | None = None, standalone: bool = False, **kwargs)[source]

Bases: InputData

Main driver class used by mapchete to handle input data discovery and indexing.

area: BaseGeometry
bbox(out_crs: str | None = None) → BaseGeometry[source]

Return data bounding box.

cleanup()[source]

Optional cleanup function called when Mapchete exits.

static default_preprocessing_task(item: Item, **kwargs) → EOProduct
driver_config_model

alias of BaseDriverConfig

open(tile, **kwargs) → EODataCube[source]

Return InputTile object.

params: BaseDriverConfig
property products: IndexedFeatures[source]

Hold preprocessed S2Products in an IndexedFeatures container.

source_items() → Generator[Item, None, None][source]
time: TimeRange | List[TimeRange] | None

mapchete_eo.eostac module

Driver class for EOSTAC static STAC catalogs.

class mapchete_eo.eostac.InputData(input_params: dict, readonly: bool = False, input_key: str | None = None, standalone: bool = False, **kwargs)[source]

Bases: InputData

In case this driver is used when being a readonly input to another process.

input_tile_cls

alias of InputTile

class mapchete_eo.eostac.InputTile(tile: BufferedTile, products: List[EOProductProtocol] | None, eo_bands: dict, time: List[TimeRange] | None = None, input_key: str | None = None, area: BaseGeometry | None = None, **kwargs)[source]

Bases: EODataCube

Target Tile representation of input data.

Parameters:

  • tile (Tile)

  • kwargs (keyword arguments) – driver specific parameters

mapchete_eo.exceptions module

Custom exceptions.

exception mapchete_eo.exceptions.AllMasked[source]

Bases: Exception

Raised when an array is fully masked.

exception mapchete_eo.exceptions.AssetEmpty[source]

Bases: AssetError

Raised when a product asset should contain data but is empty.

exception mapchete_eo.exceptions.AssetError[source]

Bases: Exception

Generic Exception class for Assets.

exception mapchete_eo.exceptions.AssetKeyError[source]

Bases: AssetError, KeyError

Raised when an asset name cannot be found in item.

exception mapchete_eo.exceptions.AssetMissing[source]

Bases: AssetError, FileNotFoundError

Raised when a product asset should be there but isn’t.

exception mapchete_eo.exceptions.BRDFError[source]

Bases: Exception

Raised when BRDF grid cannot be calculated.

exception mapchete_eo.exceptions.CorruptedGTiffError[source]

Bases: Exception

Raised when GTiff validation fails.

exception mapchete_eo.exceptions.CorruptedProduct[source]

Bases: Exception

Raised when product is damaged and cannot be read.

exception mapchete_eo.exceptions.CorruptedProductMetadata[source]

Bases: CorruptedProduct

Raised when EOProduct cannot be parsed due to a metadata issue.

exception mapchete_eo.exceptions.CorruptedSlice[source]

Bases: Exception

Raised when all products in a slice are damaged and cannot be read.

exception mapchete_eo.exceptions.EmptyCatalogueResponse[source]

Bases: Exception

Raised when catalogue response is empty.

exception mapchete_eo.exceptions.EmptyFileException[source]

Bases: Exception

Raised when no bytes are downloaded.

exception mapchete_eo.exceptions.EmptyFootprintException[source]

Bases: Exception

Raised when footprint is empty.

exception mapchete_eo.exceptions.EmptyProductException[source]

Bases: EmptySliceException

Raised when product is empty.

exception mapchete_eo.exceptions.EmptySliceException[source]

Bases: Exception

Raised when slice is empty.

exception mapchete_eo.exceptions.EmptyStackException[source]

Bases: MapcheteNodataTile

Raised when whole stack is empty.

exception mapchete_eo.exceptions.IncompleteDownloadException[source]

Bases: Exception

“Raised when the file is not downloaded completely.

exception mapchete_eo.exceptions.InvalidMapcheteEOCollectionError[source]

Bases: Exception

“Raised for unsupported collections of Mapchete EO package.

exception mapchete_eo.exceptions.ItemGeometryError[source]

Bases: Exception

Raised when STAC item geometry cannot be resolved.

exception mapchete_eo.exceptions.NoSourceProducts[source]

Bases: MapcheteNodataTile, ValueError

Raised when no products are available.

exception mapchete_eo.exceptions.PreprocessingNotFinished[source]

Bases: Exception

Raised when preprocessing tasks have not been fully executed.

mapchete_eo.product module

class mapchete_eo.product.EOProduct(item: Item)[source]

Bases: EOProductProtocol

Wrapper class around a STAC Item which provides data reading capabilities.

assets_eo_bands_to_band_locations(assets: List[str] | None = None, eo_bands: List[str] | None = None) → List[BandLocation][source]
clear_cached_data()[source]
default_dtype

alias of uint16

empty_array(count: int, grid: GridProtocol, fill_value: int = 0, dtype: dtype[Any] | None | type[Any] | _SupportsDType[dtype[Any]] | str | tuple[Any, int] | tuple[Any, SupportsIndex | Sequence[SupportsIndex]] | list[Any] | _DTypeDict | tuple[Any, Any] = None) → MaskedArray[source]
eo_bands_to_band_location(eo_bands: List[str]) → List[BandLocation][source]
classmethod from_stac_item(item: Item, **kwargs) → EOProduct[source]
get_mask() → ReferencedRaster[source]
get_property(property: str) → Any[source]
id: str
property item: Item
read(assets: List[str] | None = None, eo_bands: List[str] | None = None, grid: GridProtocol | None = None, resampling: Resampling = Resampling.nearest, nodatavals: List[float | None] | float | None = None, x_axis_name: str = 'x', y_axis_name: str = 'y', raise_empty: bool = True, **kwargs) → Dataset[source]

Read bands and assets into an xarray.Dataset.

Parameters:

  • assets – List of asset names.

  • eo_bands – List of EO band names.

  • grid – Target grid protocol.

  • resampling – Resampling algorithm.

  • nodatavals – Custom nodata values.

  • x_axis_name – Name of X axis in output.

  • y_axis_name – Name of Y axis in output.

  • raise_empty – Raise exception if no data is found.

Returns:

Dataset with assets as data variables.

Return type:

xr.Dataset

read_np_array(assets: List[str] | None = None, eo_bands: List[str] | None = None, grid: GridProtocol | None = None, resampling: Resampling = Resampling.nearest, nodatavals: List[float | None] | float | None = None, raise_empty: bool = True, apply_offset: bool = True, **kwargs) → MaskedArray[source]

Read assets or EO bands into a MaskedArray.

Parameters:

  • assets – List of asset names.

  • eo_bands – List of EO band names.

  • grid – Target grid.

  • resampling – Resampling method.

  • nodatavals – Nodata values.

  • raise_empty – Raise if empty.

  • apply_offset – Apply offset/scale metadata if present.

Returns:

Output array.

Return type:

ma.MaskedArray

mapchete_eo.product.add_to_blacklist(path: str | PathLike, blacklist: MPath | None = None) → None[source]
mapchete_eo.product.blacklist_products(blacklist: str | PathLike | None = None) → Set[str][source]
mapchete_eo.product.eo_bands_to_band_locations(item: Item, eo_bands: List[str], role: Literal['data', 'reflectance', 'visual'] = 'data') → List[BandLocation][source]

Map EO band names to asset locations.

Parameters:

  • item – STAC Item.

  • eo_bands – List of common band names.

  • role – Functional role of the assets.

Returns:

List of location objects.

Return type:

List[BandLocation]

mapchete_eo.product.find_eo_band(item: Item, eo_band_name: str, role: Literal['data', 'reflectance', 'visual'] = 'data') → BandLocation[source]

Tries to find the location of the most appropriate band using the EO band name.

This function looks into all assets and all eo bands for the given name and role.

mapchete_eo.protocols module

class mapchete_eo.protocols.DateTimeProtocol(*args, **kwargs)[source]

Bases: Protocol

datetime: str | date | datetime
class mapchete_eo.protocols.EOProductProtocol(*args, **kwargs)[source]

Bases: Protocol

bounds: Bounds
crs: CRS
classmethod from_stac_item(item: Item, **kwargs) → EOProductProtocol[source]
get_mask() → ReferencedRaster[source]
get_property(property: str) → Any[source]
id: str
property item: Item
read(assets: List[str] | None = None, eo_bands: List[str] | None = None, grid: GridProtocol | None = None, resampling: Resampling = Resampling.nearest, nodatavals: List[float | None] | float | None = None, x_axis_name: str = 'x', y_axis_name: str = 'y', **kwargs) → Dataset[source]
read_np_array(assets: List[str] | None = None, eo_bands: List[str] | None = None, grid: GridProtocol | None = None, resampling: Resampling = Resampling.nearest, nodatavals: List[float | None] | float | None = None, **kwargs) → MaskedArray[source]
class mapchete_eo.protocols.GetPropertyProtocol(*args, **kwargs)[source]

Bases: Protocol

get_property(property: str) → Any[source]

mapchete_eo.settings module

class mapchete_eo.settings.Settings(_case_sensitive: bool | None = None, _nested_model_default_partial_update: bool | None = None, _env_prefix: str | None = None, _env_file: DotenvType | None = PosixPath('.'), _env_file_encoding: str | None = None, _env_ignore_empty: bool | None = None, _env_nested_delimiter: str | None = None, _env_nested_max_split: int | None = None, _env_parse_none_str: str | None = None, _env_parse_enums: bool | None = None, _cli_prog_name: str | None = None, _cli_parse_args: bool | list[str] | tuple[str, ...] | None = None, _cli_settings_source: CliSettingsSource[Any] | None = None, _cli_parse_none_str: str | None = None, _cli_hide_none_type: bool | None = None, _cli_avoid_json: bool | None = None, _cli_enforce_required: bool | None = None, _cli_use_class_docs_for_groups: bool | None = None, _cli_exit_on_error: bool | None = None, _cli_prefix: str | None = None, _cli_flag_prefix_char: str | None = None, _cli_implicit_flags: bool | None = None, _cli_ignore_unknown_args: bool | None = None, _cli_kebab_case: bool | Literal['all', 'no_enums'] | None = None, _cli_shortcuts: Mapping[str, str | list[str]] | None = None, _secrets_dir: PathType | None = None, *, default_cache_location: str | ~os.PathLike = <mapchete.io.MPath object: s3://eox-mhub-cache/, storage_options={'asynchronous': False, 'timeout': None}>, default_catalog_crs: ~rasterio.crs.CRS = CRS.from_epsg(4326), blacklist: str | ~os.PathLike | None = None, lazy_load_stac_items: bool = True)[source]

Bases: BaseSettings

Combine default settings with env variables.

All settings can be set in the environment by adding the ‘MHUB_’ prefix and the settings in uppercase, e.g. MAPCHETE_EO_.

blacklist: str | PathLike | None
default_cache_location: str | PathLike
default_catalog_crs: CRS
lazy_load_stac_items: bool
model_config: ClassVar[SettingsConfigDict] = {'arbitrary_types_allowed': True, 'case_sensitive': False, 'cli_avoid_json': False, 'cli_enforce_required': False, 'cli_exit_on_error': True, 'cli_flag_prefix_char': '-', 'cli_hide_none_type': False, 'cli_ignore_unknown_args': False, 'cli_implicit_flags': False, 'cli_kebab_case': False, 'cli_parse_args': None, 'cli_parse_none_str': None, 'cli_prefix': '', 'cli_prog_name': None, 'cli_shortcuts': None, 'cli_use_class_docs_for_groups': False, 'enable_decoding': True, 'env_file': None, 'env_file_encoding': None, 'env_ignore_empty': False, 'env_nested_delimiter': None, 'env_nested_max_split': None, 'env_parse_enums': None, 'env_parse_none_str': None, 'env_prefix': 'MAPCHETE_EO_', 'extra': 'forbid', 'json_file': None, 'json_file_encoding': None, 'nested_model_default_partial_update': False, 'protected_namespaces': ('model_validate', 'model_dump', 'settings_customise_sources'), 'secrets_dir': None, 'toml_file': None, 'validate_default': True, 'yaml_config_section': None, 'yaml_file': None, 'yaml_file_encoding': None}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

mapchete_eo.sort module

This module holds all code required to sort products or slices.

class mapchete_eo.sort.CloudCoverSort(*, func: ~typing.Callable = <function sort_objects_by_cloud_cover>, reverse: bool = False)[source]

Bases: SortMethodConfig

func: Callable
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

reverse: bool
class mapchete_eo.sort.SortMethodConfig(*, func: Callable)[source]

Bases: BaseModel

func: Callable
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class mapchete_eo.sort.TargetDateSort(*, func: ~typing.Callable = <function sort_objects_by_target_date>, target_date: str | ~datetime.date | ~datetime.datetime | None = None, reverse: bool = False)[source]

Bases: SortMethodConfig

func: Callable
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

reverse: bool
target_date: str | date | datetime | None
mapchete_eo.sort.sort_objects_by_cloud_cover(objects: List[GetPropertyProtocol], reverse: bool = False) → List[GetPropertyProtocol][source]
mapchete_eo.sort.sort_objects_by_target_date(objects: List[DateTimeProtocol], target_date: str | date | datetime | None = None, reverse: bool = False, **kwargs) → List[DateTimeProtocol][source]

Sort objects by their distance to a target date.

mapchete_eo.source module

class mapchete_eo.source.Source(*, collection: str, catalog_crs: CRS | CRS | None = CRS.from_epsg(4326), query: str | None = None, area: str | PathLike | dict | type[BaseGeometry] | None = None, bounds: List[float] | Tuple[float, float, float, float] | dict | Polygon | None = None)[source]

Bases: BaseModel

All information required to consume EO products.

apply_item_modifier_funcs(item: Item) → Item[source]
area: str | PathLike | dict | type[BaseGeometry] | None
bounds: List[float] | Tuple[float, float, float, float] | dict | Polygon | None
catalog_crs: CRS | CRS | None
property catalog_type: Literal['search', 'static'][source]
collection: str
classmethod deprecate_max_cloud_cover(values: Dict[str, Any]) → Dict[str, Any][source]
eo_bands(base_dir: str | PathLike | None = None) → List[str][source]
get_catalog(base_dir: str | PathLike | None = None) → CollectionSearcher[source]
property item_modifier_funcs: List[Callable]
model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

query: str | None
search(time: TimeRange | List[TimeRange] | None = None, bounds: List[float] | Tuple[float, float, float, float] | dict | Polygon | None = None, area: BaseGeometry | None = None, base_dir: str | PathLike | None = None) → Generator[Item, None, None][source]

mapchete_eo.time module

mapchete_eo.time.day_range(start_date: datetime | date, end_date: datetime | date) → List[date][source]

Return a list of dates between start and end.

mapchete_eo.time.time_ranges_intersect(t1: Tuple[str | date | datetime, str | date | datetime], t2: Tuple[str | date | datetime, str | date | datetime]) → bool[source]

Check if two time ranges intersect.

mapchete_eo.time.timedelta(date: str | date | datetime, target: str | date | datetime, seconds: bool = True)[source]

Return difference between two time stamps in seconds or days.

mapchete_eo.time.to_datetime(t: str | date | datetime, append_time='min') → datetime[source]

Convert input into datetime object.

mapchete_eo.types module

class mapchete_eo.types.BandLocation(asset_name: str, band_index: ~typing.Annotated[int, ~annotated_types.Gt(gt=0)] = 1, nodataval: float = 0, roles: ~typing.List[str] = <factory>, eo_band_name: str | None = None)[source]

Bases: object

A class representing the location of a specific band.

asset_name: str
band_index: Annotated[int, Gt(gt=0)] = 1
eo_band_name: str | None = None
from_asset(name: str, band_index: Annotated[int, Gt(gt=0)]) → BandLocation[source]

Extract band location info from STAC Asset.

nodataval: float = 0
roles: List[str]
class mapchete_eo.types.GeodataType(new_class_name, /, names, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: str, Enum

Type of geodata (vector or raster).

class mapchete_eo.types.MergeMethod(*values)[source]

Bases: str, Enum

Available methods to merge assets from multiple items.

first: first pixel value from the list is returned average: average value from the list is returned all: any consecutive value is added and all collected are returned

all = 'all'
average = 'average'
first = 'first'
class mapchete_eo.types.TimeRange(start: str | date | datetime, end: str | date | datetime)[source]

Bases: object

A class handling time ranges.

end: str | date | datetime
start: str | date | datetime

Module contents