pywatemsedem.geo package

Submodules

pywatemsedem.geo.factory module

class pywatemsedem.geo.factory.Factory(resolution, epsg_code, nodata, resmap, bounds=None)

Bases: object

Factory class

Used to enable functions to generate vectors and rasters.

Notes

By default a rasterproperties instance is made in the initialisation See pywatemsedem.geo.factory.create_mask()-function. This can be toggled of by setting pywatemsedem.geo.factory.Factory.create_rasterproperties to False

create_mask(mask)

Create mask based on a mask template (raster or vector)

Parameters:

file_path (pathlib.Path | str) – File path to mask vector raster file

Notes

If pywatemsedem.geo.factory.Factory.create_rasterproperties is set to False, one needs to self-define a RasterProperties instance.

property mask

AbstractRaster mask

raster_factory(raster_input, flag_clip=True, flag_mask=True, allow_nodata_array=False)

Raster factory to load rasters in memory

Parameters:

• raster_input (str, pathlib.Path or numpy.ndarray) – Input raster file or numpy array

• flag_clip (bool, default True) – Clip raster (True)

• flag_mask (bool, default True) – Mask raster (True)

• allow_nodata_array (default False) – Allow the returned array to only contian nodata-values, see pywatemsedem.geo.rasters.AbstractRaster.mask().

Returns:

raster – See pywatemsedem.geo.rasters.AbstractRaster

Return type:

pywatemsedem.geo.rasters.AbstractRaster

property rp

RasterProperties. See pywatemsedem.geo.rasterproperties.RasterProperties

property vct_mask

AbstractVector mask, See pywatemsedem.geo.vectors.AbstractVector

vector_factory(vector_input, geometry_type, allow_empty=False)

Vector factory to load rasters in memory

Parameters:

• vector_input (str, pathlib.Path or geopandas.GeoDataFrame) – Input vector file or geopandas dataframe

• mask (bool, default True) – Mask vector (True), nodata value will be that one of pywatemsedem.geo.factory.Factory.rp.

• allow_empty (bool, default False) – Allow vector to be empty, see pywatemsedem.geo.vectors.AbstractVector

Returns:

vector – See pywatemsedem.geo.rasters.AbstractRaster

Return type:

pywatemsedem.geo.rasters.AbstractRaster

pywatemsedem.geo.factory.valid_mask_factory(func)

Check valid mask inputted when using raster or vectofactory

pywatemsedem.geo.rasterproperties module

class pywatemsedem.geo.rasterproperties.RasterProperties(bounds: List[float], resolution: int, nodata: float, epsg: int, driver: str = 'GTiff')

Bases: object

Raster properties class

Pywatemsedem makes use of rasterio and gdal for loading, writing and processing rasters/vectors. A small class is implemented to easily switch between raster geographic references of gdal and rasterio, respectively names gdal_profile and rasterio_profile. Assuming you start with loading a raster with rasterio:

from pywatemsedem.geo.utils import load_raster
from pywatemsedem.geo.rasterproperties import RasterProperties
arr, _, rasterio_profile = load_raster("test.tif")
rp = RasterProperties.from_rasterio(rasterio_profile)
rp.gdal_profile
...

Note that rasterio_profile is of type dictionary, as such one can start from a profile definition in a dictionary format, for instance for gdal:

from pywatemsedem.geo.utils import load_raster
from pywatemsedem.geo.rasterproperties import RasterProperties
gdal_profile = {
    "nodata": -9999.0,
    "epsg": "EPSG:31370",
    "res": 20.0,
    "minmax": [201620.0, 153880.0, 207500.0, 164060.0],
    "ncols": 294,
    "nrows": 509,
}
rp = RasterProperties.from_gdal(gdal_profile)
rp.rasterio_profile
...

In addition, a RasterProfile instance can be generated from known raster bounds and a raster resolution (this can be useful for generating raster geographical references when no rasterio or gdal profile definition is available):

from pywatemsedem.geo.rasterproperties import RasterProperties

bounds = [201620.0, 153880.0, 207500.0, 164060.0]
resolution = 20
nodata= -9999
epsg = 31370
rp = RasterProperties(bounds, resolution, nodata, epsg)

Notes

1. Current implementation support storing of raster properties, yet is does not aim to provide functionalities to adapt raster properties (as these functionalities are present in rasterio).

2. Current implementation does not support tiled rasters. In addition, it only support bands-interleaving as only single-band raster are used.

3. Definition interleaving: the way multiple bands of a raster are saved to the raster (e.g. pixel-based, line-based, band-based)

4. The coordinate reference system (crs) is defined in EPSG.

5. Note that dtype in gdal operation is typically derived from input raster dtype that is used to execute the gdal operation. As such dtype is not defined as a key in the gdal_profile property.

property bounds: List[float]

Raster boundary coordinates.

Returns:

list – The first entry is x_left, the second entry is y_lower, the third entry is x_right and the fourth entry is y_upper.

Return type:

[x_left, y_lower, x_right, y_upper]

property driver: str

Name of GDAL driver

see https://gdal.org/drivers/raster/index.html; only GTiff | RST | SAGA are supported.

property epsg: int

EPSG code of the raster projection should be a numeric value, see https://epsg.io/

Type:

int

classmethod from_gdal(gdal_profile: dict)

Set RasterProperties from a gdal profile

Parameters:

gdal_profile (dict)

classmethod from_rasterio(rasterio_profile: dict, epsg: int = None)

Set RasterProperties with a rasterio profile

Parameters:

• rasterio_profile (dict)

• epsg (int, default None) – EPSG code should be a numeric value, see https://epsg.io/.

property gdal_profile: dict

Return gdal profile

Returns:

gdal_profile – See definition in this function.

Return type:

dict

property ncols: int

Number of columns in the raster

Return type:

int

property nodata: float

No data value used in raster.

property nrows: int

Number of rows in the raster

Return type:

int

property rasterio_profile: dict

Return rasterio profile

Returns:

rasterio_profile – See definition in this function.

Return type:

dict

property resolution: int

Raster resolution

property xcoord

Returns 1D-vector array of x-coordinates

Return type:

numpy.ndarray

property ycoord

Returns 1D-vector array of y-coordinates

Return type:

numpy.ndarray

pywatemsedem.geo.rasterproperties.get_bounds_from_vct(vct_catchment, resolution, n_pixels_buffer=5)

Get bounds from vector catchment.

Parameters:

• vct_catchment (pathlib.Path) – Vector file of catchment.

• resolution (int) – Spatial resolution, see pywatemsedem.geo.rasterproperties.RasterProperties

• n_pixels_buffer (int, default 5) – Number of pixels that have to be taken into account to expand boundaries

Returns:

bounds – See pywatemsedem.geo.rasterproperties.RasterProperties

Return type:

list

pywatemsedem.geo.rasterproperties.synchronize_bounds(target, source, resolution)

Synchronize target geographical bounds of a raster with source bounds given a

raster resolution.

Bounds are defined as a list of xmin, ymin, xmax and ymax-values of a raster. The boundaries leading to the smallest geographical model are selected as new boundaries.

Parameters:

• target (list) – To check target boundaries, see pywatemsedem.geo.rasterproperties.RasterProperties.

• source (pathlib.Path) – Source boundaries to which target boundaries have to be checked, see pywatemsedem.geo.rasterproperties.RasterProperties.

• resolution (int) – See pywatemsedem.geo.rasterproperties.RasterProperties

Returns:

bounds – Updated bounds

Return type:

list

pywatemsedem.geo.rasters module

class pywatemsedem.geo.rasters.AbstractRaster

Bases: object

Abstract raster class based on numpy arrays and raster properties

Parameters:

• arr (numpy.ndarray) – Raster array

• rp (RasterProperties) – See pywatemsedem.geo.rasterproperties.RasterProperties

• arr_mask (numpy.ndarray, default None) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

• allow_nodata_array (boolean, default False) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

Note

If an array mask is provided, the array is automatically masked.

property arr

Return array

Return type:

numpy.ndarray

clip()

NotImplemented clip function

histogram(fig=None, ax=None, nodata=None, ylogscale=False, *args, **kwargs)

Plot density histogram of raster data values with 25th, 50th and 75th percentiles

Parameters:

• fig (matplotlib.figure.Figure, default = None) – if not given, defaults to generating new figure

• ax (matplotlib.axes.Axes, default = None) – if not given, defaults to generating new axis

• nodata (float) – Used to mask no data values

• ylogscale (bool, default = False) – Log transformation on density axis if True

Returns:

• fig (matplotlib.figure.Figure)

• ax (matplotlib.axes.Axes)

initialize(arr, rp, arr_mask=None, allow_nodata_array=False)

Initialize array and rasterproperties

is_empty()

check if array (raster) is None (empty)

Return type:

True/False

mask(arr_mask, allow_nodata_array=False)

Mask function.

Parameters:

• arr_mask (numpy.ndarray) – Array mask (1, nodata). Note that array mask should have same nodata value as raster array.

• allow_nodata_array (bool) – Allow to return a nodata-array.

plot(fig=None, ax=None, nodata=None, *args, **kwargs)

Plot raster array with imshow

Parameters:

• fig (matplotlib.figure.Figure, default = None) – if not given, defaults to generating new figure

• ax (matplotlib.axes.Axes, default = None) – if not given, defaults to generating new axis

• nodata (float, default = None) – Used to mask certain values present in arr which represent nodata (e.g. -9999)

Returns:

• fig (matplotlib.figure.Figure)

• ax (matplotlib.axes.Axes)

property rp

Return raster properties

Return type:

pywatemsedem.geo.rasterproperties.RasterProperties

update_nodata_value(to)

Update the nodata value

Parameters:

to (float) – New nodata value

write(outfile_path, format='idrisi', dtype=None, nodata=None)

Write raster data to disk.

Parameters:

• outfile_path (pathlib.Path or str) – File path output

• format ("idrisi" or "tiff", default "idrisi")

• dtype (numpy.dtype, default None) – Output raster type. If None, dtype of array is used.

• nodata (float, default None) – Nodata value for output raster. If None, nodata of rasterproperties is used.

class pywatemsedem.geo.rasters.RasterFile(file_path, rp=None, arr_mask=None, allow_nodata_array=False)

Bases: AbstractRaster

Array based on a input raster file.

Parameters:

• file_path (pathlib.Path) – File path to user input raster.

• rp (RasterProperties, default None) – See pywatemsedem.geo.rasterproperties.RasterProperties. If None, rasterproperties from inputfile are used.

• arr_mask (numpy.ndarray, default None) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

• allow_nodata_array (bool, default False) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

Note

If rasterproperties are inputted by the user, clipping is automatically done.

static clip(file_path, rp, resample='mode')

Clip function

Parameters:

• file_path (pathlib.Path) – File path to user input raster.

• rp (RasterProperties) – See pywatemsedem.geo.rasterproperties.RasterProperties

• resample (str, default "mode") – Either “near” or “mode”, see pywatemsedem.geo.utils.clip_rst()

Returns:

arr – Clipped array

Return type:

numpy.ndarray

Notes

Clipping also provides resampling to another resolution. See pywatemsedem.geo.utils.clip_rst().

class pywatemsedem.geo.rasters.RasterMemory(arr, rp, arr_mask=None, allow_nodata_array=False)

Bases: AbstractRaster

Array raster

Parameters:

• arr (numpy.ndarray) – See pywatemsedem.geo.rasters.AbstractRaster

• rp (RasterProperties) – See pywatemsedem.geo.rasterproperties.RasterProperties

• arr_mask (numpy.ndarray, default None) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

• allow_nodata_array (bool, default False) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

clip()

NotImplemented

class pywatemsedem.geo.rasters.TemporalRaster(arr, rp, arr_mask=None)

Bases: object

3-D raster with first two dimension spatial x and y dimension, with the third dimension being temporal.

Parameters:

• arr (numpy.ndarray) – 3D Raster array

• rp (RasterProperties) – See pywatemsedem.geo.rasterproperties.RasterProperties

• arr_mask (np.ndarray, default None) – See pywatemsedem.geo.rasters.AbstractRaster.mask()

Note

This class only works with array inputs!

property arr

Return array :rtype: numpy.ndarray

plot(nodata=None, *args, **kwargs)

Sequential plot raster in cols

Parameters:

nodata (float)

write(outfiles, format='idrisi', dtype=None)

Write function.

Parameters:

• outfiles (list or tuple of pathlib.Path or str) – List of output files

• format (basestring) – See pywatemsedem.geo.rasters.AbstractRaster.write()

• dtype (numpy.dtype) – See pywatemsedem.geo.rasters.AbstractRaster.write()

pywatemsedem.geo.utils module

pywatemsedem.geo.utils.add_length_lines_to_polygons(vct_lines, vct_polygons, vct_out, name_field)

Calculate the total length of line segments within a polygon

Parameters:

• vct_lines (str or pathlib.Path) – File path of the input line shapefile

• vct_polygons (str or pathlib.Path) – File path of the input polygon shapefile

• vct_out (str or pathlib.Path) – File path of the output polygon shapefile

• name_field (str) – Attribute name containing the lenght of the lines within a polygon

pywatemsedem.geo.utils.any_equal_element_in_vector(geoseries_left, geoseries_right)

Check if there are equal vectors in the left and right geoseries

Parameters:

• geoseries_left (geopandas.GeoSeries)

• geoseries_right (geopandas.GeoSeries)

Returns:

Line strings present in left and right geoseries

Return type:

bool

pywatemsedem.geo.utils.calculate_sum_rst(rst)

Calculate sum of a raster values

Parameters:

rst (pathlib.Path) – File path of the raster file

Returns:

sum of the values in the raster

Return type:

float

pywatemsedem.geo.utils.check_cuboid_condition(arr_polygon_perimeter, arr_polygon_area)

Check if a shape can be classified as a cuboid

Parameters:

• arr_polygon_perimeter (numpy.ndarray or pandas.Series) – 1D array holding in each row the perimeter of each polygon

• arr_polygon_area (numpy.ndarray or pandas.Series) – 1D array holding in each row the area of each polygon

Returns:

condition – 1D array holding True/False if cuboid

Return type:

numpy.ndarray or pandas.Series

See also

estimate_width_of_polygon

pywatemsedem.geo.utils.check_raster_properties_raster_with_template(path_check, path_template, epsg)

Checks if extent and resolution of new raster and template raster align

Parameters:

• path_check (pathlib.Path | RasterProperties) – Incoming, new raster

• path_template (pathlib.Path) – Template raster

• epsg (int) – EPSG code of the cartographic projection

pywatemsedem.geo.utils.check_rst_dimensions(rst_in, minmax, ncols, nrows, transform=None)

This function checks if the input raster has the desired dimensions.

Parameters:

• rst_in (str) – File path to input raster

• minmax (list) – Containing xmin, ymin, xmax, ymax

• ncols (int) – Number of columns in the raster

• nrows (int) – Number of rows in the raster

• transform (rasterio.transform, default None) – Transformation as defined in Rasterio.

Returns:

Raster has the required dimensions (True/False)

Return type:

bool

pywatemsedem.geo.utils.check_single_polygon(vct)

Check if the catchment polygon is a single polygon (not empty or multipolygon).

Parameters:

vct (str or pathlib.Path) – File path of the shapefile

pywatemsedem.geo.utils.check_spatial_resolution_rst(rst_in, resolution, precision=0.01)

Check if the resolution of a tiff raster is equal to a defined one

Parameters:

• rst_in (str or pathlib.Path) – File path of input raster

• resolution (int) – Resolution to compare with

• precision (float, default 0.01) – Precision to check resolution

Returns:

cond – Equal/non-equal (True/False)

Return type:

bool

pywatemsedem.geo.utils.clean_up_tempfiles(temporary_file, file_format)

Clean up extra generated tempfiles

This function can be used to clean-up extra files (for example auxilary files) generated during a write of raster or vector file.

Parameters:

• temporary_file (pathlib.Path | str)

• file_format ({"tiff","shp","rst","txt"}) – format of the temporary files

pywatemsedem.geo.utils.clip_rst(rst_in, rst_out, Cnst, resampling='near')

Clips a raster to a certain bounding box with a given resolution

Parameters:

• rst_in (str) – File path to in input raster.

• rst_out (str) – File path to the destination raster.

• Cnst (dict) –

  Dictionary with following keys:

  • epsg (str): the EPSG-code of the rst_in

  • res (int): resolution

  • nodata (int): nodata flag

  • minmax (list): list with xmin, ymin, xmax, ymax

• resampling (str, default 'near') – Either “mode” or “near”

Notes

1. “mode” and “near” have been tested, see

   https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-r

pywatemsedem.geo.utils.clip_vct(vct_in, vct_out, vct_clip, overwrite=False, lst_ignore_field=None)

Clip a shapefile by another shapefile

Parameters:

• vct_in (pathlib.Path) – File path of shapefile to be clipped.

• vct_out (pathlib.Path) – File path of the destination shapefile

• vct_clip (str) – File path of the clip boundary vector

• overwrite (bool, default False) – If True, overwrite existing file

• lst_ignore_field (list, optional) – Ignore a specific field from input in output.

Note

Uses and relies on ogr2ogr CLI

pywatemsedem.geo.utils.clip_vct_with_bounds(vct_in, vct_out, bounds, overwrite=False)

Clip a shapefile using a bounding box

Parameters:

• vct_in (pathlib.Path) – File path of shapefile to be clipped.

• vct_out (pathlib.Path) – File path of the destination shapefile

• bounds (list) – list with xmin, ymin, xmax, ymax

• overwrite (bool, default False) – if True, overwrite existing file

Note

Uses and relies on ogr2ogr CLI

pywatemsedem.geo.utils.compute_statistics_rasters_per_polygon_vector(lst_rasters, vct_polygon, vct_out, lst_names, dict_operators, normalize=True, ton=False)

Compute statistics

Parameters:

• lst_rasters (list) – File path rasters (pathlib.Path)

• vct_polygon (str) – File path to input polygon vector.

• vct_out (str) – File path to output polygon vector.

• lst_names (dict) – List of output names in output vector for files in lst_rasters.

• dict_operators (dict) – Operators. For a description of the dictionary of statistics, see inputs in pywatemsedem.geo.utils.grid_statistics(). See example for use.

• normalize (bool, default False) – Normalize with shape area (True)

• ton (bool, default False) – Use ton (true)

Note

The desired statistics are inputted after the file reference of the raster, and are formatted as a dictionary, e.g.:

dict_operators = {"COUNT":True,"SUM":True}

Examples

>>> vct_aho = "AHO.shp"
>>> vct_out = "statistics_aho.shp"
>>> rst_sewerin ="sewerin.rst"
>>> rst_sediexport ="SediExport.rst"
>>> compute_statistics_rasters_per_polygon_vector(vct_aho,
>>>                                                   vct_out,
>>>                                                   [rst_sewerin,rst_sediexport]
>>>                                                   ["River","Sewers"],
>>>                                                   {"COUNT":True,"SUM":True},
>>>                                                   ton = True)

pywatemsedem.geo.utils.copy_rst(rst_in, rt_out)

Copy a raster and converts it to an idrisi-raster

Parameters:

• rst_in (str) – File path of input raster

• rt_out (str) – File path of output raster (extension must be .rst!)

Note

Uses and relies on gdal_translate CLI

pywatemsedem.geo.utils.copy_vct(vct_in, folder_out)

Copies a shapefile to another location

Parameters:

• vct_in (str or pathlib.Path) – File path of the shapefile to be copied.

• folder_out (str or pathlib.Path) – File path of the destination shapefile.

Note

Uses and relies on ogr2ogr CLI

pywatemsedem.geo.utils.create_hillshade(rst_in, rst_hillshade)

Create a hillshade raster of the DTM

Parameters:

• rst_in (str) – File path of input raster

• rst_hillshade (str) – File path of output raster

pywatemsedem.geo.utils.create_spatial_index(vct_in)

Creates a qix-file for a given shapefile

Parameters:

vct_in (str or pathlib.Path) – File path of the input shapefile

Note

Uses and relies on ogrinfo CLI

pywatemsedem.geo.utils.define_extent_from_vct(vct_catchment, resolution, nodata, epsg, bounds=None, buffer=100)

Read the extent of the catchment

Parameters:

• vct_catchment (str or pathlib.Path) – File path of the vector shapefile

• bounds (list, default None) – if None, bounds are determined from The first entry is x_left, the second entry is y_lower, the third entry is x_right and the fourth entry is y_upper [x_left, y_lower, x_right, y_upper].

• resolution (int) – Spatial resolution

• epsg (int) – EPSG code should be a numeric value, see https://epsg.io/.

Returns:

rp

Return type:

RasterProperties see pywatemsedem.geo.rasterproperties.RasterProperties

pywatemsedem.geo.utils.delete_rst(rst_in)

Delete a raster dataset

Parameters:

rst_in (str) – File path of the raster dataset to be deleted

pywatemsedem.geo.utils.estimate_width_of_polygon(arr_polygon_perimeter, arr_polygon_area, nan_value=nan)

Estimate the width of a polygon with the length and area of the polygon see also https://gis.stackexchange.com/questions/20279/calculating-average-width-of-polygon/

Parameters:

• arr_polygon_perimeter (numpy.ndarray or pandas.Series) – 1D array holding in each row the perimeter of each polygon

• arr_polygon_area (numpy.ndarray or pandas.Series) – 1D array holding in each row the area of each polygon

• nan_value (float) – The value to fill in for nan_values

Returns:

arr_est_polygon_width – 1D array holding in each row the estimated width

Return type:

numpy.ndarray or pandas.Series

Note

1. The width is only estimated for polygons which approximate a cuboid

2. The width is estimated by

\[B_gr = 1/4 [A-√(B-C)] 1/4 [P_{poly}-√(〖P_{poly}〗^2-16A_{poly} )]\]

with

• \(P_{poly}\) = perimeter of polygon

• \(A_{poly}\) = area of polygon

See also

check_cuboid_condition

pywatemsedem.geo.utils.execute_saga(cmd_args)

Run saga executable and catch non-informative error

This function catches saga error command and ignores ‘corrupted size vs. prev_size in fastbins’ error in case output runs from saga are okay.

Parameters:

saga_cmd (list) – Saga command

pywatemsedem.geo.utils.execute_subprocess(cmd_args)

Run a command line tool

Logs error with command output and error

Parameters:

cmd_args (list) – list with all argmunts that must be given to the console

Return type:

Returns True if function call is successfull.

pywatemsedem.geo.utils.generate_vct_mask_from_raster_mask(rst_catchment, vct_catchment, resolution)

Generate a catchment fileshape from a raster file

Parameters:

• rst_catchment (str or pathlib.Path) – Input raster format of the catchment (can be any type that can be read by rasterio)

• vct_catchment (str or pathlib.Path) – output shapefile format of the catchment

• resolution (int) – resolution of the model run

pywatemsedem.geo.utils.get_extent_vct(vct)

Gets the bounding box coordinates of a shapefile

Parameters:

vct (str or pathlib.Path) – File path of the input shapefile.

Returns:

xmin, ymin, xmax, ymax

Return type:

tuple

pywatemsedem.geo.utils.get_feature_count(vct)

Count the amount of features in a shapefile

Parameters:

vct (str) – File path of the shapefile.

Returns:

nr_features – Number of features in the shapefile

Return type:

int

pywatemsedem.geo.utils.get_fields_vct(vct)

Get a list of all fields in a shapefile

Parameters:

vct (str) – File path to shapefile

Returns:

Field names of the shape/vector-file.

Return type:

list

pywatemsedem.geo.utils.get_geometry_type(vct)

Get the geometry type of a shapefile

Parameters:

vct (str) – File path to shapefile

Returns:

Geometry type (see fiona documentation for all possibilities)

Return type:

str

pywatemsedem.geo.utils.get_mask_template(CNWS_modelinputfolder, catchmentname, rst_template=None)

Get a binary raster from template raster (P-factor)

Parameters:

• CNWS_modelinputfolder (str or pathlib.Path) – File path to the CNWS_modelinputfolder

• catchmentname (str) – Catchment name

• rst_template (str or pathlib.Path, default None) – Path to a template file that can be used as template for geodata and bin mask

Returns:

arr_bindomain – In domain is equal to one, outside domain is equal to zero.

Return type:

numpy.ndarray

pywatemsedem.geo.utils.get_rstparams(CNWS_modelinputfolder, epsg=None, catchmentname='', template=None)

Get rstparams and rasterprofile from template raster (default:pkaart)

Parameters:

• CNWS_modelinputfolder (str or pathlib.Path) – the path to the CNWS_modelinputfolder

• epsg (str, default None) – the epsg code defining the coordinate system of the raster, format = “EPSG:XXXXX”

• catchmentname (str, default "") – catchment name

• template (str or pathlib.Path, default None) – File path to a template file that can be used as template for geodata and bin mask. Default the “P” raster is used.

Returns:

• profile (rasterio.profiles) – See rasterio.profiles.Profile

• rstparams (dict) – gdal dictionary holding all metadata for idrisi rasters

• arr_bindomain (numpy.ndarray) – binary mask of modelling domain

pywatemsedem.geo.utils.grid_difference(rst_in1, rst_in2, rst_out)

Make the difference between two grids

Parameters:

• rst_in1 (str) – File path to inputraster 1

• rst_in2 (str) – File path to inputraster 2

• rst_out (str) – File path to outputraster

pywatemsedem.geo.utils.grid_statistics(lst_rst, vct_in, vct_out, naming=0, COUNT=False, MIN=False, MAX=False, RANGES=False, SUM=True, MEAN=False)

Calculate zonal statistics for a raster based on polygons

Parameters:

• lst_rst (list) – File paths (str or pathlib.Path) of the input rasters

• vct_in (str or pathlib.Path) – File path of the input polygon shapefile

• vct_out (str or pathlib.Path) – File path of the output shapefile with the raster statistics

• naming (int, default 0) – 1: grid name (note: esrsi shapes ar capped on 10 characters) 0: grid id

• COUNT (bool, default False) – Count the raster cells within every polygon

• MIN (bool, default False) – Calculate the minimum value within every polygon

• MAX (bool, default False) – Calculate the maximum value within every polygon

• RANGES (bool, default False) – Calculate the range within every polygon

• SUM (bool, default False) – Calculate the sum of all pixel values within every polygon

• MEAN (bool, default False) – Calculate the mean of all pixel values within every polygon

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.idrisi_to_tiff(rst_in, tiff_out, dtype, epsg='EPSG:31370')

Convert an Idrisi RST to GeoTiff

Parameters:

• rst_in (str) – File path of the input rst.

• tiff_out (str) – File path of the output tiff file.

• dtype (str) – Data type of the destination rst, either ‘integer’ or ‘float’.

Note

Uses and relies on gdal_translate CLI

pywatemsedem.geo.utils.lines_to_direction(vct_line, rst_out, rst_template)

Converts line features to a direction raster

This function converts the direction of line features to a raster. See the docs of cnws for more information about this raster

Parameters:

• vct_line (str or pathlib.Path) – File path to the input line shapefile

• rst_out (pathlib.Path) – File path to the output raster

• rst_template (str or pathlib.Path) – File path to a template raster

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.lines_to_points(vct_lines, vct_points, distance)

Converts a shapefile with lines to a shapefile with points.

Parameters:

• vct_lines (str or pathlib.Path) – File path to the input shapefile with lines.

• vct_points (str or pathlib.Path) – File path to the output shapefile with points.

• distance (float) – Distance between two points

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.lines_to_raster(vct_line, rst_out, rst_template, field, dtype)

Converts a line shapefile to a raster

Parameters:

• vct_line (str or pathlib.Path) – File path of input line shapefile

• rst_out (str or pathlib.Path) – File path of output rasterfile

• rst_template (str or pathlib.Path) – File path to a template raster

• field (str) – The field of the shapefile containing the values for the raster

• dtype (str, default None) – Data type of the values, e.g. Byte/Int16/UInt16/UInt32/Int32/Float32…

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.load_discharge_file(filename)

Function to read the discharge file as a dictionary

Parameters:

filename (str or pathlib.Path) – discharge file

Returns:

discharge – contains discharge information

Return type:

dict

pywatemsedem.geo.utils.load_raster(rst, return_bounds=False)

load raster with rasterio

Parameters:

• rst (str or pathlib.Path) – File path of the file, .rst arr.

• return_bounds (bool, default False) – Flag to indicate whether a bounds of the arr should be returned.

Returns:

• arr (numpy.ndarray) – Array format of raster file.

• bounds (list) – List of bounds (xmin,ymin,xmax,ymax).

• profile (rasterio.profiles) – See rasterio.profiles.Profile

pywatemsedem.geo.utils.mask_array_with_val(arr, mask, mask_val)

Masking array by a mask

Parameters:

• arr (numpy.ndarray) – array of which values are masked

• mask (numpy.ndarray) – masking array with same dimensions as arr

• mask_val (float or int) – masking arr when mask is mask_val

pywatemsedem.geo.utils.mask_raster(rst_in, rst_mask, un_id, folder='masked')

Mask a raster by setting no data with no data positions of mask

The input raster is masked by the nodata value in the mask raster.

Parameters:

• rst_in (str or pathlib.Path) – File path raster to be masked

• rst_mask (str or pathlib.Path) – File path mask raster, see note for format

• un_id (int) – unique id of the raster

• folder (str, default 'masked') – Folder in which to safe masked raster

Returns:

rst_masker – File path of masked raster

Return type:

str

Note

1. This mask raster should be a (no nodata, nodata)-raster in which the nodata values indicate masking, and non-nodata values indicates not masking.

2. Masking of the input raster is facilitated by setting to-mask-values in the input raster to nodata.

pywatemsedem.geo.utils.merge_lst_vct(lst_vct, vct_out, epsg)

Merge a list of shapefiles to one shapefile

Parameters:

• lst_vct (list) – List with file paths (str) of all shapefiles to be merged.

• vct_out (str) – File path of merged vct

• epsg (str) – The epsg code defining the coordinate system of the raster, format = “EPSG:XXXXX”

Note

Uses and relies on ogr2ogr CLI

pywatemsedem.geo.utils.merge_lst_vct_saga(lst_vct, vct_out)

Merge a list of shapefiles to one shapefile

Parameters:

• lst_vct (list) – List with file paths (str) of all shapefiles to be merged.

• vct_out (str) – File path + name of the destination shapefile.

Note

Uses and relies on ogr2ogr CLI

pywatemsedem.geo.utils.merge_rasters(lst_rst, rst_out, lst_rst_masks=None)

Merge several rasters to one raster with option to mask input rasters

Parameters:

• lst_rst (list of str or list of pathlib.Path) – List of file paths of rasters which have to be merged together

• rst_out (str or pathlib.Path) – File path of merged raster

• lst_rst_masks (list) – List of file paths of mask files to be used to mask lst_rst_in, see pywatemsedem.geo.utils.mask_raster().

pywatemsedem.geo.utils.nearly_identical(geoms, p, threshold=0.75)

Identify nearly identical geometries

Parameters:

• geoms (geopandas.GeoSeries)

• p (shapely.Geometry)

Returns:

True/False

Return type:

pandas.Series

pywatemsedem.geo.utils.polygons_to_raster(vct_polygon, rst_out, rst_template, field, dtype)

Converts a polygon shapefile to a raster

Parameters:

• vct_polygon (str or pathlib.Path) – File path of input polygon shapefile

• rst_out (str or pathlib.Path) – File path of output rasterfile

• rst_template (str or pathlib.Path) – File path to a template raster

• field (str) – The field of the shapefile containing the values for the raster

• dtype (str, default None) – Data type of the values, e.g. Byte/Int16/UInt16/UInt32/Int32/Float32…

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.process_mask_shape_from_raster_file(gdf_catchment, catchment_value=1)

Process the geopandas shape format of the input raster

Parameters:

• gdf_catchment (geopandas.GeoDataFrame) – dataframe holding the mask , possible in multiple polygons (rows)

• catchment_value (int, default 1) – value that is used to define catchment

Returns:

gdf_catchment – dissolved dataframe holding mask, in one polygon (row)

Return type:

geopandas.GeoDataFrame

pywatemsedem.geo.utils.raster_array_to_pandas_dataframe(arr_raster, profile)

Convert a raster array to a pandas dataframe.

Parameters:

• arr_raster (numpy.ndarray) – Array raster format

• profile (rasterio.profiles) – See rasterio.profiles.Profile

Returns:

df – A pandas format of the array raster with

• row (int): the row id

• col (int): the column id

• val (float): the value

Return type:

pandas.DataFrame

pywatemsedem.geo.utils.raster_dataframe_to_arr(df, profile, col, dtype)

Convert a pandas dataframe column to an array

Parameters:

• df (pandas.DataFrame) –

  A pandas format of the array raster with

  • row (int): the row id

  • col (int): the column id

  • val (float): the value

• profile (rasterio.profiles) – See rasterio.profiles.Profile

• col (str) – Column name to convert to raster

• dtype (numpy.dtype)

Returns:

arr – Array of dataframe columns ‘val’

Return type:

numpy.ndarray

pywatemsedem.geo.utils.raster_to_polygon(rst_in, vct_out)

Polygonize a raster

This function converts rastercells to polygons

Parameters:

• rst_in (str or pathlib.Path) – File path of theinput raster

• vct_out (str or pathlib.Path) – File path of the destination shapefile

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.rasterprofile_to_rstparams(profile)

Transform rasterprofile to rstparams

Parameters:

profile (rasterio.profiles) – See rasterio.profiles.Profile

Returns:

rstparams – gdal dictionary holding all metadata for idrisi rasters

Return type:

dict

pywatemsedem.geo.utils.read_dtype_raster(rst_in)

Read dtype of raster

Parameters:

rst_in (pathlib.Path) – File path to the input raster

Returns:

dtype

Return type:

numpy.dtype

Note

Only works for single band rasters

pywatemsedem.geo.utils.read_rasterio_profile(rst_in)

Read all spatial dimensions of a raster

Parameters:

rst_in (pathlib.Path) – File path to the input raster

Returns:

profile – See pywatemsedem.geo.rasterproperties.RasterProperties.rasterio_profile()

Return type:

dict

pywatemsedem.geo.utils.read_rst_params(rst_in)

Read all spatial dimensions of a raster

Parameters:

rst_in (pathlib.Path) – File path to the input raster

Returns:

• minmax (list) – A list with the extreme coordinate values (xmin, ymin, xmax and ymax)

• transform (rasterio.transform) – Transformation as defined in Rasterio.

• cols (int) – The number of columns in the raster

• rows (int) – The number of rows in the raster

pywatemsedem.geo.utils.reclass_rst(rst_in, rst_out, df_reclass)

Reclass of a raster based on pandas dataframe

Parameters:

• rst_in (str or pathlib.Path) – File path of the input raster

• rst_out (pathlib.Path) – File path of the destination rst (.sdat)

• df_reclass (pandas.DataFrame) –

  DataFrame containing the mapping with the columns:

  • RST_VAL: current values

  • NEWVAL: new value

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.rst_to_vct_points(rst_in, vct_out)

Convert all no nodata values in a raster to a vector point file.

Parameters:

• rst_in (str or pathlib.Path) – File path of the raster file that should be converted to a shapefile.

• vct_out (pathlib.Path) – File path of the destination vct.

pywatemsedem.geo.utils.rstparams_to_rasterprofile(rstparams, epsg=None)

Transform rstparams dictionary to to rasterio rasterprofile dictionary

Parameters:

• profile (rasterio.profiles) – See rasterio.profiles.Profile

• epsg (str, default None) – The epsg code defining the coordinate system of the raster, format = “EPSG:XXXXX”

Returns:

rstparams – gdal dictionary holding all metadata for idrisi rasters

Return type:

dict

pywatemsedem.geo.utils.saga_intersection(vct_a, vct_b, vct_intersect)

Calculate the intersection between two shapefiles

Parameters:

• vct_a (str or pathlib.Path) – File path of input shapefile 1

• vct_b (str or pathlib.Path) – File path of input shapefile 2

• vct_intersect (str or pathlib.Path) – File path of the calculated intersection shapefile

Note

Uses and relies on saga_cmd CLI

pywatemsedem.geo.utils.set_dtype_arr_rst(arr, profile, dtype=None)

Set type for an array

Parameters:

• arr (numpy.ndarray) – arr of a raster for which dtype has to be changed

• profile (rasterio.profiles) – See rasterio.profiles.Profile

• dtype (numpy.dtype, default None) – e.g. np.float64, np.float32, ..

Returns:

• arr (numpy.ndarray) – dtype-updated arr of a raster for which dtype has to be changed

• profile (rasterio.profiles) – See rasterio.profiles.Profile with update dtype geo medata information

pywatemsedem.geo.utils.set_no_data_arr(arr, arr_mask, nodata)

Set no data to values to raster array outside mask.

Parameters:

• arr (numpy.ndarray) – To mask array

• arr_mask (numpy.ndarray) – Mask array

• nodata (float) – The no data value to set in input array

pywatemsedem.geo.utils.set_no_data_rst(rst_in, rst_out, arr_bindomain, profile, dtype=None, nodata_val=-9999)

Set all pixels of a raster outside the model domain equal to NoData

Parameters:

• rst_in (str) – File path of input raster to set no data values

• rst_out (str) – File path of output raster with no data values

• arr_bindomain (numpy.ndarray) – Array used to define domain nodata. In domain is equal to one, outside domain is equal to zero.

• profile (rasterio.profiles) – See rasterio.profiles.Profile

• dtype (numpy.dtype, default None) – e.g. np.float64, np.float32, ..

• nodata_val (int, default -9999) – Standard value for nodata

pywatemsedem.geo.utils.tiff_to_esri_shp(tiff_in, vct_out, epsg)

Transform a tiff file to an esri shape (raster to shape)

Parameters:

• tiff_in (str or pathlib.Path) – name input tiff

• vct_out (str or pathlib.Path) – name input shapefile

• epsg (str, default None) – the epsg code defining the coordinate system of the raster, format = “EPSG:XXXXX”

pywatemsedem.geo.utils.tiff_to_geopandas_df(tiff_in)

Transform a tiff file to a geopandas dataframe

Parameters:

tiff_in (str or pathlib.Path) – File path of tiff raster file

Returns:

gdf – Geopandas representation of tiff raster

Return type:

geopandas.GeoDataFrame

pywatemsedem.geo.utils.tiff_to_idrisi(tiff_in, rst_out, dtype)

Convert a GeoTiff to an Idrisi RST

Parameters:

• tiff_in (str) – File path of the input tiff file

• rst_out (str) – File path of the destination rst

• dtype (str, default Float64) – Raster type

Note

Uses and relies on gdal_translate CLI

pywatemsedem.geo.utils.valid_gdal_type(func)

Check if your input array mask is valid. Use as decorator

pywatemsedem.geo.utils.valid_mask(func)

Check if your input array mask is valid. Use as decorator

pywatemsedem.geo.utils.vct_to_rst_field(vct_in, rst_out, Cnst, field=None, alltouched=True, dtype='Float64')

Rasterizes a shapefile by a given attribute field

Parameters:

• vct_in (str or pathlib.Path) – File path of the shapefile to be rasterized.

• rst_out (pathlib.Path) – File path of the destination rst

• Cnst (dict) –

  Dictionary with following keys:

  • res (int): resolution

  • nodata (int): nodata flag

  • minmax (list): list with xmin, ymin, xmax, ymax

• field (str) – The field of the shapefile containing the values for the raster

• alltouched (bool, default True) – Enables the ALL_TOUCHED rasterization option so that all pixels touched by lines or polygons will be updated.

• dtype (str, default None) – Data type of the values, e.g. Byte/Int16/UInt16/UInt32/Int32/Float32…

Note

Uses and relies on gdal_rasterize CLI

pywatemsedem.geo.utils.vct_to_rst_value(vct_in, rst_out, rstval, Cnst, nodata=-9999, alltouched=True, dtype=None)

Rasterizes a shapefile by a given constant value

Parameters:

• vct_in (str or pathlib.Path) – File path of the shapefile to be rasterized.

• rst_out (pathlib.Path) – File path of the destination rst

• Cnst (dict) –

  Dictionary with following keys:

  • res (int): resolution

  • nodata (int): nodata flag

  • minmax (list): list with xmin, ymin, xmax, ymax

• rstval (str) – The value all features of the shapefile will get in the raster.

• alltouched (bool, default true) – Enables the ALL_TOUCHED rasterization option so that all pixels touched by lines or polygons will be updated.

• dtype (str, default None) – Data type of the values, e.g. Byte/Int16/UInt16/UInt32/Int32/Float32…

Note

Uses and relies on gdal_rasterize CLI

pywatemsedem.geo.utils.write_area_ha_to_vct(vct)

Add a field ‘AREA_HA’ to a shapefile

This function calculates the area of every feature in a shapefile in hectare

Parameters:

vct (str or pathlib.Path) – File path to the input shapefile

pywatemsedem.geo.utils.write_arr_as_rst(arr, rst_out, dtype, profile)

Write numpy.ndarray as a raster-file

Parameters:

• arr (numpy.ndarray) – 2D numpy array to be written as a raster file

• rst_out (str) – File path to the output raster

• dtype (numpy.dtype)

• profile (rasterio.profiles) – See rasterio.profiles.Profile

pywatemsedem.geo.valid module

exception pywatemsedem.geo.valid.PywatemsedemInputError

Bases: Exception

Raise when input data are not conform the pywatemsedem required input format.

exception pywatemsedem.geo.valid.PywatemsedemTypeError

Bases: Exception

Raise when input data type are not conform the pywatemsedem required type format.

pywatemsedem.geo.valid.valid_exists(rst, fun)

Check if input file exists

Parameters:

• rst (pathlib.Path) – File path to raster.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_input(func=None, dict=None)

Customizable wrapper function that allows to check arg-defined function input.

This wrapper is defined to check formats of file-based raster and vector input. It makes use of a dictionary to apply specific valid functions on non-keyword arguments. This valid function is typically applied to functions which have file paths that have to be checked in their input.

Parameters:

• func (callable) – To call function for which to check inputs. Note that only function with all keyword-arguments as parameters can be used.

• dict (dictionary) – Holding string of fun parameters as keys, and valid-callable function as values.

Return type:

function output

Examples

Use with a ‘@’-decorator to check input types for utils functions:

from pywatemsedem.geo.utils import valid_input,valid_rasterlist,valid_polygonvector
#note: only on-keyword arguments!
@valid_input(dict={"lst_rst": valid_rasterlist, "vct_in": valid_polygonvector})
def grid_statistics(lst_rst,vct_in):
# ... function code

Above example check if input of lst_rst is a list of rasters (and rasters exist) (1) and if input of vct_in is a polygoon shape (2).

Note

1. Can only be applied to non-keyword arguments.

2. Validation methods should be added to VALID_FUN.

pywatemsedem.geo.valid.valid_linesvector(vct, fun)

Check if input vector file is a valid lines shape

Parameters:

• vct (pathlib.Path) – File path to vector.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_mask(mask)

Check if mask is not equal to none and valid

Parameters:

mask

pywatemsedem.geo.valid.valid_pointvector(vct, fun)

Check if input vector file is a valid point shape

Parameters:

• vct (pathlib.Path) – File path to vector.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_polygonvector(vct, fun)

Check if input vector file is a valid polygon shape

Parameters:

• vct (pathlib.Path) – File path to vector.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_raster(rst, fun)

Check if input file is a valid raster

Parameters:

• rst (pathlib.Path) – File path to raster.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_rasterlist(lst_rst, fun)

Check if input is a valid list of rasters

Parameters:

• lst_rst (list) – List of file paths to rasters.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

pywatemsedem.geo.valid.valid_rp(rp)

Check if rasterproperties is not equal to none and valid

Parameters:

rp (pywatemsedem.geo.rasterproperties.RasterProperties)

pywatemsedem.geo.valid.valid_vector(vct, fun, req_type=None)

Check if input file is a valid vector

Parameters:

• vct (pathlib.Path) – File path to vector.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

• req_type – Required geometry type of vector, limited to “Polygon”, “LineString”, “Point” and None (i.e. don’t check).

pywatemsedem.geo.valid.valid_vectorlist(lst_vct, fun, req_type=None)

Check if input is a valid list of rasters

Parameters:

• lst_vct (list) – List of file paths to rasters.

• fun (callable) – See pywatemsedem.geo.valid.valid_input().

• req_type – Required geometry type of vector. See pywatemsedem.geo.valid.valid_vector()

Note

req_type can only be one type, not a mix.

pywatemsedem.geo.vectors module

class pywatemsedem.geo.vectors.AbstractVector

Bases: object

Abstract Vector class based on geopandas GeoDataFrame

check_if_empty()

Check if input is empty

check_type(geometry_type, req_geometry, implemented_types=['LineString', 'Polygon', 'Point'])

Check geometry types of vector to the required type

Parameters:

• geometry_type (str) – Geometry type of input dataset.

• req_geometry (str) – The required geometry types

• implemented_types (list, default "LineString", "Polygon", "Point") – List of implemented geometry types.

clip()

NotImplemented clip function

property geodata

Property to override

initialize(geodata, geometry_type, req_geometry_type=None, allow_empty=False)

Abstract vector class

Parameters:

• geodata (geopandas.GeoDataFrame) – Input data set.

• geometry_type (str) – Geometry type of input dataset

• req_geometry_type (str, default None) – Geometry type, for implemented types, see pywatemsedem.geo.vectors.AbstractVector.check_type()

• allow_empty (bool, default False) – Allow an empty geodataframe

is_empty()

check if geodata (vector) is None (empty)

Return type:

True/False

plot(color=None, column=None)

Plot shape vector with geopandas plot

Parameters:

color

Returns:

ax

Return type:

matplotlib.pyplot.axis

rasterize(rst_reference, epsg, col='NR', nodata=None, dtype_raster='float', convert_lines_to_direction=False, gdal=False)

Rasterize function for shape file

Parameters:

• rst_reference (str or pathlib.Path) – File path to reference file for raster output.

• epsg (int) – EPSG code should be a numeric value, see https://epsg.io/.

• col (str, default "NR") – Column name to map

• nodata (float, default None) – Values within dataframe ‘col’ that have to be considered as nodata in raster.

• dtype_raster (str, default "float") – Output raster type convert_lines_to_direction:

• convert_lines_to_direction (bool, default "False") – Convert lines to directions

• gdal (bool, default False) – Use gdal(true) / saga (false)-enige for mapping.

Returns:

arr – Return numpy array

Return type:

numpy.ndarray

write(outfile_path)

Write raster data to disk.

Parameters:

outfile_path (pathlib.Path or str, default None) – File path output

class pywatemsedem.geo.vectors.VectorFile(file_path, req_geometry_type=None, vct_clip=None, allow_empty=False)

Bases: AbstractVector

clipped Vector based on input vector file

clip(file_path, vct_clip)

Clip input file path with vct_clip

Parameters:

• file_path (pathlib.Path) – File path to user input raster.

• vct_clip (pathlib.Path) – Mask vector

• geometry_type (str) – Required type of geometry, see implemented geometries in pywatemsedem.geo.vectors.AbstractVector.check_type()

Return type:

geopandas.GeoDataFrame

class pywatemsedem.geo.vectors.VectorMemory(geodata, geometry_type, req_geometry_type=None, allow_empty=False)

Bases: AbstractVector

Geopandas vector

Parameters:

• geodata (geopandas.GeoDataFrame) – See pywatemsedem.geo.vectors.AbstractVector

• geometry_type (str) – See pywatemsedem.geo.vectors.AbstractVector

• req_geometry_type (str, default None) – See pywatemsedem.geo.vectors.AbstractVector

• allow_empty (bool, default False) – See pywatemsedem.geo.vectors.AbstractVector

clip()

NotImplemented clip function

Module contents