EWR tool overview

library(HydroBOT)
library(dplyr)
library(ggplot2)
library(patchwork)

The only module currently integrated into HydroBOT is the Environmental Water Requirements (EWR) tool maintained by the Murray-Darling Basin Authority. This tool is written in Python, and takes hydrographs from 235 gauges in the Murray-Darling Basin as inputs (as of version 2.3.7 used to build this website). The EWR tool then assesses whether a suite of environmental water requirements defined by the Basin states in Long Term Watering Plans (LTWPs) are met.

EWR definitions

The EWRs are defined by discharge, duration, timing, and the requirements also include frequency requirements (how many years out of 10), and inter-event durations (how long is acceptable between events). These EWRs define ecologically-relevant parts of the flow regime, and so differ between gauges due to local differences in hydrologic conditions and topography. At any given gauge or planning unit, there are typically many different EWRs. These account not only for the different flow regime components shown in Figure 1, but also several different EWRs within individual parts of the flow regime, reflecting the different ecological requirements for e.g. small freshes by different groups of organisms or life stages. There are 3369 requirements defined across the basin, each of which is defined in the Long-Term Watering Plans (LTWPs) produced by Basin states, (e.g. NSW Department of Planning, Industry and Environment 2020; DEW 2020).

Figure 1: EWRs capture a range of flow categories with different hypothesised relationships to environmental outcomes. Figure from NSW Department of Planning and Environment, NSW Long Term Water Plans: Background Information A description of the development of the 9 LTWPs in NSW Part C: Environmental water requirements.

Ecological relationships

Although the EWRs define hydrologic thresholds, and the EWR tool assesses only those hydrologic thresholds, they were developed based on hypothesised relationships to ecological values and management targets, with those links established in the LTWPs. The causal networks provided by HydroBOT for the EWR tool (causal_ewr) have extracted these links from the LTWPs, allowing HydroBOT to connect the outcomes of the EWR module to larger ‘theme’ scale outcomes.

Briefly, the ‘theme’ scales provided in the causal networks begin with ewr_code_timing and ewr_code , which are the hydrologic thresholds themselves returned by the EWR tool. The ewr_code_timing are the raw outputs, but represent different ‘versions’ of a single EWR, and so can sometimes be collapsed to ewr_code. This practice is done in the examples here but is dubious. In some cases, the _timing versions are simply different levels of leniency, but in others they seem to represent fundamentally different checks for different anticipated outcomes.

Onward, larger, theme scales include:

env_obj, which are proximate environmental objectives, e.g. NF1: No loss of native fish species. The full set of code meanings are in the LTWPs.
Target The large ecological groupings (Native Fish, Waterbirds, Ecosystem Function, and Native Vegetation).
Specific_goal often certain species or particular ecosystem functions. Care should be taken; these are highly location-specific, but that is not well QCed.
Objective typically objectives around maintaining or increasing populations or habitat
target_5_year_2024, target_10_year_2029, target_20_year_2039; similar to Objective, but future-looking

These levels are not strictly nested, and are not 1:1 relationships. It is common for several ‘ewr_code’s to link to each ’env_obj’, and each ‘ewr_code’ usually links to more than one ‘env_obj’. Thus, while we refer to ‘aggregation’, it is possible for the number of units to grow as we move up the causal network. An example for a single gauge is shown here, from ewr_code to env_obj to Specific_goal to Target , as well as from env_obj to target_5_year_2024. In practice, the set of levels and their order should be determined by the question being asked. For more details about causal networks and illustrations with the EWR network, see here.

edges <- make_edges(
  dflist = causal_ewr,
  fromtos = list(
    c("ewr_code", "env_obj"),
    c("env_obj", "Specific_goal"),
    c("Specific_goal", "Target"),
    c("env_obj", "target_5_year_2024")
  ),
  gaugefilter = "409025"
)
nodes <- make_nodes(edges)
default_network <- make_causal_plot(nodes, edges, render = FALSE)
DiagrammeR::render_graph(default_network)

Spatial context

Some gauges provide information about conditions in multiple planning units (fine-scaled spatially-defined water plannning areas, predominantly in New South Wales) and sustainable diversion limit (sdl) units, which are approximately subcatchments within the Basin and define a larger planning area. This information sharing from gauges to other units was the driving force to provide non-spatial joins of spatial data in HydroBOT.

eg <- get_ewr_gauges()

etp <- eg |>
  sf::st_drop_geometry() |>
  select(gauge, planning_unit_name) |>
  distinct() |>
  summarise(n_planning = n(), .by = gauge)

ets <- eg |>
  sf::st_drop_geometry() |>
  select(gauge, SWSDLName) |>
  distinct() |>
  summarise(n_sdl = n(), .by = gauge)

eg <- eg |>
  left_join(ets, by = "gauge") |>
  left_join(etp, by = "gauge")

cpal <- make_pal(as.factor(1:10), palette = "grDevices::Viridis")

sdlmap <- ggplot() +
  geom_sf(data = sdl_units, fill = "azure") +
  geom_sf(data = eg, mapping = aes(color = as.factor(n_sdl))) +
  scale_color_manual(values = cpal) +
  theme_hydrobot() +
  labs(color = "Number sdl units")

pumap <- ggplot() +
  geom_sf(data = planning_units, fill = "azure") +
  geom_sf(data = eg, mapping = aes(color = as.factor(n_planning))) +
  scale_color_manual(values = cpal) +
  theme_hydrobot() +
  labs(color = "Number planning units")

sdlmap / pumap +
  plot_annotation(tag_levels = "a")

Figure 2: Gauges in the EWR tool (points) superimposed on SDL units (a) and planning units (b). Points are colored by the number of larger units they are associated with; typically one but sometimes up to 2 sdl units or 10 planning units.

EWR tool and HydroBOT

HydroBOT analyses proceed by feeding the EWR tool a set of hydrographs, which then get run through the EWR tool. The outputs are then processed by the Aggregator and Comparer. These approaches form the core of the demonstrations on this website, as most examples are easiest with the real examples we can run through the included EWR tool module.

While the EWR tool can pull hydrographs on the fly, it uses a different set of functions to do so than those it uses to process scenarios and modeled data. Thus, HydroBOT tends to run saved hydrographs representing scenarios. If purely historical data will be run, it is easiest to pre-download it with e.g. hydrogauge (R) or the mdba gauge-getter (Python).

Tip

For an example workflow to assess ratios of EWR achievement across scenarios (the ‘frequency ratio’ but also a ratio of achievement based on frequency and interevents), see here.

For a similar example looking at characteristics of the interevent periods, see here.

EWR helpers in HydroBOT

HydroBOT provides some useful functions to access information about the EWR tool and clean it up:

get_ewr_version() obtains the currently installed version of py-ewr
get_causal_ewr() obtains the causal network from py-ewr
- Also see the HydroBOT data causal_ewr
get_ewr_gauges() obtains the gauges used by py-ewr
get_ewr_table() obtains the ‘parameter sheet’ in py-ewr
assess_ewr_achievement() assesses the yearly ‘events’ returned by py-ewr according to frequency and interevent requirements, allowing analysis of the raw events as well as those additional requirements
cleanewrs() a simple text cleaner of the EWR outputs
- There are a number of more specific cleaners not discussed here, see the function reference.

EWR assessment

The EWR tool py-ewr has potentially several output files. The standard one to use with HydroBOT is the yearly output, which returns one row per year per EWR. The key outcome column is event_years, which indicates whether the discharge, timing, and duration of flows met that EWR in that year. HydroBOT then can perform some further assessment, checking whether those yearly achievements met frequency and interevent requirements. This assessment (done by assess_ewr_achievement(), which gets called when the type = 'achievement' argument is used in read_and_agg() or other EWR-read-in functions (e.g. prep_ewr_output()), returns three new ‘achievement’ columns in addition to the event_years column which is also passed through:

frequency_achieved- did the event_years meet target frequencies?
interevent_achieved- was the interevent period below or equal to the maximum interevent period?
ewr_achieved- were both the frequency and interevent conditions met?

HydroBOT checks the frequency conditions with a 10-year lookback window for each year, and so if, for example, an EWR must have a frequency of 0.8, it asks whether it has been met in 8 of the most recent 10 years, rather than 80% of the entire timeseries. Thus, we can obtain an ‘achievement’ value for each year and avoid potentially major issues caused by patterns within the timeseries that would otherwise be masked. This reflects the reality of the system as well- whether the frequency of an event has been sufficient is a property of the years up to each year, not the whole series. I.e. these are not aggregations, but assessments of condition.

If temporal aggregtion is desired over the timeseries, these yearly assessments should then be aggregated with the aggregator. This aggregation can proceed with the user’s choice of outcome columns; event_years, frequency_achieved, interevent_achieved, or ewr_achieved.

These assessments provide the bulk of examples on this website and most of the uses of HydroBOT.

Interevent assessment

In some cases, we might want to assess various characteristics of the interevent periods themselves. In this case, we use the all_successful_interEvents dataset returned by the EWR tool.

Note

The all_successful_interEvents dataset returns the interevent period between successful ‘events’, (as opposed to the all_interEvents, which returns the interevent period between ‘events’ whether or not those are successful). Thus, it is not the interevents that are successful, but the events themselves.

The logic below would work for all_interEvents as well, and would only require changing type = 'all_interEvents' and paramargs = list(type = 'interevents') in read_and_agg(). See the interevent notebook.

Setting the type argument in prep_ewr_output() to ‘interevents’, (via the type of read_and_agg() or better yet as the prepargs = list(type = 'interevents') argument) calculates some necessary statistics comparing each interevent to the maximum allowable. This happens with assess_ewr_interevent(). In addition to the inter_event_length variable returned as output by the EWR tool, this function returns an additional seven data columns:

max_interevent- the maximum interevent limit defined in the LTWP
exceedance_days- the number of days in each interevent that exceed the maximum allowed
- This is just length - max, and so may be less than 0 if the interevent was below the maximum.
interevent_ratio- the days in the interevent as a fraction of the maximum
- length/max, and so will be < 1 if the event is shorter and > 1 if the event is longer than the max
exceedance_ratio- ratio of days in the event beyond the max to the max.
- interevent_ratio -1, which is the same as (days above max)/max
exceedance- binary, whether the interevent was >= the max
exceedance_only- the days in the interevent beyond the max, 0 otherwise
days_in_exceeding- the total number of days in interevent periods that were longer than the max

HydroBOT calculates all of these values, which can then be used in the aggregator, as shown specifically for some MDBA metrics.

Example analyses

Nearly all examples and analyses on this website use the EWR achievement analysis (i.e. dependent on frequency and interevents). Two specific analyses using MDBA metrics are demonstrated as well:

Frequency ratio (also covers issues of raw assessments and comparisons)
Inter-event statistics

References

DEW. 2020. “Long-Term Environmental Watering Plan for the South Australian River Murray Water Resource Plan Area.”

NSW Department of Planning, Industry and Environment. 2020. “Macquarie-Castlereagh Long Term Water Plan Part A: Macquarie-Castlereagh Catchment.”