Wildcards#

It is easy to run PyPSA-USA for multiple scenarios using the wildcards feature of snakemake. Wildcards generalise a rule to produce all files that follow a regular expression pattern which e.g. defines one particular scenario. One can think of a wildcard as a parameter that shows up in the input/output file names of the Snakefile and thereby determines which rules to run, what data to retrieve and what files to produce.

Note

Detailed explanations of how wildcards work in snakemake can be found in the relevant section of the snakemake documentation.

The {interconnect} wildcard#

The {interconnect} wildcard sets the geographic scope of the model run. Models can be run for the western, eastern, texas, or usa grid. The interconnects follow the representation described by Breakthrough Energy.

A visual representation of each {interconnect} is shown below:

_images/cutouts.png

The {simpl} wildcard#

The {simpl} wildcard specifies the number of buses the substation-level network is pre-clustered to in the rule cluster_resources (which runs directly after aggregate_to_substations, before any per-bus data is built).

Under the simplify-early architecture, {simpl} is the resolution at which the model’s data layers are built: demand disaggregation, renewable resource profiles and their land-use potentials, and generator placement all operate on the {simpl}-bus network (elec_s{simpl}.nc) produced by cluster_resources. The rule cluster_network then reduces the network further to the final {clusters} transmission resolution. {simpl} therefore controls the resource resolution, while {clusters} controls the transmission resolution — see the spatial configuration page for how the two interact.

The {clusters} wildcard#

The {clusters} wildcard specifies the number of buses the {simpl}-level network is reduced to in the rule cluster_network. The number of clusters must be lower than the number of {simpl} buses and at least the number of balancing authorities (or, for the ReEDS networks, the zone counts listed on the spatial configuration page — cluster_network reports the correct minimum if the value is infeasible).

A plain integer (e.g. 33) aggregates buses and conventional generators only: renewable generators are moved to the clustered buses but keep their distinct {simpl}-level resource zones, so a clustered bus may host several wind or solar generators with different profiles and potentials. This preserves resource quality on a coarser transmission grid instead of averaging the best and worst sites of a zone into one profile. A letter suffix changes which carriers are aggregated:

Value

Behaviour

33

Default. Aggregate only conventional carriers; renewables keep {simpl}-level resource zones.

33m

Same as 33 (kept as an alias for older configs).

33s

“Small”: aggregate all carriers to the clustered buses (one generator per carrier and bus). Cheapest to solve; averages renewable resource classes within a zone.

33a

Aggregate no carriers — all generators keep their {simpl}-level resolution.

33c

Aggregate all except conventional carriers; conventional plants keep {simpl}-level detail.

all

Skip spatial reduction entirely (one cluster per bus).

Carriers listed in clustering: cluster_network: exclude_carriers are never aggregated. For non-aggregated carriers, land-use limits remain enforced at the {simpl}-level land_region, so resource potentials are not artificially merged.

The {ll} wildcard#

The {ll} wildcard specifies what limits on transmission expansion are set for the optimisation model. It is handled in the rule prepare_network.

The wildcard consists of a type letter followed by a factor, e.g. v1.25, copt:

  • v (volume): limits the total volume of line expansion — capacity increases weighted by line length (MW·km) — across AC lines and AC/DC links.

  • c (cost): limits the total cost of line expansion — capacity increases weighted by capital cost — across AC lines and AC/DC links.

After the type letter you can specify:

  • a float, e.g. v1.25: branches become extendable and total expansion is limited to 25 % above today’s length- (or cost-) weighted capacity. A factor of exactly 1.0 (e.g. v1.0) keeps branches non-extendable, i.e. fixed at today’s capacities.

  • opt, e.g. vopt or copt: branches become extendable and expansion is optimised purely on capital cost, with no global cap (only per-branch s_nom_max/p_nom_max bounds apply).

We recommend using the line-volume limit (v...) for constraining transmission expansion.

Note

In result filenames the wildcard is preceded by the letter l (e.g. elec_s75_c33_ec_lv1.0_...): the l is part of the filename pattern, while the wildcard value itself starts with v or c. A bare all is accepted by the wildcard pattern but is not currently handled by prepare_network.

The {opts} wildcard#

The {opts} wildcard triggers optional constraints and temporal settings, which are activated in either the prepare_network or the solve_network step. It may hold multiple triggers separated by -, i.e. REM-3h contains the REM regional emissions limit trigger and the 3h temporal averaging switch. In sector-coupled runs the same tokens apply, with RPS and REM dispatching their sector-specific variants.

The mathematical formulation of every constraint token, its configuration keys, and its source code location are documented on the custom constraints page.

The regions for REM, ERM, and RPS can be defined using either the ReEDS zone name (p##), the state code (e.g. TX, CA, MT), the PyPSA-USA interconnect name (western, eastern, texas, usa), or the NERC region name.

Warning

TCT targets can only be used with renewable generators and utility scale batteries in sector studies.

There are currently:

Trigger

Description

Definition

Status

nH; e.g. 3h-6h

Resample the time resolution by averaging over every n snapshots. Writes clustering: temporal: resolution_elec, which can equivalently be set directly in the config.

prepare_network: average_every_nhours()

In active use

nSEG; e.g. 4380SEG

Apply time-series segmentation with the tsam package, aggregating each investment period to n adjacent segments of varying length based on renewable capacity factors, hydro inflow, and load. Writes clustering: temporal: resolution_elec, which can equivalently be set directly in the config.

prepare_network: apply_time_segmentation()

In active use

Co2Lx.xx; e.g. Co2L0.05

Add a system-wide absolute limit on carbon-dioxide emissions (sets electricity: co2limit_enable: true). The factor scales the reference budget: Co2L0.05 caps emissions at 5% of electricity: co2base (which must be defined). Always append a factor — a bare Co2L mis-parses the 2 in the token name as the value. Alternatively set electricity: co2limit_enable and co2limit directly in the config. See Custom Constraints.

prepare_network: add_co2limit(); parsed in update_config_from_wildcards()

In active use

CH4Lxxx; e.g. CH4L200

Add a system-wide absolute limit on gas-fired primary energy (sets electricity: gaslimit_enable: true). The appended number gives the limit in TWh thermal: CH4L200 limits gas use to 200 TWh thermal per year. Always append a value — a bare CH4L mis-parses the 4 in the token name as the value. Alternatively set electricity: gaslimit_enable and gaslimit (MWh thermal) directly in the config.

prepare_network: add_gaslimit()

In active use

Ep; e.g. Ep or Ep50

Add a carbon-dioxide price to the marginal cost of emitting generators and storage units, weighted by emission intensity over efficiency (sets costs: emission_prices: enable: true). An appended number sets the price in $/tCO2; otherwise costs: emission_prices: co2 is used.

prepare_network: add_emission_prices()

In active use

Ept

Request time-varying monthly carbon-dioxide prices (sets costs: emission_prices: co2_monthly_prices: true). No workflow step currently consumes this setting; because the token also matches the Ep parser, its net effect equals plain Ep (static price from costs: emission_prices: co2).

Parsed in update_config_from_wildcards()

Parsed; monthly pricing not implemented (falls back to static Ep)

RPS

Add Renewable Portfolio Standard (RPS) and Clean Energy Standard (CES) constraints enforcing minimum shares of generation (MWh) from eligible carriers per REC trading zone and planning horizon. Merges state targets from electricity: portfolio_standards (CSV) with ReEDS RPS/CES trajectories. Regions may be ReEDS zones, state codes, interconnects, or NERC regions. See Custom Constraints.

solve_network: add_RPS_constraints()

In active use

REM

Add regional carbon-dioxide emission limits per planning horizon, configured via the CSV at electricity: regional_Co2_limits (default repo_data/config/policy_constraints/regional_Co2_limits.csv). In sector studies this token instead dispatches sector-specific emission limits. See Custom Constraints.

solve_network: add_regional_co2limit()

In active use

ERM

Add Energy Reserve Margin (ERM) constraints requiring energy-backed firm capacity above demand in every snapshot for defined regions. Generators are credited at their availability; storage and transmission contribute through shadow reserve dispatch. Configured via electricity: erm; defaults to a 15% margin for all regions. See Custom Constraints.

solve_network: add_ERM_constraints()

In active use

TCT

Add minimum or maximum nominal-capacity targets per carrier group, region, and planning horizon, configured via the CSV at electricity: technology_capacity_targets. In myopic runs, max = 0 targets additionally force retirement of existing capacity. See Custom Constraints.

solve_network: add_technology_capacity_target_constraints()

In active use

<carrier>+{p|e|c|m}<factor>; e.g. solar+c0.5

Scale an attribute of a single carrier: p = p_nom_max, e = e_nom_max, c = capital_cost, m = marginal_cost. Parsed into adjustments: electricity, which no workflow step currently applies to the network.

Parsed in update_config_from_wildcards()

Parsed; currently without effect (adjustments has no consumer)

Energy Reserve Margin (ERM) Configuration#

The ERM constraint ensures that each region has sufficient firm capacity to meet demand plus a reserve margin at every timestep. Unlike traditional planning reserve margins that only consider peak demand, ERM enforces the constraint across all snapshots.

Key Features:

  • Resources must be “energy-backed” - storage devices must have sufficient state of charge to contribute to the reserve

  • Supports multiple non-overlapping regions with different reserve margins

  • Defaults to 15% reserve margin for all regions if not specified

Configuration:

To enable ERM, add ERM to the opts wildcard in your scenario configuration:

scenario:
  opts: [ERM-3h]  # or [REM-ERM-3h] to combine with other opts

To customize the ERM values per region, add an erm section under electricity in your config file:

electricity:
  erm:
    all: 0.15        # 15% reserve margin for all regions (default)
    # Or specify per region:
    # western: 0.15
    # SPP: 0.12
    # CISO: 0.17

If no erm configuration is provided, a default of {'all': 0.15} (15% reserve margin for all regions) is used.

Valid region identifiers:

  • all - applies to all buses in the network

  • State codes: TX, CA, MT, etc.

  • Interconnect names: western, eastern, texas

  • NERC region names

  • ReEDS zone names: p1, p2, etc.

The {sector} wildcard#

The {sector} wildcard specifies which sectors to include. In the configuration it is set via scenario: sector:; an empty string (sector: "", the default) runs an electricity-only study. The Snakefile normalises the value automatically: electricity is always included, so "" becomes E and G becomes E-G. The wildcard itself must match ([EG]-)*[EG], i.e. dash-separated sector codes.

Sector

Code

Description

Status

Electricity

E

Electrical sector. Will always be run.

Runs

Natural Gas

G

All sectors added

Development

Atlite cutout files#

There is no {cutout} wildcard among the workflow’s global wildcard constraints. Weather cutouts enter the workflow as files named cutouts/{interconnect}_{cutout}.nc, where the cutout identifier takes the form {dataset}_{year} (e.g. era5_2019) as configured under atlite: cutouts:. They are built by the rule build_cutout only when enable: build_cutout: true is set; otherwise a prepared cutout is expected on disk.

Valid dataset names include: era5. Valid years range from 1940 to 2022.

Note

Under the default renewable dataset (renewable: dataset: godeeep), renewable profiles are taken from the GODEEEP dataset and atlite cutouts are not used. Cutouts are only required when renewable: dataset: atlite is selected. Data for era5_2019 has been prepared and is downloaded automatically during the workflow; other years require the user to prepare the cutout themselves.