# totalwatsed3 Aggregation
> Fuses WEPP hillslope water, sediment, and ash-transport outputs into a single daily table served as `totalwatsed3.parquet`.

## Overview
`totalwatsed3.parquet` is produced by `wepppy.wepp.interchange.totalwatsed3.run_totalwatsed3`. The helper reads the per-hillslope interchange artifacts emitted by WEPP (`H.pass.parquet`, `H.wat.parquet`) and, when present, optional soil/element diagnostics (`H.soil.parquet`, `H.element.parquet`) plus WEPP–Ash Transport summaries under `<run>/ash`. The rows are grouped by simulation date (`year`, `julian`, `sim_day_index`, etc.) so that each record represents summed hillslope contributions for a given day, not routed watershed outlet flow:

- Most hydrologic volumes (m³) are summed from `H.wat`, then converted back to depths (mm) via watershed area.
- `Runoff` depth is computed from `H.pass` runoff volume (`runvol`) over aggregated watershed area.
- Sediment masses are derived from the pass file (`H.pass`) and converted to both kg (`seddep_*`) and a dimensionless volumetric concentration (`sed_vol_conc`).
- Optional ash metrics are joined by date and scaled using either supplied area hints or the Watershed controller (per-ash-type mass totals plus volumetric concentration/black fraction when bulk densities are known).

The resulting table is written to `<run>/wepp/output/interchange/totalwatsed3.parquet` using the schema defined in `totalwatsed3.py`.

## Source Artifacts

| Artifact | Path | Purpose |
| --- | --- | --- |
| Hillslope PASS | `H.pass.parquet` | Event-scale runoff, detachment, and sediment concentration (`sedcon_*`) for every WEPP hillslope. |
| Hillslope WAT | `H.wat.parquet` | Daily hydrology terms (`Area`, `Q`, `Ep`, …) per hillslope/OFE combination. |
| Hillslope SOIL (optional) | `H.soil.parquet` | Daily soil profile state; `TSMF` is used for full-profile true moisture fraction when present. |
| Hillslope ELEMENT (optional) | `H.element.parquet` | Daily event partitioning; `QRain` and `QSnow` are area-weighted into watershed-scale depths when present. |
| Ash Transport (optional) | `<run>/ash/H{wepp_id}_ash.parquet` | Daily ash water/wind transport statistics per hillslope, merged when present. |

Only the PASS file includes per-particle sediment concentrations (`sedcon_1`-`sedcon_5`, kg m⁻³). Those values are multiplied by the per-event runoff volume (`runvol`, m³) to recover per-class sediment mass in kilograms. The water-balance terms come from `H.wat` where inputs are expressed as depths (mm); the aggregator multiplies each depth by the contributing area to recover volumes, performs any needed sums, and finally divides by area again to restore depths.

The required `wepppyo3.wepp_interchange.totalwatsed3_to_parquet` producer streams
projected batches of at most 8,192 rows. It retains date aggregates and joins for
one hillslope per worker, with at most twelve workers. Compact physical chunk
metadata replaces full column statistics after indexing; readers reconstruct at
most sixteen row groups at a time. Python resolves paths, options, and ash
controller metadata, and retains the existing `Path` return value. Missing native
support fails explicitly; there is no Python/DuckDB/pandas producer fallback.
The legacy pandas schema metadata is constructed from field types and installed
package versions without importing pandas or creating a DataFrame.

After publication, interchange README generation reads only the schema and three
preview rows from each input. Reading entire tables here can exceed a 12 GiB
worker even when native aggregation itself succeeds.

## Output Snapshots

Key output columns (see `SCHEMA` in `totalwatsed3.py` for the complete list):

- `runvol`, `sbrunv`, `tdet`, `tdep`: watershed totals (m³ or kg).
- `seddep_1`–`seddep_5`: total mass per sediment class (kg) computed as `Σ sedcon_i * runvol`.
- `sed_del`: total daily sediment delivery (kg), computed as `Σ seddep_i`.
- `sed_vol_conc`: watershed-wide volumetric sediment concentration (m³ of solids per m³ of runoff).
- `Area`, `P`, `RM`, `Q`, `Dp`, `latqcc`, … : volumetric sums from `H.wat`, later converted to depths (`Precipitation`, `Lateral Flow`, etc.).
- `SoilWaterTotal`, `ProfileDepth`, `ProfilePorosityCap`, `ProfileFCStore`, `ProfileWPStore`: optional producer-authoritative storage/capacity terms from `H.wat` in mm. They are null for legacy WEPP executables that do not emit the additive columns.
- `TSMF`: area-weighted true soil moisture fraction from `H.soil` when available (null when absent).
- `QRain`, `QSnow`: area-weighted runoff-partition depths from `H.element` when available (null when absent).
- MOFE aggregation rules (multi-OFE hillslopes): per-OFE columns in `H.wat` are not all physically summable across OFEs without producing a non-canonical total. `totalwatsed3` handles this by:
  - **`latqcc`**: uses only the outlet-facing (last) OFE per hillslope/day to avoid counting internal lateral-routing transfers multiple times. The native producer tracks the maximum OFE for one hillslope at a time and contributes only that OFE to the daily lateral-flow sum.
  - **`Runoff` (the user-facing column)**: computed from `H.pass.runvol` volume divided by aggregated watershed area, not from summed `Q` or `QOFE`. `runvol` is threshold-filtered delivery volume, not all physical runoff.
  - **`Q` and `QOFE` aggregate columns** (the literal `SUM(col * 0.001 * Area)` over OFEs): retained for diagnostic continuity but **not** the canonical hillslope totals. QOFE normalization is producer-version-dependent: older local normalization and current Q aliases both exist. The discrepancy depends on geometry, not a universal OFE-count factor. See [Water and PASS variables](../../weppcloud/routes/usersum/weppcloud/water-pass-variables.md) and the [current aggregation contract](../../../docs/schemas/hillslope-water-aggregation-contract.md).

- Ash transport columns when the ash directory is available:
  - Totals and per-area masses: `wind_transport`, `water_transport`, `ash_transport`, `transportable_ash` (+ `_per_ha`).
  - Per-ash-type masses: `{wind,water,ash}_transport_{black,white}` (+ `_per_ha` via per-type area).
  - Ash volumetrics: `ash_vol_conc` (ash solids volume / runoff), `sed+ash_vol_conc` (sediment + ash solids volume / runoff), `ash_black_pct_by_vol` (% of ash solids volume that is black ash).

Nulls from missing PASS or ash rows are filled with zeros before the final Arrow table is materialized.

## Storage Terms

`Total-Soil Water` in `H.wat` is WEPP's unfrozen profile-water term. It is not interchangeable with `TSW` in `H.soil`, which is a top-layer diagnostic. When available, prefer `SoilWaterTotal` for full-profile storage closure because WEPP emits it as `watcon + frozwt`.

The optional `H.wat` capacity terms are direct WEPP outputs in mm: `ProfileDepth = solthk(nsl)`, `ProfilePorosityCap = sum(por * dg)`, `ProfileFCStore = sum(thetfc * dg)`, and `ProfileWPStore = sum(thetdr * dg)`. `totalwatsed3` area-weights these fields when present and preserves nulls when parsing legacy layouts.

## Volumetric Concentration Details

### WEPP Provenance
Within WEPP/WEPP-Forest the per-class sediment concentrations are derived directly from erosion physics:

- `sloss.for:305-316` computes `sedcon(ipart,nplane)` on the last OFE of every hillslope as:  
  `sedcon = avsole / (runoff * efflen) * frcflw`, resulting in kg m⁻³.
- `wshimp.for:235-244` performs the same calculation for impoundment routed flows: class-specific sediment masses (`clout`, `slout`, …) are divided by `runvol` to yield kg m⁻³.

The WEPP particle system sets explicit specific gravities (unitless) for the five canonical classes inside `prtcmp.for:122-126`:

| Class | Description | Specific Gravity | Density (kg m⁻³) |
| --- | --- | --- | --- |
| 1 | Primary clay | 2.60 | 2 600 |
| 2 | Primary silt | 2.65 | 2 650 |
| 3 | Small aggregates | 1.80 | 1 800 |
| 4 | Large aggregates | 1.60 | 1 600 |
| 5 | Primary sand | 2.65 | 2 650 |

Specific gravity × 1 000 kg m⁻³ (density of water) yields the absolute particle density used to convert mass to volume.

### Implementation in `totalwatsed3`

The required native producer implements the following steps for every grouped day:

1. Aggregate class masses per day: `seddep_i = Σ sedcon_i * runvol`.
2. Compute total sediment delivery: `sed_del = Σ_i seddep_i`.
3. Convert each class to solids volume: `V_i = seddep_i / ρ_i` where `ρ_i` is the density from the table above.
4. Sum the solids volume: `V_total = Σ_i V_i`.
5. Divide by total runoff: `sed_vol_conc = V_total / runvol_total` (guarded so zero runoff yields zero concentration).
6. When ash is present, compute `ash_vol_conc` using per-hillslope ash masses and bulk densities from `Ash.meta` (defaulting to baked-in black/white values); `sed+ash_vol_conc` adds sediment solids; `ash_black_pct_by_vol` reports the black-ash share of ash solids volume.

This mirrors the physics in WEPP: the numerator reconstructs the actual cubic meters of sediment solids mobilized that day, while the denominator is the water volume those solids traveled with. The resulting value is dimensionless (m³ m⁻³) and represents the watershed-average volumetric sediment concentration for that day. `totalwatsed3` stores it immediately after the `seddep_*` columns so future stakeholders can append per-class volumetric fractions alongside it.

### Proof Sketch

- WEPP emits `sedcon_i` in kg m⁻³ (`sloss.for:305-316`, `wshimp.for:235-244`).
- `totalwatsed3` multiplies `sedcon_i` by `runvol` (m³) to obtain kg (`seddep_i`).
- Dividing each `seddep_i` by the class density (kg m⁻³) produces a solids volume (m³).
- Summing those volumes and dividing by the aggregated `runvol` (m³) cancels the units, giving m³ m⁻³, matching the physical definition of volumetric concentration.

Consequently `sed_vol_conc` is a direct, unit-consistent translation of the WEPP physics into the aggregated daily dataset, with specific-gravity metadata baked in so per-class volumetric columns can be added without re-litigating the derivation.

## Physical runoff versus diagnostics

Named `Runoff` retains threshold-filtered PASS volume over watershed area.
Raw `Q` and `QOFE` columns retain their all-OFE local-area volume sums and are
explicitly labeled internal/version-sensitive diagnostics; neither certifies
physical hillslope outlet runoff or whole-system closure. Physical WAT consumers
use terminal Q separately. Sparse-day TotalWatSed lateral behavior remains a
legacy limitation. AshPost subtracts the PASS contribution already present in
this baseline before adding modeled ash runoff; it verifies numerical agreement
and source/model provenance before publishing corrected results.
