Skip to content

Dumps with no recorded depth use the IPCC uncategorised MCF (0.6), from a single MCF table - #49

Open
HughRunyan wants to merge 6 commits into
mainfrom
msw-ai-data-source-review
Open

Dumps with no recorded depth use the IPCC uncategorised MCF (0.6), from a single MCF table#49
HughRunyan wants to merge 6 commits into
mainfrom
msw-ai-data-source-review

Conversation

@HughRunyan

Copy link
Copy Markdown
Collaborator

What changes

Methane correction factors are now defined once, in a new SWEET_python.mcf
module, and both dump types take the IPCC "uncategorised SWDS" default of 0.6.

Site type MCF before MCF after
Sanitary landfill 1.0 1.0 (unchanged)
Controlled dumpsite 0.7 0.6
Open dumpsite 0.4 0.6
Any dump, depth > 5 m 0.8 0.8 (unchanged)
Any dump, depth ≤ 5 m 0.4 (by fallthrough) 0.4 (explicitly)

IPCC 2006 Vol. 5 Ch. 3 Table 3.1 splits unmanaged sites by waste depth — deeper
than 5 m is "unmanaged deep" (0.8), shallower is "unmanaged shallow" (0.4) — and
prescribes 0.6 where the depth is unknown. SWEET applied the 0.8 bump
wherever a depth was supplied but fell back to 0.4 when it was not, which treats
"we do not know the depth" as "we know it is shallow".

In practice no dump carries a depth at all. In the August 2026 Climate TRACE
input table, depth is populated for 450 of 4,635 sanitary landfills (MCF 1.0
regardless) and for 0 of 8,067 dumpsites and 0 of 959 controlled dumpsites.
"Controlled dumpsite" is not an IPCC category and takes the same uncategorised
value for the same reason: the rows it could plausibly sit in span a wide range
depending on how well a site is run, which is exactly what our inputs do not
record. 0.7 was asserting a management standard we have no evidence for.

Four related pieces come with it:

  1. The depth rule now works in both directions. It was one-directional —
    raise to 0.8 above 5 m, otherwise fall through to the type default — so once
    that default became 0.6 there was no way to reach 0.4 even for a site known
    to be shallow. Now: > 5 m → 0.8, ≤ 5 m → 0.4, absent (None/NaN) → 0.6.
    The comparison is strict, so exactly 5.0 m reads shallow.
  2. No caller fabricates a depth any more. That distinction only works if a
    caller with no depth says so, and several substituted a plausible number
    instead: City.sdst_prepopulate returned 100 m for a landfill or controlled
    dump and 3 m for a dumpsite, derived purely from the site type;
    City.sinar_city_and_site and City.site_only_estimate hardcoded 3 m. All
    pass None now. WasteMAP's paired branch removes the same fabrication on its
    side (the DST form's '3' default and the prepopulate endpoint's 3.0
    fallback).
  3. CityLandfillSpec.depth is new. The city-level advanced DST was the only
    DST path with no way to express depth at all, which is why WasteMAP's advanced
    DST had a Deep/Shallow/Unknown control that could not be sent anywhere.
    Omitted by default, so existing city-adst calls are unchanged.
  4. Ten copies collapse to one. The table and the depth rule were duplicated
    across four site-type-name tables, three landfill-type-index tables,
    dst_common.MCF_BY_TYPE, the dump conversion in
    City.implement_dst_changes_simple_v1_5, and the generic city dumpsite bucket
    in City._calculate_divs — and had drifted into two families, dict-keyed
    (controlled dump 0.7) and list-keyed (0.6). All ten now call
    mcf.mcf_for_site(site_type_idx, depth). dst_common re-exports
    MCF_BY_TYPE, SITE_TYPE_NAMES, DEEP_SITE_DEPTH_M, DEEP_DUMP_MCF and
    DEEP_MCF_DUMP_TYPES, so existing importers are unaffected.

Model-output impact

MCF multiplies generation linearly, so the effect is proportional and easy to
predict:

  • +50% modelled methane generation at every open dump
  • −14% at every controlled dumpsite
  • +50% for any /sdst run on a site with no recorded depth (0.4 → 0.6)
  • −33% for a /sdst run on a dump with a recorded depth at or below 5 m
    (0.6 → 0.4), which previously could not be expressed
  • Sanitary landfills and dumps recorded deeper than 5 m are unchanged

Two consequences worth carrying forward:

  • MCF no longer distinguishes the two dump types, so a scenario that converts
    an open dump to a controlled dump now changes emissions only through cover
    oxidation and gas capture, which SWEET models separately. The strategy of
    raising MCF by reclassifying a site is gone from the DST.
  • Because open dumps outnumber controlled dumpsites roughly 8:1 in the Climate
    TRACE input table, the net direction for a full TRACE run is up, on the
    order of +16% global CH4.

Bug fixed along the way

City.sinar_city_and_site tested site_type in (1, 2) against a site-type
string in its second per-landfill branch, while the first branch correctly used
site_type_idx, so the deep-dump MCF bump could never fire there. The hardcoded
3 m depth on that path meant the condition was already always false, so nothing
changes in output; the defect would have surfaced the moment a real depth reached
it. Both branches call mcf.mcf_for_site now.

mcf.mcf_for_site also tolerates a None/NaN depth, so
City.advanced_baseline no longer raises TypeError on its own
depth: float = None default.

Paired branches

Same branch name in each repo, per our cross-repo convention:

  • RMI_Climate_TRACE_Waste_Methanemsw-ai-data-source-review (PR #93). The
    TRACE deploy workflow resolves SWEET by matching branch name, so this branch
    must exist on the remote for a TRACE run to pick the change up.
  • WasteMAPmsw-ai-data-source-review (backend + SDST) and
    adst-depth-reaches-the-model (ADST, off the proto line).
  • This repo also carries adst-depth-reaches-the-model as an alias at the same
    commit, so WasteMAP's ADST branch resolves a matching SWEET ref.

Acceptance Criteria

  • Every MCF value and the depth rule resolve through one module,
    SWEET_python.mcf; no other module hardcodes an MCF table
  • Both dump types return 0.6 when no depth is supplied
  • A supplied depth selects 0.8 (> 5 m) or 0.4 (≤ 5 m) on both dump types, and
    never applies to a sanitary landfill
  • None and NaN depths are accepted everywhere and mean "unknown"
  • No SWEET call site invents a depth from the site type
  • CityLandfillSpec.depth threads a per-landfill depth into
    dst_common.mcf_series, defaulting to omitted
  • dst_common still exports its previous MCF names for existing importers
  • Changelog records the change and its model-output effect

Definition of Done

  • Acceptance criteria met
  • Tests pass — tests/test_mcf.py is new (22 cases; MCF had no coverage at
    all before), full suite 114 passed
  • Docs updated — changelog/2026-08.md and changelog/README.md
  • Reviewed and merged, together with the paired branches above

HughRunyan and others added 4 commits August 25, 2026 06:52
…hallow (0.4)

IPCC splits unmanaged SWDS by waste depth: >5 m is "unmanaged deep" (MCF 0.8),
shallower is "unmanaged shallow" (0.4), and a site whose depth is unknown is
"uncategorised" (0.6). SWEET applied the 0.8 bump wherever a depth was supplied
but fell back to 0.4 when it was not, treating "depth unknown" as "known to be
shallow".

No dump carries a depth in practice. In the August 2026 Climate TRACE input
table, depth is populated for 450 of 4,635 sanitary landfills (where MCF is 1.0
regardless) and for 0 of 8,067 dumpsites and 0 of 959 controlled dumpsites, so
the deep-dump rule never fires and 0.4 was applied universally.

Changed in every MCF table: dst_common.MCF_BY_TYPE; the site-type tables in
sinar_city_and_site, site_only_estimate, site_only_estimate_trace and
citysite_estimate_trace; the landfill-type tables in sdst_v1_5,
implement_dst_changes_advanced and advanced_baseline; the pre-implementation
dump MCF in implement_dst_changes_simple_v1_5; and the generic city dumpsite
bucket in _calculate_divs. The depth > 5 m rule is untouched.

Model-output change: MCF multiplies generation linearly, so methane generation
rises 50% at every open-dump site.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…F table

Follows the open-dump 0.4 -> 0.6 move in 6aa9e6b with the other half of the
same argument.

Controlled dumpsites drop 0.7 -> 0.6. "Controlled dumpsite" is not an IPCC
category and does not map onto one: the rows a partly-managed site could sit in
span a wide range depending on how well it is run, which is exactly what our
inputs do not record. 0.7 asserted a management standard we have no evidence
for; 0.6 ("uncategorised SWDS") is the value IPCC prescribes for not knowing.
Waste depth is populated for 0 of 959 controlled dumpsites, same as for
dumpsites.

A supplied depth now selects the specific IPCC category in both directions.
The rule only ever raised MCF to 0.8 above 5 m and otherwise fell through to
the type default, so once that default became 0.6 there was no route to
"unmanaged shallow" (0.4) even for a site known to be shallow. Now: above 5 m
0.8, at or below 0.4, absent (None/NaN) 0.6. Live on the two Climate TRACE site
paths, which read a real waste_depth, and on advanced_dst, whose depth field is
optional.

Not applied on the /sdst request paths, which pass the new
trust_shallow_depth=False. Those callers always send a number - the DST form
defaults depth to 3 m and sdst_prepopulate fabricates 3 m for any dumpsite - so
"no answer" cannot be told apart from "3 m", and a shallow reading would drop
every default scenario to 0.4. Closing that needs the caller to send a null.

The table itself moves to a new SWEET_python.mcf module. It had been copied into
ten call sites and drifted apart into a dict-keyed family (controlled dump 0.7)
and a list-keyed family (0.6); all ten now resolve through mcf.mcf_for_site.
dst_common re-exports its old names, so importers are unaffected. First test
coverage for MCF, in tests/test_mcf.py.

Also fixed along the way:
- sinar_city_and_site tested `site_type in (1, 2)` against a site-type string in
  its second per-landfill branch, so the deep-dump bump could never fire there.
  Masked by the hardcoded depth, so no output change.
- sinar_city_and_site and site_only_estimate hardcoded `depth = 3` as a stand-in
  for "no depth available", which read as a claim the site was shallow. Both now
  pass None. No output change.
- advanced_baseline no longer raises TypeError on its own `depth=None` default.

Model-output change: MCF multiplies generation linearly, so modelled methane
generation falls 14% at every controlled dumpsite. MCF no longer distinguishes
the two dump types, so converting an open dump to a controlled dump changes
emissions only through oxidation and gas capture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two follow-ons to the MCF work, both from the same principle: a depth is a claim
about a site, so code with no depth to offer has to say "unknown" rather than
pick a plausible number.

City-level ADST gains CityLandfillSpec.depth, threaded into mcf_series exactly
as the single-site endpoint does. It was the only DST path that could not
express depth at all, which is why WasteMAP's advanced DST shipped a
Deep/Shallow/Unknown control with nowhere to send it. Optional, so existing
calls are unchanged.

sdst_prepopulate stops deriving a depth from the site type - 100 m for a
landfill or controlled dump, 3 m for a dumpsite. Nothing on that path ever knew
a depth; it now returns None. Under the two-directional rule the old values were
actively wrong, reading as "definitely deep" or "definitely shallow".

With no caller left inventing a depth, mcf_for_site's trust_shallow_depth guard
goes away and every path takes a supplied depth at face value: >5 m is 0.8, <=5 m
is 0.4, absent is 0.6.

Model-output change for /sdst on a site with no recorded depth: MCF 0.4 -> 0.6,
so modelled generation rises 50%. Pairs with WasteMAP branches
msw-ai-data-source-review (backend + SDST form) and adst-depth-reaches-the-model.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assigned and never read. Harmless, but it is the last of the fabricated
depths, and leaving it invites someone to wire it up as if it meant something.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@HughRunyan HughRunyan added enhancement New feature or request model-output-change Changes model OUTPUT values (expected progress, not breaking); results differ from prior runs labels Aug 26, 2026
HughRunyan and others added 2 commits August 26, 2026 06:43
…d placeholders

pipeline_runner.main filled an absent waste_depth with 3 on the frame handed to
SWEET as the canonical row. Depth is NULL for all 9,026 dumps in the Climate
TRACE input, so that placeholder alone would have held every dump at the shallow
0.4 and turned this entry's move to 0.6 into a no-op for dumpsites and a
0.7 -> 0.4 drop for controlled dumps. Fixed in RMI_Climate_TRACE_Waste_Methane
on the paired branch (1e2a6d0).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The cutoff was a truncation of the decay tail, not a neutral choice. Methane
emitted today comes from decades of accumulated stock, so zeroing deposition
before the cutoff understates every site with a long landfilling history, worst
in cold/dry climates where the IPCC k values are lowest and the tail longest.
Russia deposited as much municipal waste before 1990 as it did 1990-2021.
Measured on a Russian-profile depthless dumpsite with no reported open year:
2021 emissions rise 1.21x (1.31x in 2015, decaying to 1.16x by 2026 -- the
restored stock is old, so the uplift decays, which is the expected physics).

1970 rather than 1950 because the WPP2024 population series backing the waste
projection is credible per-year that far back while per-capita generation before
~1970 is not, and the residual tail before 1970 is small at every k in
defaults_2019.

_population_series_from_pop_data's bounds now default to the modeling window
instead of literal years. Its all-or-nothing column guard means a pops_yearly.csv
that starts later than MODEL_START_YEAR returns None for EVERY country and
silently reverts a whole run to frozen-CAGR growth; tying the default to the
constant turns that into a visible mismatch instead of a quiet regression.

Requires the matching pops_yearly.csv (1970-2050) in blob before any run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request model-output-change Changes model OUTPUT values (expected progress, not breaking); results differ from prior runs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant