Quantitative Postmortems
Forecast Validation Dashboard
Measured accuracy of CE model projections vs realised outcomes. Each review includes a quantitative error estimate, a benchmark comparison, documented reason for miss, and the model change made in response.
Scope notice:
These reviews cover directional accuracy of CE model families against publicly observed outcomes. Error percentages are CE internal estimates — not independently audited. Quantitative predictions for some reviews are reconstructed from v1–v3 model parameters; original forecast documents predating v3 may not be fully preserved. CE v3.7 reproducibility layer enables forward-looking bitwise-exact scenario replay from this release onwards. See Reproducibility Manifest.
| ID | Forecast | Window | Model Family | Metric | Predicted | Actual | Error % | Direction | Accuracy |
|---|---|---|---|---|---|---|---|---|---|
| FR-001 | CMIP5 high-emissions manufacturing stress case | 2014 → 2024 | Climate | Physical disruption cost index (%) | 3.2% | 4.1% | 28% | under | Partial |
| FR-002 | Early-pandemic macro baseline miss | 2019 → 2021 | Economic | Global GDP growth rate 2020 (%) | 2.3% | -3.1% | 235% | over | Inaccurate |
| FR-003 | Solar LCOE trajectory vs legacy IAM projection | 2016 → 2025 | Energy / Integrated Assessment | Utility-scale solar LCOE ($/MWh, 2025) | 50$/MWh | 27$/MWh | 85% | over | Partial |
| FR-004 | Atlantic coastal insurance market contraction | 2018 → 2025 | Climate-Economy | Insurance carrier count reduction (Florida, 2018–2025, %) | 35% reduction | 35% reduction | 8% | none | Accurate |
Postmortem Detail
FR-001
CMIP5 high-emissions manufacturing stress case
Partially Accurate Climate 2014 → 2024The model directionally captured hotter operating conditions and more frequent disruption, but overstated the uniformity of regional stress.
What worked
The physical direction of travel was right: heat and disruption pressure rose consistently with CMIP5 RCP8.5 projections.
What failed
The scenario compressed regional heterogeneity — Vietnam, Bangladesh, and Thai manufacturing zones invested in passive cooling and water recycling at rates not captured by the global hazard model.
Root cause of miss
Regional heterogeneity of adaptation investment offset projected heat losses in South Asia; Sub-Saharan Africa underestimated
Actual outcome
Observed disruption increased, but regional volatility remained more uneven than the original scenario implied.
Model change made: Regional transmission coefficients disaggregated to 6 climate zones; adaptation pathways added as endogenous input variable in v2.0
FR-002
Early-pandemic macro baseline miss
Inaccurate Economic 2019 → 2021Pre-shock macro baselines missed the scale and speed of pandemic-era distortion. This was an acknowledged out-of-sample event — no structural break was modeled.
What worked
Policy-sensitive sub-models adapted faster once the shock path was made explicit as a scenario input in H2 2020.
What failed
Baseline assumptions were not built for compound global disruption and policy overhang. Pandemic was not in the tail risk library.
Root cause of miss
No pandemic risk pathway existed in the tail scenario library; compound structural-break logic was absent
Actual outcome
Growth collapsed then rebounded sharply while inflation and logistics shocks persisted longer than expected — a compound shock with policy overhang that no pre-2020 model anticipated.
Model change made: Structural break detection added in v2.5; pandemic tail scenario introduced; policy overhang persistence parameter added (calibrated to BIS 2022 post-pandemic study)
FR-003
Solar LCOE trajectory vs legacy IAM projection
Partially Accurate Energy / Integrated Assessment 2016 → 2025Long-run transition direction was right; the model underestimated the pace of solar cost decline by roughly 100%. This was a favourable forecast miss — technology beat projections.
What worked
Strategic direction of technology substitution and cost-curve trend held correctly.
What failed
Wright's Law learning rate for solar was underestimated (40% actual vs 25% assumed). Short-run manufacturing scale-up and policy incentives (IRA, EU Solar Manifesto) not captured.
Root cause of miss
Solar learning rate significantly below empirical rate; supply chain investment and policy incentives not endogenous to model
Actual outcome
Solar LCOE fell faster than any 2016-era projection. Wright's Law learning rates for solar PV have been empirically ~40% since 2010, vs the 25% assumed in CE v2.3.
Model change made: Technology learning rates recalibrated from BNEF and IRENA actuals in v3.6; Wright's Law engine upgraded with annual back-cast validation gate in v3.7
FR-004
Atlantic coastal insurance market contraction
Accurate Climate-Economy 2018 → 2025Forecasts linking hazard concentration to insurance repricing and capital pressure were directionally strong and within the predicted range for primary markets.
What worked
The model linked climate exposure to financial transmission channels (carrier capital adequacy, reinsurance pricing) rather than stopping at hazard maps.
What failed
Florida assignment-of-benefits reform (2023) and Citizens Property Insurance expansion temporarily buffered private market exit pace 2020–2022, creating a 2-year lag vs projection.
Root cause of miss
Regulatory friction (Citizens expansion, AOB reform timing) not captured; delayed rather than avoided the predicted outcome
Actual outcome
Premium pressure, carrier withdrawal, and availability constraints all intensified in exposed regions at rates consistent with the scenario. Regulatory friction temporarily offset some contraction in 2019–2021.
Model change made: Regulatory intervention delay parameter added in v3.5; State-level insurance regulatory friction scenarios added; Note: CAT model is prototype-grade — not actuarially certified