brexit | Carlos Mendez

The Synthetic Control Ladder in Python: A Guided Tour of mlsynth on the Brexit Referendum

Sun, 02 Aug 2026 00:00:00 +0000

Abstract

Estimating what a policy cost requires building a version of the world in which it never happened, and the software for doing that is scattered across a dozen packages with a dozen different interfaces. This tutorial is a guided tour of mlsynth, a Python library that puts ninety-two panel-data causal estimators behind a single configuration dictionary and a single result object, and it uses that library to climb the whole ladder of single-treated-unit estimators — difference-in-differences, synthetic control, demeaned synthetic control, synthetic difference-in-differences in three flavours, matching-and-synthetic-control and augmented synthetic control — one class per stage. The data are quarterly log real GDP for 24 OECD economies from 1995Q1 to 2020Q4, leaving the United Kingdom as the treated unit, 23 donors and 86 pre-treatment quarters. Dating the referendum at 2016Q3 and matching on outcomes alone, the estimated shortfall in UK GDP at the end of 2018 is 3.04% under synthetic control, 2.99% under demeaned SC, 2.80% under SDID, 2.73% under MASC and 3.04% under augmented SC, widening to between 3.83% and 4.19% a year later — every one above the 2.4% previously published for this dataset. An in-sample placebo tournament over twenty artificial treatment dates ranks the SDID family first at 0.0066 log points of root mean squared error against 0.0086 for plain synthetic control. Three findings emerge that only a package-level reading produces: three of the library’s defaults each change the answer by more than the spread across the entire ladder, one estimator silently rounds the number you are most likely to quote, and the three routes mlsynth offers for “controlling for a covariate” disagree by 1.8 percentage points.

1. Overview

On 23 June 2016 the United Kingdom voted to leave the European Union. Three and a half years later UK GDP was some number of percentage points below where it would otherwise have been. The trouble is “otherwise have been”: there is one United Kingdom, it took the treatment, and the version that stayed in the EU exists nowhere in the data.

The standard move is to build that missing country out of the countries we do observe — weight the other OECD economies, add them up, and require the blend to track the real UK quarter by quarter through the two decades before the referendum. That is the synthetic control method, and it is one stage of a ladder that starts at difference-in-differences and climbs through several increasingly flexible estimators.

The R edition of this post climbs that ladder the hard way: every estimator is hand-coded in twenty lines before its package is called, and four separate R packages are needed to cover the six stages. This post makes a different argument. Every stage here is a single class from one library, mlsynth, and the interesting work is not in deriving the estimators but in driving the package: which configuration field selects which estimator, what the result object actually contains, and — the part that turns out to matter most — where a default will quietly hand you a different estimator than the one you meant to fit.

That last point is not a minor caveat. By the end of this post you will have seen three separate defaults that change the headline estimate materially, two of them by more than the spread across the entire six-stage ladder. Knowing the econometrics is not enough. You have to know the software.

1.1 Learning objectives

By the end of this tutorial you will be able to:

Install mlsynth and read its one-config-dict, one-result-object interface, including the Pydantic validation that turns a typo into an exception instead of a silently wrong answer.
Map each stage of the synthetic-control ladder onto a specific mlsynth class and configuration field, and explain why mlsynth.DSC is not the DSC on this ladder.
Extract the average treatment effect on the treated, the donor weights, the time weights, the counterfactual path and the event study from a fitted result.
Identify the three defaults — zeta, set_f and the covariate method — that change the answer materially, and set each one deliberately.
Compare estimators on a common in-sample placebo tournament and read the resulting ranking with appropriate scepticism.
Choose among the wider mlsynth catalogue when your design is not the canonical one-treated-unit case.

1.2 The road ahead

Each stage of this ladder exists because the stage below it gets something wrong. The diagram traces that sequence of complaints, and names the mlsynth class that answers each one.

flowchart TD
D["<b>Panel data</b><br/>24 countries, 104 quarters<br/>one treated unit"] --> Q0{"Which donors<br/>count, and by<br/>how much?"}
Q0 -->|"all of them, equally"| DID["<b>Stage 0 — DiD</b><br/>FDID(...).fit().did"]
DID --> Q1{"But the donors do<br/>not look like the UK"}
Q1 --> SC["<b>Stage 1 — SC</b><br/>VanillaSC"]
SC --> Q2{"But the blend must<br/>match the LEVEL, not<br/>just the shape"}
Q2 --> DSC["<b>Stage 2 — DSC</b><br/>TSSC(method='MSCa')"]
DSC --> Q3{"But every pre-period<br/>counts the same"}
Q3 --> SDID["<b>Stage 3 — SDID</b><br/>SDID(zeta=0.0)"]
SDID --> PIVOT["<b>The pivot</b><br/>extrapolation bias vs<br/>interpolation bias"]
PIVOT --> MASC["<b>Stage 4 — MASC</b><br/>MASC(set_f=...)"]
PIVOT --> ASCM["<b>Stage 5 — ASCM</b><br/>VanillaSC(augment='ridge')"]
MASC --> SEL["<b>Which stage?</b><br/>in-sample placebo<br/>tournament"]
ASCM --> SEL
SEL --> INF["<b>Inference</b><br/>six methods, one flag"]
style D fill:#141413,stroke:#6a9bcc,color:#fff
style DID fill:#8b9dc3,stroke:#141413,color:#fff
style SC fill:#6a9bcc,stroke:#141413,color:#fff
style DSC fill:#00d4c8,stroke:#141413,color:#141413
style SDID fill:#d97757,stroke:#141413,color:#fff
style MASC fill:#6a9bcc,stroke:#141413,color:#fff
style ASCM fill:#6a9bcc,stroke:#141413,color:#fff
style PIVOT fill:#141413,stroke:#d97757,color:#fff
style SEL fill:#1a3a8a,stroke:#141413,color:#fff
style INF fill:#1a3a8a,stroke:#141413,color:#fff

Read the diagram top to bottom as a conversation. Every arrow labelled “but” is an objection to the stage above it, and every box below an objection is the estimator that answers it. The two boxes hanging off the pivot are not a further step up but two different reactions to the same discovery, which is why the ladder branches there rather than continuing.

2. Key concepts

Eight ideas carry the whole post. Two repay slow reading: the distinction between unit weights and time weights, and the difference between an estimator and the solver that fits it.

The missing counterfactual and the donor pool. There is one United Kingdom and it took the treatment. The path it would have followed without Brexit is in no dataset. Synthetic control builds that path from countries that were not treated, and those countries are the donor pool.

Example

After dropping the twelve OECD countries with incomplete records, 24 remain. The UK is the treated unit; the other 23 — from Australia to the United States — are the donor pool. In mlsynth you never name them: the library reads the donor pool off the treat column, which is 1 for the treated unit in post-treatment periods and 0 everywhere else.

Analogy

The master tape of a song is lost and the band has broken up. You hire session musicians and rehearse them against a bootleg until they are indistinguishable from the original, then have them play a song the original band never recorded. The donor pool is the pool of session musicians; the pre-treatment period is the rehearsal.

The simplex. Synthetic control weights must be non-negative and sum to one. That set of allowed weight vectors is the simplex, and the blends it can reach form the convex hull of the donors.

Example

mlsynth reports the constraint it imposed. VanillaSC returns weights.summary_stats["constraint"] as "simplex (non-negative, sum to 1)", and only nine of the 23 donors come back with any weight at all: Hungary 0.2231, the United States 0.1926, Japan 0.1826, Canada 0.1751, Norway 0.1350, and four smaller ones.

Analogy

Hammer a pin into a corkboard for every donor country, stretch a rubber band around all the pins and let it snap tight. Everything inside is reachable by some blend; nothing outside is. Reaching outside would need a negative amount of some country, like a recipe calling for minus two eggs.

Unit weights and time weights. Unit weights say how much each donor country counts. Time weights say how much each pre-treatment quarter counts. Both are chosen by the same kind of optimisation, run in two different directions.

Example

mlsynth keeps them in separate places, and finding the second one is the single most common stumbling block. Unit weights are result.donor_weights; SDID’s time weights are result.cohorts[a].time_weights. Ours put 0.9585 on 2016Q2 and roughly nothing on the other 85 quarters.

Analogy

A mixing desk has two banks of faders. The first sets how loud each instrument is; the second sets which seconds of the rehearsal tape you play back when you check the mix. Difference-in-differences leaves both banks flat. Synthetic control moves the first. Synthetic difference-in-differences moves both.

The intercept, which is a unit fixed effect in disguise. Sometimes the blend moves in near-perfect parallel with the treated unit but sits at a slightly different level. The intercept is the average pre-treatment gap, subtracted off, and adding it is exactly the same as putting a unit fixed effect in the regression.

Example

For the UK, TSSC’s MSCa variant estimates an intercept of $+0.00241$ log points, about a quarter of one per cent of GDP. That is why demeaned SC lands at 2.99% and plain SC at 3.04%: a small intercept is evidence that the SC fit was already level-balanced.

Analogy

Your bathroom scale reads two kilograms heavy. You do not throw it out; you subtract two. Plain synthetic control insists on a scale that is already exactly right and will reject a perfectly consistent one. Demeaned synthetic control just calibrates the offset.

Extrapolation bias and interpolation bias. Two different ways a weighted counterfactual goes wrong. Extrapolation bias: the blend’s characteristics do not match the treated unit’s. Interpolation bias: the characteristics match, but the outcome is a curved function of them, so averaging outcomes is not the same as the outcome at the average. Neither name means quite what you would guess.

Example

The unit weights $\omega$ attack extrapolation bias; matching attacks interpolation bias; SDID’s time weights $\lambda$ are what let one estimator attack both. That claim is the theoretical contribution of the paper this post replicates, and section 16 is where we test it.

Analogy

Extrapolation bias is guessing a stranger’s weight from a photograph of someone else. Interpolation bias is averaging the heights of a five-year-old and a fifty-year-old and calling the result the height of a typical twenty-seven-year-old — both inputs are real people, but growth is not linear in age.

The configuration object. Every mlsynth estimator takes one dictionary, validated by Pydantic. Five fields are always the same: df, outcome, treat, unitid, time. Everything else is estimator-specific.

Example

Swapping VanillaSC for SDID in a script means changing one word. The five data fields, the DataFrame, the treatment indicator and the call pattern Estimator(config).fit() are all identical. The configs set extra="forbid", so writing backendd= instead of backend= raises MlsynthConfigError rather than being ignored.

Analogy

A camera body with interchangeable lenses. The grip, the shutter button and the memory card do not change when you swap a wide angle for a macro; only the glass does, and only the glass has its own settings.

The solver’s fingerprint. An estimator is a mathematical object. Fitting it requires an optimiser, and on a badly conditioned problem two correct optimisers stop in different places.

Example

mlsynth hands the synthetic-control problem to a convex solver and returns 3.039%. R’s synthdid walks the same objective with Frank-Wolfe on a capped iteration budget and stops at 3.06%. Neither is buggy. Section 15 shows why the difference is the interesting part.

Analogy

Two hikers are told to find the lowest point of a wide, almost flat valley in fog. They both walk downhill and they both stop when the ground stops obviously falling away. They end up two hundred metres apart, and both are following the instructions correctly.

A three-letter trap: DSC. mlsynth ships a class named DSC. It is not the estimator on this ladder, and importing the wrong one raises no error at all.

Example

mlsynth.DSC is Distributional Synthetic Control (Gunsilius 2023): it matches whole outcome distributions in Wasserstein space and needs micro-level data with many observations per unit-period. This post’s DSC is Demeaned Synthetic Control, which in mlsynth is TSSC(method="MSCa"). Section 6 is devoted to this.

Analogy

Two colleagues in the same building are both called J. Smith. Sending the quarterly report to the wrong one does not bounce. It just arrives somewhere useless, and you find out weeks later.

With the vocabulary in place, we can install the library.

3. Setup: installing mlsynth

3.1 Install and version pinning

mlsynth is on PyPI, but both the README and the documentation still recommend installing from GitHub — and there is a concrete reason to follow that advice rather than reach for pip install mlsynth.

The PyPI release numbered 1.0.0 is behind git main at the same version number. Installing from PyPI gives you a package that reports mlsynth.__version__ == "1.0.0" but is missing VanillaSCConfig.w_constr, which sections 9.2 and 19 use. Nothing in the version string warns you. So install from git, and pin the commit:

# The moving target:
# pip install -U "git+https://github.com/jgreathouse9/mlsynth.git"
#
# The exact commit this post was verified against:
# pip install -U "git+https://github.com/jgreathouse9/mlsynth.git@15f168b"
#
# Optional extras:
# "mlsynth[design] @ git+..." SCIP solver, for SYNDES / MAREX designs
# "mlsynth[bayes] @ git+..." NumPyro, for SPOTSYNTH's Bayesian mode
# "mlsynth[all] @ git+..." everything
import warnings
warnings.filterwarnings("ignore")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import mlsynth
from mlsynth import FDID, MASC, SDID, TSSC, VanillaSC
from mlsynth.exceptions import MlsynthConfigError, MlsynthDataError
print("mlsynth", mlsynth.__version__)
print("estimators exported:", len(mlsynth.__all__))

mlsynth 1.0.0
estimators exported: 92

Everything below was produced with mlsynth 1.0.0 at commit 15f168b, Python 3.13.11, NumPy 2.3.5, pandas 3.0.1, Matplotlib 3.10.8 and CVXPY 1.8.1. mlsynth requires Python 3.10 or later — the README’s claim of 3.9 is out of date, and pyproject.toml is the authority. The core dependencies are pandas, NumPy, Matplotlib, SciPy, scikit-learn, statsmodels, CVXPY, ECOS, Pydantic and PyArrow; both optional solver backends are lazily imported, so a base install can always import mlsynth and construct any estimator class.

3.2 The library at a glance

Ninety-two exported names is a lot, and the temptation is to reach for whichever class name looks closest to what you want. Resist it: several of the names are near-homonyms of each other. The library ships a machine-readable index of the whole catalogue, which is the fastest way to find out what a class actually does.

from mlsynth._guides_api import get_llm_guide
guide = get_llm_guide() # "concise" | "full" | "practitioner"
print(guide[:420])

# mlsynth
> mlsynth is a strongly-typed Python library of synthetic-control and
> difference-in-differences estimators for causal inference with panel data.
> Every estimator exposes a Pydantic config and a `.fit()` that returns a
> standardized results object. Most are validated against the source paper's
> empirical result, Monte Carlo, or an authoritative reference implementation
> (see the Replications page).

That guide is worth reading in full before you pick an estimator, and section 20 tabulates the part of it relevant to this ladder. For now the important sentence is the second one: every estimator exposes a Pydantic config and a .fit() that returns a standardized result. That uniformity is the whole reason this post can cover six estimators without six separate interfaces to learn.

4. The mlsynth data contract

4.1 Five fields, one long panel

Every estimator in the library wants the same five things: a long-format DataFrame with one row per unit-period, and the names of the outcome, treatment, unit and time columns. The treat column is a 0/1 indicator that is 1 for the treated unit in post-treatment periods and 0 everywhere else — including for the treated unit before treatment.

That convention is worth dwelling on, because it is where most first attempts go wrong. treat is not “this unit is ever treated”; it is “this unit is under treatment right now”. mlsynth reads both the donor pool and the treatment date off that single column.

panel = pd.read_csv("brexit_analysis.csv")
print(f"{panel.shape[0]} rows x {panel.shape[1]} columns")
print(f"{panel.country.nunique()} countries, quarters t = {panel.t.min()}..{panel.t.max()}")
print(panel[["country", "quarter_label", "t", "log_rgdp", "treated"]].head(3).to_string(index=False))

2496 rows x 16 columns
24 countries, quarters t = 1..104
country quarter_label t log_rgdp treated
Australia 1995Q1 1 -0.351514 0
Australia 1995Q2 2 -0.345316 0
Australia 1995Q3 3 -0.339262 0

The panel is quarterly log real GDP for 24 OECD economies over 1995Q1–2020Q4, assembled by Born, Müller, Schularick and Sedláček [2] and redistributed in the replication package of de Brabander, Juodis and Miyazato Szini [1]. There are no missing values, 23 donors, and 86 pre-treatment quarters if we date the treatment at 2016Q3.

4.2 Pydantic configs, and what happens when you get it wrong

The configs are Pydantic v2 models with extra = "forbid". In practice this means the library refuses to accept a field it does not recognise, which is a much better failure mode than silently ignoring it.

base = dict(df=window(T_2018Q4), outcome="log_rgdp", treat="treat",
unitid="country", time="tt", display_graphs=False)
for bad, why in [
(dict(base, backendd="outcome-only"), "misspelled keyword"),
(dict(base, outcome="gdp_log"), "column not in the DataFrame"),
]:
try:
VanillaSC(bad).fit()
except (MlsynthConfigError, MlsynthDataError) as exc:
print(f"{why:<28s} -> {type(exc).__name__}: {str(exc).splitlines()[0][:70]}")

misspelled keyword -> MlsynthConfigError: 1 validation error for VanillaSCConfig
column not in the DataFrame -> MlsynthDataError: Missing required columns in DataFrame 'df': gdp_log

There are four exception types — MlsynthConfigError, MlsynthDataError, MlsynthEstimationError and MlsynthPlottingError — and they tell you which phase failed. A MlsynthConfigError means you wrote the config wrong; a MlsynthDataError means the DataFrame does not satisfy the contract (empty, missing columns, duplicate unit-period pairs); a MlsynthEstimationError means the optimiser gave up. Catching the first two separately from the third is a good habit in any loop over specifications.

4.3 What the data look like before we assume anything

Before fitting anything, look at the twenty-four series.

fig, ax = plt.subplots(figsize=(9.5, 6))
for c in DONORS:
ax.plot(QDEC, Y[c], color=GREY_DONOR, lw=0.8, alpha=0.75)
ax.plot(QDEC, Y[TREATED], color=ORANGE, lw=2.4, label="United Kingdom", zorder=5)
ax.axvline(QDEC[T0], color=LIGHT_TEXT, ls="--", lw=1.0)
ax.set_xlabel("year"); ax.set_ylabel("log real GDP")
plt.savefig("python_sc_dsc_sdid_01_gdp_paths.png", dpi=300, bbox_inches="tight")
plt.show()

The UK is one line among twenty-four, and nothing in the picture tells you what would have happened without the referendum. Notice also that the series are strongly trending and highly correlated: this is close to a set of random walks with drift, which will matter enormously in section 11 when we look at what the time weights do.

4.4 The one trick that makes everything else simple

Every mlsynth estimator reports an ATT averaged over all post-treatment periods. But the question here — and the question in the published tables — is the shortfall at two specific quarters, 2018Q4 and 2019Q4.

There is a neat way to get exactly that without any post-estimation arithmetic. Keep the 86 pre-treatment quarters plus the single quarter of interest, renumber time so it runs 1 to 87, and “the average over all post periods” becomes an average over one period. A bare .fit() then returns precisely the number you want.

T0 = 86 # pre-treatment quarters, through 2016Q2
EVAL = {"2018Q4": 96, "2019Q4": 100} # evaluation quarters, as values of `t`
TREATED = "United Kingdom"
def window(post, pre=T0):
"""`pre` pre-treatment quarters + the given post quarter(s), renumbered 1..pre+1."""
post = [post] if np.isscalar(post) else list(post)
sub = panel[(panel.t <= pre) | (panel.t.isin(post))].copy()
sub["tt"] = sub.groupby("country")["t"].rank(method="dense").astype(int)
sub["treat"] = ((sub.country == TREATED) & (sub.tt > pre)).astype(int)
return sub
def cfg(post, pre=T0, **extra):
"""The five fields every mlsynth estimator wants, plus estimator-specific ones."""
return dict(df=window(post, pre), outcome="log_rgdp", treat="treat",
unitid="country", time="tt", display_graphs=False, **extra)
def pct(att):
"""mlsynth reports treated minus counterfactual. Flip into a % GDP shortfall."""
return -100.0 * att

Those three helpers are the entire scaffolding of this post. cfg is the reason every subsequent code block is one line long, and pct is the reason every number is positive: mlsynth reports the ATT as treated minus counterfactual, which is negative when the treatment hurt, and the literature reports Brexit as a positive percentage loss.

In formal terms, the estimand throughout is the average treatment effect on the treated at a single period,

$$\tau_t = Y_{\text{UK},t}(1) - Y_{\text{UK},t}(0),$$

where $Y_{\text{UK},t}(0)$ is the unobserved no-Brexit path. Every stage is a different estimator of that same quantity, and we report $-100 \times \tau_t$ so that a larger number means a larger loss.

5. Anatomy of a fit

5.1 Config in, standardized result out

The call pattern never changes: build a config, construct the estimator, call .fit(), read the result.

res = VanillaSC(cfg(96, inference=False)).fit()
print(f"type(result) {type(res).__name__}")
print(f"result.att {res.att:+.6f}")
print(f"result.pre_rmse {res.pre_rmse:.6f}")
print(f"result.fit_diagnostics.r_squared_pre {res.fit_diagnostics.r_squared_pre:.6f}")
print(f"result.method_details.method_name {res.method_details.method_name}")
print(f"len(result.donor_weights) {len(res.donor_weights)}")
print(f"result.counterfactual.shape {np.shape(res.counterfactual)}")

type(result) BaseEstimatorResults
result.att -0.030388
result.pre_rmse 0.005589
result.fit_diagnostics.r_squared_pre 0.998060
result.method_details.method_name VanillaSC[outcome-only]
len(result.donor_weights) 9
result.counterfactual.shape (87,)

Two things are worth noticing. len(result.donor_weights) is 9, not 23: mlsynth returns only the donors with non-zero weight, so do not treat that dictionary as a dense vector over the donor pool. And method_details.method_name reports VanillaSC[outcome-only], which is the backend the "auto" setting resolved to — the library tells you which algorithm it actually ran, which is exactly the information you need when comparing against another implementation.

The result is a BaseEstimatorResults (aliased EffectResult), organised into six sub-models plus a set of flat convenience accessors:

Sub-model	Contains
`effects`	`att`, `att_percent`, `att_std_err`, `additional_effects`
`fit_diagnostics`	`rmse_pre`, `r_squared_pre`, `rmse_post`
`time_series`	`observed_outcome`, `counterfactual_outcome`, `estimated_gap`, `time_periods`
`weights`	`donor_weights`, `time_weights`, `unit_weights`, `summary_stats`
`inference`	`p_value`, `ci_lower`, `ci_upper`, `standard_error`, `method`, `details`
`method_details`	`method_name`, `is_recommended`, `parameters_used`

Flat accessor	Resolves to
`.att`	`effects.att`
`.att_ci`	`(inference.ci_lower, inference.ci_upper)`
`.counterfactual`	`time_series.counterfactual_outcome`
`.gap`	`time_series.estimated_gap`
`.donor_weights`	`weights.donor_weights`
`.weight_vector`	dense array of `donor_weights.values()`
`.pre_rmse`	`fit_diagnostics.rmse_pre`

Learn the seven flat accessors and you can read the output of any effect estimator in the library without opening its documentation.

5.2 Plotting: what the package gives you for free

Every effect result carries a .plot() method driven by a PlotConfig. Set display_graphs=True and you get a figure without writing any Matplotlib at all.

res = VanillaSC(cfg(96, inference=False)).fit()
fig, axes = plt.subplots(1, 2, figsize=(11, 4.2))
# display=False on every call. Without it .plot() ends in plt.show(), which in
# a notebook flushes the figure after the FIRST panel and leaves the second empty.
res.plot(kind="counterfactual", ax=axes[0], display=False)
res.plot(kind="gap", ax=axes[1], display=False)
plt.tight_layout()
plt.show()

That figure is deliberately unstyled — it is what the library draws with no help from us, and it is already publishable. kind takes "auto", "counterfactual" or "gap"; passing ax= lets you compose several results into one panel.

The display=False above is not cosmetic, and it is worth a paragraph because it is a fourth silent default. .plot() ends with if pc.display: plt.show(), and the PlotConfig it consults is self.plot_config or PlotConfig() — but the fitted result’s plot_config is None, so the display_graphs=False you set on the config never reaches it and the fallback PlotConfig() has display=True. In a script the stray plt.show() is a harmless no-op under the Agg backend. In a notebook it flushes and closes the figure after the first panel, so the second panel renders blank and a stray <Figure size 640x480 with 0 Axes> appears underneath. The per-call display=False override is the fix; the same applies any time you compose two .plot() calls into one figure.

Cosmetics are configured through the nested plot field rather than through Matplotlib, so a house style travels with the config:

from mlsynth.config_models import PlotConfig
styled = VanillaSC(cfg(96, inference=False, plot=PlotConfig(
observed_color="#d97757",
counterfactual_colors=["#6a9bcc"],
counterfactual_linestyle="--",
xlabel="quarter index (renumbered 1..87)",
ylabel="log real GDP",
title="VanillaSC via PlotConfig",
display=False,
))).fit()

The legacy flat fields treated_color and counterfactual_color still work — note that the latter takes a list, not a string — but PlotConfig is the maintained route and supports themes, save targets and per-call overrides.

Every remaining figure in this post is drawn by hand in the site’s dark palette, because a tutorial benefits from consistent colours. In your own work, display_graphs=True is usually enough.

5.3 Where estimators deviate from the standard result

Three of the six estimators return a richer object than BaseEstimatorResults, and each adds exactly what its method needs:

Estimator	Returns	The extra you need
`FDID`	`FDIDResults`	`.fdid` and `.did`, two `FDIDMethodFit` objects — the forward-selected fit and the plain two-way DiD benchmark
`TSSC`	`TSSCResults`	`.variants` (a dict of the four MSC variants), `.selection` (the Step-1 tests), `.recommended_method`
`SDID`	`SDIDResults`	`.inference_detail`, `.event_study`, `.cohorts[a]` (which is where the time weights live)

All three still populate the standard sub-models and flat accessors, so .att works everywhere. The extras are additive, not alternative.

6. A naming hazard, before we import anything

This section exists because getting it wrong costs nothing at runtime and everything in interpretation.

mlsynth.DSC is not the DSC on this ladder.

mlsynth.DSC is Distributional Synthetic Control (Gunsilius 2023, Econometrica). It reconstructs the treated unit’s whole outcome distribution as a Wasserstein-space average of donor distributions and returns quantile treatment effects. It requires micro-level data: many individual observations per unit-period. Handed a panel like ours, with one observation per country-quarter, it is answering a question the data cannot support.
The DSC on this ladder is Demeaned Synthetic Control (Doudchenko and Imbens [8]; Ferman and Pinto [9]) — synthetic control plus an intercept. In mlsynth it is TSSC(method="MSCa").

Same three letters, different estimators, and no error is raised. The mapping used throughout this post follows mlsynth issue #312, which is itself a reading of the paper we replicate.

The library has several more acronym neighbours worth knowing about before you reach for one by name:

Class	Is actually	Not to be confused with
`DSC`	Distributional SC (Gunsilius)	demeaned SC = `TSSC(method="MSCa")`
`DSCAR`	Dynamic SC for Auto-Regressive processes	either of the above
`SCD`	SC with Differencing	`DSC`
`DRSC`	Distribution-Regression SC	`DSC`
`MEDSC`	Mediation SC	demeaned SC

The general lesson is that in a ninety-two-class library, the class name is a mnemonic and not a definition. Check the docstring before you fit.

7. One regression, six sets of weights

Before the code, one unifying idea. Every stage on this ladder is the same weighted two-way regression,

$$(\hat\tau, \hat\mu, \hat\alpha, \hat\beta) = \arg\min \sum_{i,t} \omega_i \lambda_t \left( Y_{it} - \mu - \alpha_i - \beta_t - \tau D_{it} \right)^2,$$

with a different choice of the unit weights $\omega_i$ and the time weights $\lambda_t$. In words: fit a two-way fixed-effects regression, but let some units and some periods count more than others. The stages differ only in how those two weight vectors are chosen.

graph TD
R["<b>One weighted two-way regression</b><br/>min Σ ω<sub>i</sub> λ<sub>t</sub> (Y<sub>it</sub> − μ − α<sub>i</sub> − β<sub>t</sub> − τD<sub>it</sub>)²"]
R --> A["ω uniform, λ uniform<br/><b>DiD</b>"]
R --> B["ω fitted, λ uniform, no intercept<br/><b>SC</b>"]
R --> C["ω fitted, λ uniform, intercept<br/><b>DSC</b>"]
R --> D["ω fitted, λ fitted, intercept<br/><b>SDID</b>"]
R --> E["<b>Change the feasible set instead</b>"]
E --> F["blend with m-nearest-neighbour matching<br/><b>MASC</b>"]
E --> G["allow negative weights, penalise them<br/><b>ASCM</b>"]
style R fill:#141413,stroke:#6a9bcc,color:#fff
style A fill:#8b9dc3,stroke:#141413,color:#fff
style B fill:#6a9bcc,stroke:#141413,color:#fff
style C fill:#00d4c8,stroke:#141413,color:#141413
style D fill:#d97757,stroke:#141413,color:#fff
style E fill:#141413,stroke:#d97757,color:#fff
style F fill:#6a9bcc,stroke:#141413,color:#fff
style G fill:#6a9bcc,stroke:#141413,color:#fff

The diagram splits the ladder into two families. The first four stages change which weights the same regression uses. The last two change what weights are allowed at all — MASC by mixing in a different estimator, ASCM by relaxing the simplex. That distinction is why the ladder branches rather than continuing upward, and it is the reason section 16’s tournament cannot simply declare the top stage the winner.

Here is the whole ladder as mlsynth code, which is the table to bookmark:

Stage	$\omega$	$\lambda$	mlsynth call
DiD	uniform	uniform	`FDID(cfg).fit().did`
SC	fitted, simplex	uniform	`VanillaSC(cfg)`
DSC	fitted, simplex + intercept	uniform	`TSSC(cfg, method="MSCa")`
SDID	fitted, simplex + intercept	fitted, simplex	`SDID(cfg, zeta=0.0)`
MASC	convex blend of matching and SC	uniform	`MASC(cfg, set_f=...)`
ASCM	SC weights plus a ridge correction	uniform	`VanillaSC(cfg, augment="ridge")`

Now we climb it.

8. Stage 0 — Difference-in-differences

mlsynth has no standalone DiD class, and this trips people up. The plain two-way estimator comes free inside FDID (Forward DiD, Li 2024), which fits its own estimator and the textbook benchmark side by side and exposes them as .fdid and .did.

fdid_res = {k: FDID(cfg(e)).fit() for k, e in EVAL.items()}
did = {k: r.did for k, r in fdid_res.items()}
print(f"DiD 2018Q4 {pct(did['2018Q4'].att):.2f}% 2019Q4 {pct(did['2019Q4'].att):.2f}%")
print(f"SE (analytic) {100 * did['2018Q4'].att_se:.2f}")
wd = did["2018Q4"].donor_weights
print(f"donor weights: {len(wd)} donors, all equal to {next(iter(wd.values())):.6f} = 1/{len(wd)}")
print(f"pre-treatment RMSE {did['2018Q4'].pre_rmse:.5f}, R^2 {did['2018Q4'].r_squared:.4f}")

DiD 2018Q4 4.98% 2019Q4 6.18%
SE (analytic) 2.19
donor weights: 23 donors, all equal to 0.043478 = 1/23
pre-treatment RMSE 0.02170, R^2 0.9706

Difference-in-differences puts the Brexit cost at 4.98% of GDP by the end of 2018 — far above every other stage, and far above the 2.4% previously published. The uniform $1/23 = 0.043478$ weights are the giveaway that nothing was fitted: DiD assumes the average of all twenty-three OECD economies would have moved in parallel with the UK, and the pre-treatment RMSE of 0.0217 says it did not. That number is roughly four times the 0.0056 that synthetic control achieves, and it is the entire reason the rest of the ladder exists.

A free seventh stage. The same call also gives you Forward DiD, which selects a subset of donors greedily and then runs DiD on them. It is not on the paper’s ladder, but it costs nothing:

f18 = fdid_res["2018Q4"].fdid
print(f"Forward DiD selects {len(f18.selected_names)} donors: {', '.join(f18.selected_names)}")
print(f"Forward DiD 2018Q4 {pct(f18.att):.2f}% pre-RMSE {f18.pre_rmse:.5f}")

Forward DiD selects 4 donors: Norway, Hungary, Austria, United States
Forward DiD 2018Q4 2.42% pre-RMSE 0.00880

Forward DiD picks four donors, cuts the pre-treatment RMSE from 0.0217 to 0.0088, and lands at 2.42% — remarkably close to Born et al.’s published 2.4%, and the lowest estimate anywhere in this post. Three of its four donors (Norway, Hungary, the United States) are also among synthetic control’s five largest weights, which is reassuring: two quite different selection procedures are finding the same countries.

9. Stage 1 — Synthetic control

DiD’s complaint is that the donor average does not look like the UK. Synthetic control fixes that by fitting the unit weights, subject to the simplex constraint

$$\hat\omega = \arg\min_{\omega \in \mathbb{W}} \sum_{t=1}^{T_0} \left( Y_{\text{UK},t} - \sum_j \omega_j Y_{j,t} \right)^2, \qquad \mathbb{W} = \left\{ \omega : \omega_j \ge 0, \sum_j \omega_j = 1 \right\}.$$

In words, pick the non-negative weights summing to one that make the blend track the UK as closely as possible over the 86 pre-treatment quarters. In code, Y is the log_rgdp column, the sum over $j$ runs over DONORS, and $T_0$ is T0 = 86.

sc = {k: VanillaSC(cfg(e, inference=False)).fit() for k, e in EVAL.items()}
s18 = sc["2018Q4"]
print(f"SC 2018Q4 {pct(s18.att):.2f}% 2019Q4 {pct(sc['2019Q4'].att):.2f}%")
print(f"backend chosen by 'auto': {s18.method_details.method_name}")
print(f"pre-RMSE {s18.pre_rmse:.6f} R^2 {s18.fit_diagnostics.r_squared_pre:.5f}")
ss = s18.weights.summary_stats
print(f"weights: {ss['n_nonzero']} nonzero, sum {ss['sum_of_weights']:.6f}, {ss['constraint']}")
for c, w in sorted(s18.donor_weights.items(), key=lambda kv: -kv[1]):
if w > 0.01:
print(f" {c:<16s} {w:.4f}")

SC 2018Q4 3.04% 2019Q4 4.17%
backend chosen by 'auto': VanillaSC[outcome-only]
pre-RMSE 0.005589 R^2 0.99806
weights: 9 nonzero, sum 1.000000, simplex (non-negative, sum to 1)
Hungary 0.2231
United States 0.1926
Japan 0.1826
Canada 0.1751
Norway 0.1350
Ireland 0.0523
Italy 0.0196
Portugal 0.0124

Synthetic control puts the shortfall at 3.04% by the end of 2018 and 4.17% a year later, with a pre-treatment $R^2$ of 0.998. The synthetic UK is roughly a fifth Hungary, a fifth the United States, a fifth Japan, a sixth Canada and an eighth Norway. Fourteen of the twenty-three donors get nothing at all — this sparsity is a feature of the simplex constraint, not an accident, and it is why synthetic control estimates are usually easy to describe in a sentence.

Note that the United States carries about a fifth of the counterfactual. That will matter in section 19, when we ask whether the answer survives dropping it.

9.1 The five backends

VanillaSC has a backend field with five settings, and understanding when each applies saves a lot of confusion:

`backend`	What it does	When it applies
`"auto"` (default)	`"outcome-only"` without covariates, `"mscmt"` with them	always a safe default
`"outcome-only"`	convex simplex fit on pre-treatment outcomes	no covariates; unique solution
`"mscmt"`	global differential-evolution search over predictor weights $V$	covariates (Becker-Klössner)
`"malo"`	corner search over $V$ (Malo et al. 2024)	covariates
`"penalized"`	Abadie-L’Hour unique/sparse estimator	covariates, when uniqueness matters

The distinction that matters: with no covariates the problem is convex and has a unique solution; with covariates it becomes a bilevel program whose predictor weights are generically not identified. That is why VanillaSC reports a v_agreement diagnostic alongside covariate fits — small means the predictor weights are well identified, large means they are fragile. We come back to this in section 17.

9.2 One estimator, four solvers

Here is where the package-level reading earns its keep. Plain synthetic control is a single mathematical object, but mlsynth can reach it four different ways, and R’s synthdid reaches it a fifth.

(tssc_att below is a three-line helper that reads TSSC’s estimate off its gap series rather than its rounded .att field. Section 10.1 explains why it is needed; for now take it as “the unrounded ATT”.)

for label, klass, kw in [
("VanillaSC, backend='auto'", VanillaSC, dict(inference=False)),
("VanillaSC, backend='outcome-only'", VanillaSC, dict(backend="outcome-only", inference=False)),
("VanillaSC, w_constr='simplex'", VanillaSC, dict(w_constr="simplex", inference=False)),
("TSSC, method='SC'", TSSC, dict(method="SC", inference=False)),
]:
r = klass(cfg(96, **kw)).fit()
att = tssc_att(r, "SC") if klass is TSSC else r.effects.att
print(f"{label:<36s} 2018Q4 {pct(att):.3f}")
# Reference value from the R edition, not refitted here (see section 15).
print(f"{'R synthdid (Frank-Wolfe, published)':<36s} 2018Q4 3.060")

VanillaSC, backend='auto' 2018Q4 3.039
VanillaSC, backend='outcome-only' 2018Q4 3.039
VanillaSC, w_constr='simplex' 2018Q4 3.039
TSSC, method='SC' 2018Q4 3.039
R synthdid (Frank-Wolfe, published) 2018Q4 3.060

Four independent code paths inside mlsynth — two backends, an explicit constraint family, and a completely different estimator class — agree to three decimals at 3.039%. R’s synthdid stops at 3.06%. Section 15 explains why, and argues that this is the most interesting number in the post.

The w_constr field, incidentally, is a general escape hatch. It accepts "simplex", "ols", "lasso", "ridge" and "L1-L2", which lets you ask what the estimate would be under a different feasible set without changing estimator classes. Section 19 uses it.

9.3 The counterfactual path

The windowed fits above answer “how large is the shortfall at 2018Q4”. To draw the whole counterfactual path, fit once on the untruncated panel.

full = cfg(list(range(T0 + 1, 105))) # all 18 post-treatment quarters
sc_full = VanillaSC(dict(full, inference=False)).fit()
cf_sc = np.asarray(sc_full.counterfactual, float).ravel()
gap_sc = np.asarray(sc_full.gap, float).ravel()
print(f"pre-RMSE {sc_full.pre_rmse:.6f}, mean post gap {gap_sc[T0:].mean():+.5f} log points")

pre-RMSE 0.005589, mean post gap -0.02882 log points

The average post-treatment gap is $-0.0288$ log points, or about 2.9% of GDP averaged across all eighteen quarters after the referendum. The bottom panel is the one to look at: the gap oscillates around zero for eighty-six quarters and then goes negative and stays there. That persistence, rather than the size of any single quarter’s gap, is what makes the result credible.

10. Stage 2 — Demeaned synthetic control

Synthetic control insists the blend match the UK’s level. Sometimes a blend tracks the shape perfectly while sitting slightly above or below, and plain SC will reject it in favour of a worse-shaped blend at the right level. Demeaned SC adds one free parameter — a constant offset — so the blend has to match the shape but not the level.

In mlsynth this is the MSCa variant of the two-step estimator of Li and Shankar (2023).

dsc = {k: TSSC(cfg(e, method="MSCa", inference=False)).fit() for k, e in EVAL.items()}
d18 = dsc["2018Q4"].variants["MSCa"]
print(f"DSC 2018Q4 {pct(tssc_att(dsc['2018Q4'])):.2f}% 2019Q4 {pct(tssc_att(dsc['2019Q4'])):.2f}%")
print(f"MSCa intercept: {d18.intercept:+.6f} log points = {100 * d18.intercept:+.3f}% of GDP")

DSC 2018Q4 2.99% 2019Q4 4.12%
MSCa intercept: +0.002410 log points = +0.241% of GDP

The estimated offset is $+0.0024$ log points, about a quarter of one per cent of GDP. That is small, and its smallness is informative: it says the SC fit was already close to level-balanced, which is why DSC’s 2.99% sits so near SC’s 3.04%.

10.1 A precision trap worth knowing about

TSSC rounds every scalar it reports. The series it returns are full precision, but .att, .rmse_pre and the donor weights are rounded before they reach you.

print(f"variants['MSCa'].att {d18.att!r} -> {pct(d18.att):.4f}%")
print(f"gap[-1] (full precision) {np.asarray(d18.gap, float)[-1]!r}")
print(f" -> {pct(tssc_att(dsc['2018Q4'])):.4f}%")
print(f"variants['MSCa'].rmse_pre {d18.rmse_pre!r}")

variants['MSCa'].att -0.03 -> 3.0000%
gap[-1] (full precision) np.float64(-0.0298873295228968)
-> 2.9887%
variants['MSCa'].rmse_pre 0.006

Take the headline off .att and DSC reads exactly 3.00%. Take it off the gap series and it reads 2.99%, which is what mlsynth’s own replication table reports and what the published paper reports. A hundredth of a percentage point is not going to change anyone’s policy view, but it will make you think you have failed to replicate a table when you have not. The fix is three lines:

def tssc_att(res, variant="MSCa", n_post=1):
"""Full-precision ATT for a TSSC variant, read off the unrounded gap series."""
gap = np.asarray(res.variants[variant].gap, float).ravel()
return float(gap[-n_post:].mean())

The general lesson generalises past TSSC: when a package hands you both a scalar summary and the series it was computed from, and the two disagree, trust the series.

10.2 The four variants and the Step-1 selection

Left to itself, TSSC fits all four variants of the Li-Shankar estimator and runs a subsampling procedure to pick one. method= forces a single variant and skips the selection entirely.

t_all = TSSC(cfg(96, draws=500, seed=SEED)).fit() # no method= : fit all four
print(f"TSSC recommends: {t_all.recommended_method}")
for m in ("SC", "MSCa", "MSCb", "MSCc"):
v = t_all.variants[m]
ic = "none" if v.intercept is None else f"{v.intercept:+.5f}"
print(f" {m:<5s} loss {pct(tssc_att(t_all, m)):5.3f}% "
f"(.att reports {v.att:+.5f}) intercept {ic}")
for name, test in t_all.selection.tests.items():
print(f" test '{name}': stat {test.statistic:+.5f} "
f"CI [{test.ci_lower:+.5f}, {test.ci_upper:+.5f}] rejected {test.rejected}")

TSSC recommends: SC
SC loss 3.039% (.att reports -0.03000) intercept none
MSCa loss 2.989% (.att reports -0.03000) intercept +0.00241
MSCb loss 3.041% (.att reports -0.03000) intercept none
MSCc loss 3.021% (.att reports -0.03000) intercept +0.00282

 test 'joint': stat +2.21193 CI [+0.02894, +9.10199] rejected False

All four variants land between 2.99% and 3.04%, and all four report .att as exactly $-0.03$ — a compact demonstration of the rounding trap. The four differ in which constraints they impose: SC is the simplex with no intercept, MSCa adds an intercept, MSCb relaxes the sum-to-one constraint, and MSCc relaxes both. The Step-1 test does not reject the restriction, so TSSC recommends plain SC.

The cost of that convenience is real. Fitting four variants with 500 subsampling draws takes about 17 seconds; forcing one variant with inference=False takes 0.01 seconds — a factor of roughly 1,700. If you are running an estimator inside a loop, as we do in section 16, set method= and inference=False. Note that inference=False requires method to be set; asking for no inference without naming a variant raises, because there would be nothing to select with.

11. Stage 3 — Synthetic difference-in-differences

DSC still treats all eighty-six pre-treatment quarters as equally informative. SDID fits the time weights too, solving the transposed version of the same problem: which blend of quarters, judged across all donors, best predicts the treatment quarter.

sdid = {k: SDID(cfg(e, zeta=0.0, vce="placebo", B=500, seed=SEED)).fit()
for k, e in EVAL.items()}
s = sdid["2018Q4"]
print(f"SDID 2018Q4 {pct(s.effects.att):.2f}% 2019Q4 {pct(sdid['2019Q4'].effects.att):.2f}%")
inf = s.inference_detail
print(f"ATT {inf.att:+.5f}, SE {inf.se:.5f}, CI [{inf.ci[0]:+.5f}, {inf.ci[1]:+.5f}]")
print(f"p = {inf.p_value:.4f}, method '{inf.method}', n_placebo {inf.n_placebo}")

SDID 2018Q4 2.80% 2019Q4 3.94%
ATT -0.02801, SE 0.02636, CI [-0.07968, +0.02365]
p = 0.2016, method 'placebo', n_placebo 500

SDID puts the shortfall at 2.80% and 3.94%. Note the standard error: 0.0264 against a point estimate of 0.0280, giving a confidence interval that comfortably contains zero and a placebo p-value of 0.20. With one treated unit and twenty-three donors, that is the honest state of the evidence — section 18 returns to it.

11.1 The one setting that carries the result

default = SDID(cfg(96, vce="noinference")).fit()
print(f"zeta left at its default: {pct(default.effects.att):.2f}%")
print(f"zeta = 0.0: {pct(sdid['2018Q4'].effects.att):.2f}%")

zeta left at its default: 2.67%
zeta = 0.0: 2.80%

zeta is a ridge penalty on the unit weights, and it is on by default. Leave it alone and SDID reports 2.67%; set it to zero, which is what the paper solves, and it reports 2.80%. That 0.13-percentage-point gap is larger than the entire spread between SC, DSC and ASCM.

This is not an mlsynth quirk. Every implementation in every language penalises by default: R’s synthdid needs zeta.omega = 0, Stata’s sdid needs zeta_omega(0), and mlsynth needs zeta=0.0. Stata’s version is the nastiest, because its documented default of 1e-6 is a magic sentinel that requests the full penalty. The lesson survives translation: if you are replicating a published synthetic-DiD number, find out what the authors did with the penalty before you conclude anything.

One companion flag deserves a note because the documentation emphasises it:

ia = SDID(cfg(96, zeta=0.0, intercept_adjust=True, vce="noinference")).fit()
print(f"intercept_adjust=True: {pct(ia.effects.att):.4f}%")

intercept_adjust=True: 2.8012%

Identical to four decimals. intercept_adjust matters when there are several post-treatment periods to average over; under the truncate-and-renumber trick there is exactly one, so there is nothing to adjust. Worth setting anyway if you fit on an untruncated panel.

11.2 Time weights, cohorts and the event study

SDID’s time weights are the one output that is not where you would first look. They live on the cohort object, not on weights:

coh = list(sdid["2018Q4"].cohorts.values())[0]
lam = np.asarray(coh.time_weights, float)
print(f"cohorts: {list(sdid['2018Q4'].cohorts)} (n_treated={coh.n_treated}, n_post={coh.n_post})")
print(f"lambda: {len(lam)} weights summing to {lam.sum():.6f}")
for i in np.where(lam > 1e-4)[0]:
print(f" {QLAB[i]:<8s} {lam[i]:.4f}")

cohorts: [87] (n_treated=1, n_post=1)
lambda: 86 weights summing to 1.000000
2008Q4 0.0386
2014Q3 0.0029
2016Q2 0.9585

The time weights put 95.85% of their mass on 2016Q2, the last pre-treatment quarter, and essentially nothing on the other eighty-five. Difference-in-differences would place the dashed uniform weight, $1/86 \approx 0.0116$, on all of them.

Why? Because log real GDP behaves close to a random walk. If the outcome is a random walk, the best predictor of next quarter is this quarter, and the eighty-five quarters before it add noise rather than information. The source paper reports exactly this collapse and attributes it to the same cause. Do not read it as a bug; read it as SDID correctly discovering that most of the pre-treatment history is not informative about the level at the treatment date.

mlsynth also aggregates an event-study estimator alongside the headline ATT, which the R packages on this ladder do not:

full_sdid = SDID(dict(full, zeta=0.0, vce="placebo", B=200, seed=SEED)).fit()
es = full_sdid.event_study
et, tau = np.asarray(es.event_times, float), np.asarray(es.tau, float)
print(f"event times {et.min():.0f}..{et.max():.0f}")
print(f"post-treatment tau mean {tau[et > 0].mean():+.5f}")
print(f"pre-treatment tau mean {tau[(et < 0) & (et >= -20)].mean():+.5f}")

event times -86..17
post-treatment tau mean -0.02809
pre-treatment tau mean +0.00208

The pre-treatment effects average $+0.0021$ — essentially zero — while the post-treatment effects average $-0.0281$. That flat pre-treatment path is a falsification test the single ATT number cannot give you, and it is available from result.event_study for the cost of one extra line.

11.3 Three flavours of SDID

The time weights have to be fitted against something in the post-treatment period, and there are three natural choices. The published paper reports all three, and only the last falls out of a bare .fit().

Variant	$\lambda$ is fitted to predict	Evaluated at
(i)	the first treated quarter, 2016Q3	any horizon
(ii)	the average of 2016Q3 through the evaluation date	that date
(iii)	the evaluation quarter alone	that date

Variants (i) and (ii) need the weights the result already hands you, applied at a different quarter. This is a good demonstration of reading $\omega$ and $\lambda$ back out of a fitted result and using them yourself:

$$\hat\tau_t = \left( Y_{\text{UK},t} - \sum_j \hat\omega_j Y_{j,t} \right) - \sum_{s \le T_0} \hat\lambda_s \left( Y_{\text{UK},s} - \sum_j \hat\omega_j Y_{j,s} \right).$$

In words: take the gap at the evaluation quarter, then subtract the $\lambda$-weighted average gap over the pre-treatment period. The second term is the bias adjustment, and it is the only thing that differs between the three flavours.

def sdid_weights(post, pre=T0):
"""Fit SDID on a given post window; return (omega dict, lambda array)."""
res = SDID(cfg(post, pre, zeta=0.0, vce="noinference")).fit()
return res.donor_weights, np.asarray(list(res.cohorts.values())[0].time_weights, float)
def sdid_loss(w, lam, t, pre=T0):
"""Apply an (omega, lambda) pair at an arbitrary quarter."""
wv = np.array([w[c] for c in DONORS])
gap = lambda s: float(Y.loc[s, TREATED] - Y.loc[s, DONORS].to_numpy() @ wv)
bias = float(sum(lam[s - 1] * gap(s) for s in range(1, pre + 1)))
return -100.0 * (gap(t) - bias)
w_i, lam_i = sdid_weights(T0 + 1) # (i)
w_ii, lam_ii = sdid_weights(range(T0 + 1, 97)) # (ii)
w_iii, lam_iii = sdid_weights(96) # (iii)
print(f"SDID (i) 2018Q4 {sdid_loss(w_i, lam_i, 96):.3f}")
print(f"SDID (ii) 2018Q4 {sdid_loss(w_ii, lam_ii, 96):.3f}")
print(f"SDID (iii) 2018Q4 {sdid_loss(w_iii, lam_iii, 96):.3f}")

SDID (i) 2018Q4 2.771
SDID (ii) 2018Q4 2.801
SDID (iii) 2018Q4 2.801

The three variants land within 0.03 percentage points of each other, because all three put essentially all their time weight on the same last pre-treatment quarter. Whatever else is uncertain here, the choice among SDID flavours is not where the uncertainty lives — a conclusion the published paper’s placebo table appears to contradict, and section 16 shows why that appearance is an artefact.

11.4 Choosing an inference method

SDID’s vce field takes four values, and the right choice depends on what you are doing:

`vce`	Method	Use when
`"placebo"` (default)	refit treating each donor as pseudo-treated	one treated unit — the only valid choice here
`"jackknife"`	leave-one-treated-unit-out	several treated units
`"bootstrap"`	resample units with replacement	many treated units
`"noinference"`	skip it	inside a loop, where you only want the point estimate

With a single treated unit, jackknife and bootstrap have nothing to resample, so placebo is the only defensible option. "noinference" is what section 16’s tournament uses, and it is the difference between a two-minute loop and a two-hour one.

12. Stage 4 — MASC

Sections 9 to 11 improve the counterfactual by choosing better weights within the simplex. MASC (Kellogg, Mogstad, Pouliot and Torgovitsky [11]) does something different: it forms a convex combination of synthetic control and $m$-nearest-neighbour matching,

$$\hat{Y}^{\text{MASC}} = \phi \cdot \hat{Y}^{\text{match}}_m + (1 - \phi) \cdot \hat{Y}^{\text{SC}},$$

and chooses $m$ and $\phi$ jointly by rolling-origin cross-validation. The motivation is the bias decomposition: synthetic control attacks extrapolation bias, matching attacks interpolation bias, and the cross-validation buys whichever trade-off the data prefer.

M_GRID = list(range(1, 11))
SET_F = list(range(6, T0 + 1))
masc = {k: MASC(cfg(e, m_grid=M_GRID, set_f=SET_F)).fit() for k, e in EVAL.items()}
m18 = masc["2018Q4"]
print(f"MASC 2018Q4 {pct(m18.att):.2f}% 2019Q4 {pct(masc['2019Q4'].att):.2f}%")
print(f"weights.summary_stats: {m18.weights.summary_stats}")

MASC 2018Q4 2.73% 2019Q4 3.83%
weights.summary_stats: {'constraint': 'simplex (matching+SC blend)', 'phi_hat': 0.1576922233857826, 'm_hat': 10}

MASC gives 2.73%, the lowest estimate on the ladder. The two tuned dials are in weights.summary_stats, not in method_details.parameters_used — which is None for this estimator, so looking there first will send you away empty-handed. The cross-validation picked $m = 10$ neighbours and $\phi = 0.158$, so the estimate is roughly one-sixth matching and five-sixths synthetic control.

12.1 The argument that decides the answer

set_f and min_preperiods both control the cross-validation fold set, and they are mutually exclusive. The default is not the fold set the paper uses, and the difference is not subtle.

for label, kw in [
("set_f=range(6, 87) [the paper]", dict(m_grid=M_GRID, set_f=SET_F)),
("min_preperiods=None [default]", dict(m_grid=M_GRID)),
("min_preperiods=43 [ceil(T0/2)]", dict(m_grid=M_GRID, min_preperiods=43)),
]:
vals = [pct(MASC(cfg(e, **kw)).fit().att) for e in EVAL.values()]
print(f"{label:<34s} 2018Q4 {vals[0]:.3f} 2019Q4 {vals[1]:.3f}")

set_f=range(6, 87) [the paper] 2018Q4 2.726 2019Q4 3.828
min_preperiods=None [default] 2018Q4 3.191 2019Q4 4.325
min_preperiods=43 [ceil(T0/2)] 2018Q4 3.191 2019Q4 4.325

One argument moves the estimate by 0.47 percentage points, from 2.73% to 3.19%. That is bigger than the gap between the highest and lowest stages of the entire ladder excluding DiD. The default min_preperiods resolves to $\lceil T_0/2 \rceil = 43$, which uses only the second half of the pre-treatment period for cross-validation; passing set_f=range(6, 87) uses folds starting at quarter 6, which is what the paper does and what R’s masc reproduces to three decimals.

This is the second of the three defaults promised in the overview. Like zeta, it is not wrong — it is a defensible choice that happens not to be the one your reference used.

Tracing the cross-validation makes the trade-off visible. Refitting with a single-element m_grid forces each neighbour count in turn:

rows = []
for m in M_GRID:
r = MASC(cfg(96, m_grid=[m], set_f=SET_F)).fit()
st = r.weights.summary_stats
rows.append(dict(m=m, loss=pct(r.att), phi_hat=st["phi_hat"]))
print(pd.DataFrame(rows).to_string(index=False, float_format=lambda v: f"{v:.5f}"))

 m loss phi_hat
1 2.89028 0.03808
2 3.01677 0.00531
3 2.97685 0.06197
4 3.09494 0.08790
5 3.18100 0.09211
6 3.23477 0.08848
7 3.07342 0.04640
8 2.96191 0.11266
9 2.95586 0.12382
10 2.72551 0.15769

Two patterns. First, $\phi$ rises with $m$ — the more neighbours you average over, the more weight the cross-validation is willing to put on matching, because averaging more neighbours reduces the variance that makes matching unattractive. Second, the estimate is not monotone in $m$: it wanders between 2.73% and 3.23% with no obvious structure. That non-monotonicity is worth remembering when someone reports a single MASC number without saying what grid produced it.

13. Stage 5 — Augmented synthetic control

Every stage so far assumes the treated unit lies inside the convex hull of the donors, because the simplex cannot reach outside it. If the fit is imperfect — if there is pre-treatment imbalance that no non-negative blend can close — augmented SC (Ben-Michael, Feller and Rothstein [12]) fits a ridge regression to whatever imbalance is left and corrects for it. The correction is allowed to use negative weights.

ascm = {k: VanillaSC(cfg(e, augment="ridge", inference=False)).fit() for k, e in EVAL.items()}
a18 = ascm["2018Q4"]
aw = np.array(list(a18.donor_weights.values()))
print(f"ASCM 2018Q4 {pct(a18.att):.2f}% 2019Q4 {pct(ascm['2019Q4'].att):.2f}%")
print(f"weights: sum {aw.sum():.5f}, {int((aw < -1e-6).sum())} negative, "
f"min {aw.min():+.4f}, max {aw.max():+.4f}")
print(f"pre-RMSE {a18.pre_rmse:.6f} vs SC's {s18.pre_rmse:.6f}")

ASCM 2018Q4 3.04% 2019Q4 4.19%
weights: sum 1.00000, 8 negative, min -0.0090, max +0.2262
pre-RMSE 0.005428 vs SC's 0.005589

Augmented SC lands at 3.04%, indistinguishable from plain SC. The reason is visible in the weights: eight are negative, but the largest in magnitude is $-0.0090$, and the pre-treatment RMSE improves only from 0.005589 to 0.005428. There was very little imbalance left for the ridge correction to fix, because the UK sits comfortably inside the convex hull of twenty-three OECD economies. Augmentation earns its keep when the treated unit is extreme; here it has almost nothing to do.

The augment field is the whole interface — one keyword on the class you already used for plain SC. Two companions are worth knowing:

res_ascm = {k: VanillaSC(cfg(e, augment="ridge", residualize=True, inference=False)).fit()
for k, e in EVAL.items()}
print(f"residualize=True: 2018Q4 {pct(res_ascm['2018Q4'].att):.2f}% "
f"2019Q4 {pct(res_ascm['2019Q4'].att):.2f}%")

residualize=True: 2018Q4 3.04% 2019Q4 4.19%

residualize=True is the paper’s “ASCM res.” column — it residualises the outcome on covariates before augmenting. With no covariates supplied there is nothing to residualise on, so it returns the same numbers, which is the correct behaviour rather than a silent failure. ridge_lambda lets you fix the penalty by hand instead of letting the cross-validation choose it.

14. The whole ladder, side by side

Every section above appended one row to a running ledger; analysis.py writes it out as att_headline.csv.

ladder = pd.read_csv("att_headline.csv")
print(ladder[["method", "command", "loss_2018Q4", "loss_2019Q4",
"r_post_2018Q4", "published_2018Q4"]].to_string(index=False))

method command loss_2018Q4 loss_2019Q4 r_post_2018Q4 published_2018Q4
DiD FDID(...).fit().did 4.98 6.18 4.98 NaN
SC VanillaSC(...) 3.04 4.17 3.06 3.06
DSC TSSC(..., method="MSCa") 2.99 4.12 2.98 2.98
SDID SDID(..., zeta=0.0) 2.80 3.94 2.79 2.79
MASC MASC(..., set_f=range(6, 87)) 2.73 3.83 2.73 2.73
ASCM VanillaSC(..., augment="ridge") 3.04 4.19 3.04 3.04

Reading across the columns: r_post_2018Q4 is what the R edition of this post reports using synthdid, Synth, masc and augsynth; published_2018Q4 is de Brabander, Juodis and Miyazato Szini [1]. Three of the six stages — DiD, MASC and ASCM — agree with both to two decimals. DSC lands a hundredth above the published 2.98 only because 2.9887 sits on a rounding boundary; the R edition reports the same estimate as 2.99. The two real disagreements are SC and SDID, and both differ in the same direction and for the same reason.

Every stage puts the cost of the referendum above the 2.4% that Born, Müller, Schularick and Sedláček [2] published for this same dataset, and the excluding-DiD range is a fairly tight 2.73% to 3.04% at the end of 2018, widening to 3.83% to 4.19% a year later.

The weights tell a consistent story. Hungary, Canada, the United States, Japan and Norway carry the counterfactual under every method, and only ASCM ever goes negative — eight times, all of them tiny. Five estimators built on quite different principles are picking essentially the same five countries.

The dot plot makes the shape of the disagreement clear: DiD is off on its own, and the other seven estimates cluster within about a third of a percentage point of each other at 2018Q4. The width of that cluster, not any single point in it, is the honest answer.

14.1 Comparing counterfactuals with one call

Building that comparison by hand is instructive, but mlsynth ships utilities for it:

from mlsynth import compare_estimators, plot_counterfactual_comparison
comparison = compare_estimators(
{"SC": VanillaSC(cfg(96, inference=False)),
"SDID": SDID(cfg(96, zeta=0.0, vce="noinference")),
"MASC": MASC(cfg(96, m_grid=M_GRID, set_f=SET_F))},
show_bands=False,
)
ax = plot_counterfactual_comparison(comparison)

compare_estimators fits several estimators on one panel and lines their counterfactuals up on a common time axis; compare_counterfactuals does the same for already-fitted results; plot_counterfactual_comparison draws them with their prediction intervals. For an exploratory comparison this is one call instead of thirty lines, and it is the right starting point before you invest in custom figures.

15. The disagreement is the finding

Two cells in section 14’s table disagree with R: SC (3.04 against 3.06) and SDID (2.80 against 2.79). Neither is a bug in either library, and the explanation is the most useful thing in this post.

The synthetic-control objective on this panel has a condition number of roughly $7.5 \times 10^5$. In geometric terms that means a long, narrow, nearly flat valley of near-optimal weight vectors: many quite different $\omega$ give almost the same pre-treatment fit. Any optimiser has to decide when to stop walking down it.

R’s synthdid walks the valley with Frank-Wolfe on a capped iteration budget and stops at 3.06%.
mlsynth hands the identical problem to a convex solver which runs it to optimality and returns 3.039%.
Stata’s sdid inherits synthdid’s Frank-Wolfe and stops in the same place; tighten its convergence with max_iter(100000) min_dec(1e-9) and its SDID estimate drifts from 2.79% to 2.80%, which is where mlsynth already is.

All three legs ship with this post, so you can run the comparison yourself rather than take it on trust: cheatsheet_python.py, cheatsheet_R.R and cheatsheet_stata.do. Same data, same treatment date, same two evaluation quarters, same comparative table at the end — and each file hard-codes the others’ column, so a disagreement shows up the moment you run any one of them.

The Stata file is the one worth reading even if you never open Stata. Two things in it. Its ASCM row is a different estimator — allsynth implements the bias-corrected synthetic control of Abadie and L’Hour rather than the ridge-augmented version VanillaSC(augment="ridge") gives you — and its MASC row is empty, because MASC has no Stata implementation. Reporting those honestly rather than approximating them is the point.

Stata’s version of the penalty trap is also the nastiest of the three. Its documented default is zeta_omega(1e-6), which looks like a value but is a magic sentinel: sdid.ado reads if (EOmega==1e-6) EtaOmega = (yNtr*yTpost)^(1/4), so passing the documented default explicitly still requests the full penalty. Only a literal 0 switches it off.

So three implementations, written independently in three languages, sort themselves into exactly two camps — and the split is by solver, not by language or by author. The R edition of this post reached the same conclusion from a completely different direction, by running the Frank-Wolfe iteration ladder by hand and watching the estimate converge to 3.039 as the budget grew. Section 9.2’s figure is the same finding a third time: four independent code paths inside mlsynth, all convex, all landing on 3.039.

The practical lesson is not that one library is right. It is that a synthetic control estimate carries its solver’s fingerprint, and a second-decimal disagreement between implementations is the normal state of affairs rather than a cause for alarm. When you replicate a published synthetic-control number and land 0.02 away, the first hypothesis should be the optimiser, not the data.

16. Which stage should you choose?

The estimates cluster, but they do not coincide, and the ladder gives no reason to prefer the top stage. The source paper’s answer is an in-sample placebo tournament: advance the treatment date to a quarter when nothing happened, build the counterfactual on data up to that point only, and compare with what actually occurred. The true effect is zero, so every estimate is pure error.

flowchart LR
A["Pick a fake<br/>treatment date k<br/>(2010Q1 … 2014Q4)"] --> B["Fit every stage<br/>on quarters 1..k"]
B --> C["Predict quarter<br/>k + h"]
C --> D["Compare with<br/>what happened.<br/>True effect = 0"]
D --> E["Score:<br/>RMSE, MAB"]
E --> A
style A fill:#6a9bcc,stroke:#141413,color:#fff
style B fill:#141413,stroke:#6a9bcc,color:#fff
style C fill:#141413,stroke:#6a9bcc,color:#fff
style D fill:#d97757,stroke:#141413,color:#fff
style E fill:#00d4c8,stroke:#141413,color:#141413

The loop runs twenty times, once for each last-pre-treatment quarter from 2010Q1 to 2014Q4, and each pass refits all seven estimators from scratch. Everything the earlier sections taught about defaults now pays off: vce="noinference", method="MSCa", inference=False and an explicit m_grid are what keep this to thirteen seconds rather than several hours.

def placebo_one(k, h):
"""Every stage refit as if the treatment had happened at quarter k+1."""
e = k + h
row = {"k": k, "last_pre": QLAB[k - 1], "horizon": h}
row["SC"] = VanillaSC(cfg(e, pre=k, inference=False)).fit().effects.att
row["DSC"] = tssc_att(TSSC(cfg(e, pre=k, method="MSCa", inference=False)).fit())
w1, l1 = sdid_weights(k + 1, pre=k) # variant (i)
w2, l2 = sdid_weights(range(k + 1, k + 5), pre=k) # variant (ii)
w3, l3 = sdid_weights(k + 4, pre=k) # variant (iii)
row["SDID (i)"] = -sdid_loss(w1, l1, e, pre=k) / 100.0
row["SDID (ii)"] = -sdid_loss(w2, l2, e, pre=k) / 100.0
row["SDID (iii)"] = -sdid_loss(w3, l3, e, pre=k) / 100.0
row["MASC"] = MASC(cfg(e, pre=k, m_grid=M_GRID, set_f=list(range(6, k + 1)))).fit().effects.att
row["ASCM"] = VanillaSC(cfg(e, pre=k, augment="ridge", inference=False)).fit().effects.att
return row
placebo = pd.DataFrame([placebo_one(k, h) for h in (1, 4) for k in range(61, 81)])

 Horizon h = 1 quarter
method RMSE MAB MedAB
SC 0.0086 0.0068 0.0051
DSC 0.0086 0.0068 0.0051
SDID (i) 0.0066 0.0037 0.0016
SDID (ii) 0.0066 0.0038 0.0017
SDID (iii) 0.0066 0.0039 0.0020
MASC 0.0080 0.0062 0.0045
ASCM 0.0086 0.0068 0.0051

At a one-quarter horizon the whole SDID family scores 0.0066 root mean squared error, against 0.0080 for MASC and 0.0086 for SC, DSC and ASCM alike. The gap is even larger in mean absolute bias: 0.0037 for SDID (i) against 0.0068 for SC, close to a factor of two. The time weights are doing real work, and this is the paper’s central theoretical claim surviving an empirical test.

16.1 The published table is not comparing like with like

Now look at how the published version of that table is produced. In the replication code, SC, DSC, SDID (i), MASC and ASCM are all graded one quarter ahead; SDID (ii) and (iii) are graded four quarters ahead. Forecasting a year out is a strictly harder task, so part of the reported gap is the exam, not the student.

Running every estimator at both horizons settles it:

Method	RMSE, $h = 1$	RMSE, $h = 4$	Published
SC	0.0086	0.0145	0.0089
DSC	0.0086	0.0146	0.0087
SDID (i)	0.0066	0.0132	0.0067
SDID (ii)	0.0066	0.0133	0.0134
SDID (iii)	0.0066	0.0133	0.0134
MASC	0.0080	0.0140	0.0080
ASCM	0.0086	0.0146	0.0086

Every cell reproduces the published value to within 0.0003 — except the two that were graded on a different exam. Graded on the same task, the three SDID variants are indistinguishable: 0.0066 at one quarter, 0.0132–0.0133 at four. The published conclusion that variants (ii) and (iii) “perform the worst” is an artefact of the horizon, not a property of the estimators. This is the same finding the R edition reports, arrived at with a different library, which is about as much corroboration as a result of this kind can get.

What survives is the finding that matters more: at either horizon the whole SDID family beats every other stage, and the ordering below it is stable — SDID, then MASC, then SC, ASCM and DSC, and those last three are indistinguishable at one quarter and separated by less than 0.0001 at four.

17. Do covariates help? Three meanings of “control for”

So far everything has matched on outcomes alone. The obvious next question is whether adding the covariates — consumption, investment, export and import shares, labour productivity growth and the employment-population ratio — improves the counterfactual.

The answer mlsynth gives is more interesting than yes or no: it asks which of three different things you mean. SDIDConfig.covariates is not a list. It is a dictionary keyed by method, and passing a bare list raises.

Key	Method	What it does
`"adjust"`	Kranz (2022) two-step	residualise the outcome on covariates first, then run SDID on the residuals
`"match"`	de Brabander et al. [1], eqs. 11–12	put the covariates inside the unit-weight problem
`"optimized"`	Arkhangelsky et al. [10], fn. 4	estimate weights and covariate coefficients jointly

These are three different estimators, and they are all defensible readings of “SDID with covariates” in the literature.

COVARIATES = ["cons_share", "inv_share", "exp_share", "imp_share",
"labprod_growth", "emp_pop"]
for meth in ("adjust", "match", "optimized"):
kw = dict(zeta=0.0, vce="noinference", covariates={meth: COVARIATES})
if meth == "match":
kw["match_pre_periods"] = "last"
vals = [pct(SDID(cfg(e, **kw)).fit().effects.att) for e in EVAL.values()]
print(f"covariates={{'{meth}': ...}} 2018Q4 {vals[0]:5.2f} 2019Q4 {vals[1]:5.2f}")

covariates={'adjust': ...} 2018Q4 3.61 2019Q4 4.85
covariates={'match': ...} 2018Q4 1.85 2019Q4 2.93
covariates={'optimized': ...} 2018Q4 3.11 2019Q4 4.56

The three routes disagree by 1.76 percentage points at 2018Q4 — from 1.85% to 3.61%, against 2.80% with no covariates at all. That spread is more than five times the spread across the entire outcomes-only ladder. Adding covariates does not refine the answer here; it replaces one well-identified number with three poorly-identified ones.

match_pre_periods is the companion setting for the "match" route, taking "all", "half", "last" or an integer. It controls which pre-treatment periods the covariate means are computed over, and mlsynth’s own benchmarks report that its agreement with R’s Synth degrades from a weight-vector correlation of 0.998 under "last" to 0.636 under "all" — precisely because the predictor weights become less identified as more periods enter.

17.1 The same problem in VanillaSC, and a clean demonstration of why

VanillaSC has its own covariate route: the Abadie-Diamond-Hainmueller bilevel program, where an outer loop searches over predictor weights $V$ and an inner loop solves for donor weights $\omega$. Section 9.1 noted that those predictor weights are generically not identified. That claim sounds abstract until you test it, and the test costs one keyword.

Fit the identical model twice, with the same seed and the same data, changing only how long the differential-evolution search is allowed to run. If $V$ were well identified, the budget would not matter.

for label, budget in [("default (maxiter=300, popsize=15)", {}),
("reduced (maxiter=120, popsize=12)",
dict(mscmt_maxiter=120, mscmt_popsize=12))]:
r = VanillaSC(cfg(96, covariates=COVARIATES, backend="mscmt",
canonical_v="min.loss.w", seed=SEED,
inference=False, **budget)).fit()
ss = r.weights.summary_stats
print(f"{label:<36s} {pct(r.effects.att):5.2f}% "
f"pre-RMSE {r.fit_diagnostics.rmse_pre:.6f} "
f"v_agreement {ss['v_agreement']:.5f}")
top = sorted(ss["predictor_weights"].items(), key=lambda kv: -abs(kv[1]))[:4]
print(" " + ", ".join(f"{k} {v:.3f}" for k, v in top))

default (maxiter=300, popsize=15) 1.32% pre-RMSE 0.009662 v_agreement 0.05530
imp_share 1.000, exp_share 0.000, labprod_growth 0.000, emp_pop 0.000
reduced (maxiter=120, popsize=12) 1.11% pre-RMSE 0.009928 v_agreement 0.08281
labprod_growth 0.419, emp_pop 0.396, cons_share 0.141, inv_share 0.045

Same estimator, same seed, same data. Shortening the search moves the estimate from 1.32% to 1.11% — and, far more strikingly, moves the predictor weights from a corner solution that puts all the weight on the import share to a spread across labour productivity, employment, and the consumption and investment shares. Those are not slightly different answers to the same question. They are different economic stories about what makes a country comparable to the UK.

Three diagnostics all point the same way. The pre-treatment RMSE rises from 0.005589 to about 0.0097 — adding six covariates makes the pre-treatment fit nearly twice as bad, because the optimiser now spends its effort matching predictor means instead of the outcome path. v_agreement, the gap between the two canonical choices of $V$, is 0.055 to 0.083 rather than near zero. And the answer moves with the optimiser budget, which is the definition of a non-identified problem.

This is why the R edition’s placebo tournament found covariates make the counterfactual worse rather than better, and why the headline specification here matches on outcomes alone. When you have eighty-six pre-treatment quarters of the outcome itself, six covariate means are not adding information — they are adding a poorly identified optimisation problem. The fact that mlsynth reports v_agreement at all is what let us see it.

18. Inference

VanillaSC exposes nine inference methods behind a single inference= field, which is unusually generous. Here is what each returns on this panel:

rows = []
for meth in ("placebo", "scpi", "lto", "conformal", "ttest", "jackknife_plus"):
try:
r = VanillaSC(cfg(96, inference=meth, alpha=0.05)).fit()
inf = r.inference
rows.append(dict(method=meth, att=r.att, p_value=inf.p_value,
ci_lower=inf.ci_lower, ci_upper=inf.ci_upper,
reported_as=inf.method))
except Exception as exc:
rows.append(dict(method=meth, att=np.nan,
reported_as=f"FAILED: {type(exc).__name__}"))
print(pd.DataFrame(rows).to_string(index=False))

 method att p_value ci_lower ci_upper reported_as
placebo -0.030388 0.041667 NaN NaN in-space placebo (RMSPE ratio)
scpi -0.030388 NaN -0.055333 -0.009374 scpi prediction intervals (Cattaneo-Feng-Titiunik 2021)
lto -0.030388 0.008333 NaN NaN leave-two-out refined placebo (Lei-Sudijono 2025)
conformal -0.030388 0.020000 -0.046603 -0.014292 conformal prediction intervals (Chernozhukov-Wuthrich-Zhu 2021)
ttest -0.030388 0.005788 -0.040042 -0.020228 debiased SC t-test (Chernozhukov-Wuthrich-Zhu 2025)
jackknife_plus NaN NaN NaN NaN FAILED: MlsynthEstimationError

Reported honestly: jackknife_plus raises MlsynthEstimationError on this configuration and is excluded. The other five all reject at the 5% level: four report p-values between 0.006 and 0.042, and scpi reports no p-value but an interval that stops short of zero.

That looks decisive, and it should be read with more caution than it invites. The five methods are not five independent tests — they all use the same point estimate and the same donor pool, and they differ in how they build a reference distribution from twenty-three donors. Note also that SDID’s own placebo inference in section 11 gave p = 0.20 with an interval containing zero. Different estimator, different variance estimator, very different verdict. Read these as orders of magnitude rather than as digits.

18.1 Placebo in space

The most interpretable of the five is worth doing explicitly: give every donor the treatment in turn and see where the UK ranks.

rows = []
for country in [TREATED] + DONORS:
sub = panel.copy()
sub["tt"] = sub.groupby("country")["t"].rank(method="dense").astype(int)
sub["treat"] = ((sub.country == country) & (sub.tt > T0)).astype(int)
r = VanillaSC(dict(df=sub, outcome="log_rgdp", treat="treat", unitid="country",
time="tt", display_graphs=False, inference=False)).fit()
gap = np.asarray(r.gap, float).ravel()
pre, post = np.sqrt(np.mean(gap[:T0] ** 2)), np.sqrt(np.mean(gap[T0:] ** 2))
rows.append(dict(country=country, rmspe_pre=pre, rmspe_post=post, ratio=post / pre))
placebo_space = (pd.DataFrame(rows)
.assign(rank=lambda d: d["ratio"].rank(ascending=False).astype(int))
.sort_values("ratio", ascending=False))
print(placebo_space.head(6).round(4).to_string(index=False))

The statistic is the ratio of post-treatment to pre-treatment root mean squared prediction error,

$$R_j = \frac{\text{RMSPE}_j^{\text{post}}}{\text{RMSPE}_j^{\text{pre}}},$$

which asks whether unit $j$’s gap grew after the treatment date relative to how well it was fitted beforehand. Dividing by the pre-treatment fit is what stops a badly fitted donor from looking treated.

 country rmspe_pre rmspe_post ratio rank
United Kingdom 0.0056 0.0327 5.8488 1
Belgium 0.0043 0.0151 3.5231 2
Finland 0.0202 0.0666 3.2905 3
New Zealand 0.0134 0.0397 2.9693 4
Hungary 0.0210 0.0593 2.8193 5
Austria 0.0042 0.0111 2.6324 6

The UK’s ratio of 5.85 is the largest of all twenty-four units, giving a permutation p-value of $1/24 = 0.042$. That is as small as this design can produce: with twenty-four units the smallest attainable p-value is 0.042, so the test is at its floor and could not have been more favourable.

18.2 What this can and cannot tell you

It can tell you that the UK’s post-2016 divergence is unusual relative to what these twenty-three donors do. It cannot tell you the effect is 3.04% rather than 2.73%, it cannot separate Brexit from anything else distinctive that happened to the UK after mid-2016, and with twenty-four units it has essentially no power to detect anything subtler. Every interval in this section is wide, and the SDID interval contains zero.

19. Robustness: the specification zoo

Four departures from the headline specification, each one line of config. The SDID column reports variant (i) for the date and donor-pool rows and variant (ii) — the post’s headline — for the zeta rows, because those are the variants each check was run on.

Departure	SC	DSC	SDID
treatment date 2016Q3 (headline)	3.04	2.99	2.77 (i)
treatment date 2016Q2	3.09	3.05	3.18 (i)
drop the United States	3.06	3.04	2.83 (i)
`zeta = 0` (the paper)	—	—	2.80 (ii)
`zeta` at its default	—	—	2.67 (ii)
`w_constr='simplex'`	3.04	—	—
`w_constr='ols'`	3.37	—	—
`w_constr='ridge'`	3.37	—	—
`w_constr='lasso'`	2.97	—	—

Four observations.

The treatment date matters most for SDID. Moving from 2016Q3 to 2016Q2 barely touches SC (3.04 to 3.09) but moves SDID from 2.77% to 3.18%. That is exactly what section 11.2 predicts: SDID puts 96% of its time weight on the last pre-treatment quarter, so changing which quarter that is changes the bias adjustment directly. The estimator that is best on the placebo tournament is also the one most sensitive to the treatment date — a trade-off worth stating out loud.

Dropping the United States barely moves anything. The US carries about a fifth of the counterfactual, and removing it entirely shifts SC from 3.04% to 3.06% and SDID from 2.77% to 2.83%. The result does not hinge on one donor.

The constraint set is worth more than the estimator choice. Relaxing the simplex to OLS or ridge moves SC from 3.04% to 3.37%, a third of a percentage point — larger than the gap between SC and MASC. w_constr is a research decision, not a tuning knob.

And zeta again. 2.80% against 2.67%, for a setting most users will never see.

20. Beyond the ladder: the mlsynth catalogue

Six of ninety-two classes have appeared in this post. The rest are worth knowing about, because the reason to invest in this library rather than four single-purpose ones is that the config you have already written works for all of them. Here is the map, organised by what makes your design non-canonical:

If your design has…	Reach for	Note
The canonical one treated unit	`VanillaSC`, `TSSC`, `FDID`	the workhorses; everything in this post
Staggered adoption, several cohorts	`SDID`, `SequentialSDID`, `SSC`, `PPSCM`, `CSCM`	`SDID` handles both cases; `dataprep` detects which
Many donors relative to periods	`CLUSTERSC`, `MLSC`, `FSCM`, `SparseSC`, `MSQRT`, `PDA`, `SCUL`, `RESCM`	regularisation or donor selection
Bayesian uncertainty	`BVSS`, `BSCM`, `BFSC`, `MVBBSC`, `MTGP`, `BPSCS`	posterior rather than placebo intervals
A treated unit outside the convex hull	`ISCM`, `NSC`	relax the hull rather than augment the fit
Spillovers onto donors	`SpSyDiD`, `SPILLSYNTH`, `SPOTSYNTH`, `RRSC`	`SPOTSYNTH` also detects which donors are contaminated
Missing cells in the panel	`MCNNM`, `SNN`, `RMSI`	matrix completion
Multiple outcomes	`SCMO`	joint rather than one-at-a-time
Micro-level distributions	`DSC`, `MicroSynth`	the distributional DSC, not this post’s
Endogenous treatment	`SIV`, `ORTHSC`, `PROXIMAL`	instruments and proximal inference
No untreated donors at all	`SHC`	synthetic historical control
A design to run before treating	`MAREX`, `SYNDES`, `PANGEO`, `SPCD`, `MUSC`	these return a `DesignResult`, not an `EffectResult`
A continuous treatment	`CTSC`	dose rather than on/off
Factor structure in the outcome	`FMA`, `CFM`, `CSCIPCA`, `TASC`, `DSCAR`	interactive fixed effects

The library sorts its outputs into exactly two families, which is worth internalising because it determines what .fit() gives you back. EffectResult is an observational report — measure an effect on data you already have. DesignResult is a research design — choose which units to treat before any intervention, and it resolves into an EffectResult once outcomes exist. Everything in this post is the first kind.

For the current list on your installed version, get_llm_guide() is authoritative; the counts quoted in the README and the documentation prose disagree with each other and with __all__.

21. Discussion

What Brexit cost. Taking the ladder as a whole, the referendum had cost the UK between 2.7% and 3.0% of GDP by the end of 2018, and between 3.8% and 4.2% by the end of 2019. That is above the 2.4% previously published for this dataset, and the reason is not exotic: the earlier figure came from a specification that matched on covariates, and section 17 shows covariates make the counterfactual worse here rather than better.

Three caveats belong with the number. It is a net gap between the UK and a blend of OECD economies, so anything else distinctive that happened to the UK after mid-2016 is inside it. The no-interference assumption is strong over four years when the United States carries a fifth of the weight. And the estimate is a point on a specification cloud rather than a parameter that has been pinned down — every interval in section 18 is wide, and SDID’s contains zero.

What the software taught. This is where a package-first reading pays off, and the findings are not econometric.

Three defaults change the answer materially, and two of them by more than the spread across the ladder. zeta moves SDID by 0.13 percentage points, set_f moves MASC by 0.47, and the choice among the three covariate methods moves SDID by 1.76. The ladder’s own spread, excluding DiD, is 0.31, so set_f and the covariate method each clear it on their own while zeta stays just inside. You can pick the wrong default and be further from the truth than if you had picked the wrong estimator.

One estimator silently rounds the number you are most likely to quote. TSSC returns .att as exactly $-0.03$ while its gap series carries the full $-0.029887$. Nothing warns you.

Names are mnemonics, not definitions. mlsynth.DSC is not this post’s DSC, and importing it raises no error.

A version number is not a version. The PyPI release numbered 1.0.0 is behind git main at the same version number, and is missing a config field this post uses. mlsynth.__version__ will tell you "1.0.0" in both cases. Pin the commit, not the release.

A solver leaves a fingerprint. Four convex code paths inside mlsynth agree at 3.039% while R’s Frank-Wolfe stops at 3.06%. Three languages, two camps, split by optimiser rather than by author.

So what should you actually do? Fit the ladder, not a stage, and publish the cloud rather than a point. mlsynth makes that cheap — six estimators, one config, thirteen seconds for a twenty-date placebo tournament — which removes the main practical excuse for reporting a single specification. And when you report, say which defaults you set. On this dataset that sentence carries more information than the choice of estimator.

22. Summary and next steps

The Brexit referendum cost the UK 2.7–3.0% of GDP by end-2018 and 3.8–4.2% by end-2019, at every stage of the ladder, against 2.4% previously published for the same data.
mlsynth puts all six stages behind one interface: Estimator({"df": ..., "outcome": ..., "treat": ..., "unitid": ..., "time": ...}).fit(), returning a result with seven flat accessors that work everywhere.
Three defaults matter, and two of them more than the estimator choice. SDID penalises unit weights unless you set zeta=0.0 (2.80% vs 2.67%); MASC cross-validates on a different fold set unless you set set_f (2.73% vs 3.19%); and SDID’s three covariate methods disagree by 1.76 percentage points. Against a ladder that spans 0.31, the last two each clear it alone.
The time weights earn their keep. SDID’s placebo RMSE is 0.0066 against 0.0086 for plain SC, a 23% reduction, and the advantage holds at both forecast horizons.
But the published ranking among SDID variants does not survive a matched horizon. Graded on the same task, the three flavours score 0.0066, 0.0066 and 0.0066.
Covariates hurt here. They raise the pre-treatment RMSE from 0.0056 to 0.0099 and produce a v_agreement of 0.083, both signs of a poorly identified predictor-weight problem.
A limitation to carry forward: with one treated unit and 23 donors, the smallest attainable permutation p-value is 0.042. The design is at its inferential floor, and no estimator choice changes that.
Next: the same panel with CLUSTERSC or PDA if you have many more donors; SequentialSDID if adoption is staggered; SPOTSYNTH if you suspect spillovers onto the donor pool. All three take the config you already wrote.

All three cheat sheets ship with this post, so the cross-language comparison in section 15 is reproducible without leaving the bundle: cheatsheet_python.py (about half a minute), cheatsheet_stata.do (twenty seconds with standard errors off, three minutes with them on) and cheatsheet_R.R (thirty seconds with SE <- FALSE, four minutes otherwise). Each prints the same ladder and the same comparative table on the same data, and each hard-codes the other languages’ column so you can check one against another directly. The R edition of this post hand-codes every estimator before calling its package and is the place to go for the derivations. On the wider site, the classic synthetic control on the Basque Country, augmented synthetic control on the Kansas tax cuts and synthetic difference-in-differences on Proposition 99 in Stata cover single stages in isolation.

23. Exercises

Move the treatment date. Re-run the headline table with pre=T0-1 (treatment materialising 2016Q2). Which stage moves most, and can you explain it from the time weights?
Break the rounding. Compute DSC’s estimate from variants["MSCa"].att and from the gap series across all twenty placebo windows in section 16. How large does the discrepancy get?
Time the inference default. VanillaSC’s inference defaults to True, which runs in-space placebo. Time a fit with inference=False against the default, and decide when the difference matters.
Read the simplex. Print weights.summary_stats for all six stages. Which report n_negative > 0, and which report a constraint other than the plain simplex?
Grade the horizon. Extend section 16’s tournament to $h = 8$. Does the SDID family’s advantage survive a two-year forecast?
Separate MASC’s two dials. Fix m=10 and vary set_f; then fix set_f and vary m_grid. Which of the two drives the 0.47-point swing?
Try a fourth covariate route. Fit SDID with covariates={"match": [...]} under each of match_pre_periods in {"all", "half", "last", 20}. How wide is the resulting range, and how does it compare with the range across methods?
Pick a different estimator entirely. Fit CLUSTERSC and PDA on this panel with the config you already have. Do they land inside the ladder’s range, and what does that tell you about the range?

24. References

Acknowledgements

AI tools (Claude Code, Gemini, NotebookLM) were used to make the contents of this post more accessible to students. Nevertheless, the content in this post may still have errors. Caution is needed when applying the contents of this post to true research projects.

From DiD to SDID: A Ladder of Synthetic Control Estimators, and What Brexit Cost the UK

Fri, 31 Jul 2026 00:00:00 +0000

Abstract

The United Kingdom voted to leave the European Union on 23 June 2016, and because there is only one United Kingdom, that decision can only be costed against a country that never existed. This tutorial builds that country seven times over, hand-coding every stage of the single-treated-unit ladder before running it with its R package: difference-in-differences, synthetic control, demeaned synthetic control, synthetic difference-in-differences in three flavours, matching-and-synthetic-control and augmented synthetic control. The data are quarterly log real GDP for 24 OECD economies from 1995Q1 to 2020Q4: the UK treated, 23 donors, 86 pre-treatment quarters. Dating the treatment at 2016Q3 and matching on outcomes alone, the estimated shortfall in UK GDP at the end of 2018 is 3.06% under synthetic control, 2.99% under demeaned SC, 2.76% under SDID, 2.73% under MASC and 3.04% under augmented SC, widening to between 3.83% and 4.20% a year later — every one larger than the 2.4% previously reported for this dataset. A placebo tournament over twenty artificial treatment dates ranks the SDID family first, at 0.0067 log points of root mean squared error against 0.0089 for plain SC, and shows covariates make the counterfactual worse rather than better. Two things the published tables hide: the headline synthetic-control number is partly an artefact of where the optimiser stopped, and the ranking among the three SDID variants dissolves once they are graded on the same forecast horizon. Porting the ladder to Stata and Python sharpens the first: three independent implementations split by solver, not by language.

1. Overview

On 23 June 2016 the United Kingdom voted to leave the European Union. Three and a half years later, at the end of 2019, UK GDP was some number of percentage points below where it would otherwise have been. The trouble is the phrase “otherwise have been.” There is one United Kingdom, it took the treatment, and the version of it that stayed in the EU does not exist anywhere in the data.

The standard move is to build that missing country out of the countries we do observe. Take the other OECD economies, give each one a weight, add them up, and require that the resulting blend tracks the real UK quarter by quarter through the two decades before the referendum. If the blend and the UK were indistinguishable for eighty-six quarters, the argument goes, the blend is a credible stand-in for the UK afterwards. Whatever gap opens up after 2016 is the estimated effect.

That is the synthetic control method, and it is the second stage of a ladder. This tutorial climbs the whole ladder. We start at the bottom, with difference-in-differences, which is the same construction with the weights frozen at one twenty-third. Then we let the data choose the weights (synthetic control). Then we allow the blend to sit at a different level from the UK, provided it moves in parallel (demeaned synthetic control). Then we let the data also choose which pre-treatment quarters to hold the blend accountable for (synthetic difference-in-differences). Then two more recent estimators that attack the problem from different directions: a cross-validated blend of matching and synthetic control (MASC), and a ridge-augmented synthetic control that is allowed to use negative weights (ASCM).

Every stage opens with the same question: what does the previous stage get wrong? That question has a precise answer, and it is the intellectual centre of this post. Following de Brabander, Juodis and Miyazato Szini [1], we will decompose the error of any weighted counterfactual into two pieces: an extrapolation bias and an interpolation bias. Synthetic control’s unit weights attack only the first. Nearest-neighbour matching attacks only the second. SDID’s time weights are what let a single estimator attack both. Once you have that picture, the ladder stops being a list of acronyms and becomes a sequence of answers to one question.

We do everything twice. Each estimator is first written from scratch, in ten or twenty lines of R, so you can see exactly which optimisation problem is being solved and what is being held fixed. Only then do we call the package — synthdid, Synth, masc, augsynth — and check that the two agree. When they do not agree, we find out why, and in one case the answer turns out to be more interesting than the agreement would have been.

The empirical stakes are real. Born, Müller, Schularick and Sedláček [2] estimated, using this same dataset, that the referendum had cost the UK about 2.4% of GDP by the end of 2018. Every stage of the ladder we build puts the number higher.

Three companion posts on this site cover pieces of this ground in isolation: the classic synthetic control on the Basque Country, the augmented synthetic control on the Kansas tax cuts, and synthetic difference-in-differences on Proposition 99 in Stata. This post is the one that puts them on a single ladder and asks which stage to choose.

1.1 Learning objectives

By the end of this tutorial you will be able to:

Derive each estimator as the same weighted two-way fixed-effects regression with a different choice of unit weights $\omega$ and time weights $\lambda$.
Implement DiD, SC, DSC, SDID, MASC and ASCM from scratch in base R, and reproduce each one with synthdid, Synth, masc and augsynth.
Decompose the bias of any weighted counterfactual into extrapolation and interpolation components, and say which weights target which.
Estimate the effect of the Brexit referendum on UK GDP under two treatment dates, with and without covariates.
Select among estimators using an in-sample placebo tournament — and recognise when such a tournament is not comparing like with like.

1.2 The road ahead

The roadmap below is the shape of the post. Read it as a sequence of complaints: each estimator exists because of something the one before it could not do. The three diamonds are the whole story; everything else is bookkeeping.

flowchart TD
D["<b>Data</b><br/>24 OECD economies<br/>1995Q1-2020Q4<br/>log real GDP"] --> Q0{"How much should<br/>each donor country<br/>count?"}
Q0 -->|"all the same"| DID["<b>1. DiD</b><br/>omega = 1/J<br/>parallel trends"]
Q0 -->|"let the data decide"| SC["<b>2. SC</b><br/>omega on the simplex<br/>match level AND trend"]
SC --> Q1{"Must the blend sit at<br/>the same LEVEL<br/>as the UK?"}
Q1 -->|"no, absorb the gap"| DSC["<b>3. DSC</b><br/>demeaned omega<br/>+ constant adjustment"]
DSC --> Q2{"Should every<br/>pre-treatment quarter<br/>count the same?"}
Q2 -->|"no, weight them too"| SDID["<b>4. SDID</b><br/>omega AND lambda<br/>three variants"]
SDID --> BIAS["<b>The pivot</b><br/>extrapolation bias<br/>vs interpolation bias"]
BIAS --> MASC["<b>5. MASC</b><br/>trade the two off<br/>by cross-validation"]
BIAS --> ASCM["<b>6. ASCM</b><br/>de-bias imperfect fit<br/>with a ridge leash"]
MASC --> R["<b>Results</b><br/>2.7 to 3.1 per cent<br/>at 2018Q4"]
ASCM --> R
R --> SEL["<b>Which one?</b><br/>in-sample placebo<br/>over 20 fake dates"]
SEL --> INF["<b>Inference</b><br/>beyond the paper"]
style D fill:#6a9bcc,stroke:#141413,color:#fff
style Q0 fill:#f5f5f5,stroke:#141413,color:#141413
style Q1 fill:#f5f5f5,stroke:#141413,color:#141413
style Q2 fill:#f5f5f5,stroke:#141413,color:#141413
style DID fill:#d97757,stroke:#141413,color:#fff
style SC fill:#6a9bcc,stroke:#141413,color:#fff
style DSC fill:#6a9bcc,stroke:#141413,color:#fff
style SDID fill:#00d4c8,stroke:#141413,color:#141413
style BIAS fill:#141413,stroke:#00d4c8,color:#fff
style MASC fill:#d97757,stroke:#141413,color:#fff
style ASCM fill:#d97757,stroke:#141413,color:#fff
style R fill:#6a9bcc,stroke:#141413,color:#fff
style SEL fill:#00d4c8,stroke:#141413,color:#141413
style INF fill:#141413,stroke:#6a9bcc,color:#fff

Notice that the dark node in the middle is not an estimator. It is the section that explains why the last two branches exist at all, and it is the part of this material that transfers to problems that have nothing to do with Brexit.

2. Key concepts

Eight ideas carry the whole post. Two of them repay slow reading: the distinction between unit weights and time weights, and the distinction between extrapolation bias and interpolation bias. Neither of those bias names means what you would guess, which is exactly why they need a card.

The missing counterfactual and the donor pool. There is one United Kingdom and it took the treatment. The path it would have followed without Brexit is not in any dataset. Synthetic control builds that path from countries that were not treated. Those countries are the donor pool.

Example

After dropping the twelve OECD countries with incomplete records, 24 remain. The UK is the treated unit. The other 23 — from Australia to the United States — are the donor pool. None of them held a referendum on EU membership in 2016.

Analogy

The master tape of a song is lost and the original band has broken up. You hire session musicians and rehearse them against a bootleg recording until they are indistinguishable from the original. Then you have them play a song the original band never recorded. The donor pool is the pool of session musicians; the pre-treatment period is the rehearsal.

The simplex and the convex hull. Synthetic control weights must be non-negative and must sum to one. That set of allowed weight vectors is called the simplex. The blends it can produce form a region called the convex hull of the donors.

Example

With 23 donors, the simplex is the set of 23 non-negative numbers adding to one. In our fit only about eight donors get a weight above 0.01: Hungary near 0.22, the United States near 0.20, Japan near 0.18, Canada near 0.16, Norway near 0.13. The other fifteen get essentially nothing.

Analogy

Hammer a pin into a corkboard for every donor country, positioned by its economic characteristics. Stretch a rubber band around all the pins and let it snap tight. Everything inside the band is reachable by some blend; nothing outside it is. If the UK’s pin lands outside the band, no recipe of non-negative parts can reach it — you would need a negative amount of some country, like a recipe calling for minus two eggs.

Unit weights and time weights. Unit weights say how much each donor country counts. Time weights say how much each pre-treatment quarter counts. The two are chosen by the same kind of optimisation, run in two different directions.

Example

Our unit weights put about 0.22 on Hungary and 0.00 on France. Our time weights put 0.96 on 2016Q2 and roughly zero on the other 85 quarters. Both vectors are non-negative and both sum to one.

Analogy

A mixing desk has two banks of faders. The first bank sets how loud each instrument is; the second sets which seconds of the rehearsal tape you play back when you check the mix. Difference-in-differences leaves both banks flat. Synthetic control moves the first. Synthetic difference-in-differences moves both.

The bias-adjustment term, which is a unit fixed effect in disguise. Sometimes the blend moves in near-perfect parallel with the treated unit but sits at a slightly different level. The bias-adjustment term is the average pre-treatment gap, subtracted off. Adding it is exactly the same as putting a unit fixed effect in the regression.

Example

For the UK the demeaned synthetic control’s adjustment is $+0.0024$ log points, about a quarter of one per cent of GDP. That is why the DSC estimate of 2.99% at 2018Q4 sits so close to the SC estimate of 3.06%. A small adjustment is evidence that the SC fit was already level-balanced.

Analogy

Your bathroom scale reads two kilograms heavy. You do not throw it out; you subtract two. Synthetic control insists on a scale that is already exactly right and will reject a perfectly consistent one. Demeaned synthetic control just calibrates the offset.

Extrapolation bias and interpolation bias. Two different ways a weighted counterfactual can be wrong. Extrapolation bias: the blend’s characteristics do not match the treated unit’s. Interpolation bias: the characteristics match, but the outcome is a curved function of them, so averaging outcomes is not the same as the outcome at the average.

Example

Synthetic control chooses its weights to minimise pre-treatment prediction error, which is exactly minimising the first kind of error. It does nothing about the second unless log GDP happens to be a linear function of the underlying drivers. SDID’s time weights are what attack the second.

Analogy

You are roasting a 3.4 kilogram turkey and the chart lists only 3 kg and 4 kg. Extrapolation bias is misreading the scale and looking up 5 kg — right chart, wrong row. Interpolation bias is reading both rows correctly and averaging their times — right rows, but roasting time bends with weight, so the average of two times is not the time for the average bird. The two errors are independent, and fixing one does nothing for the other.

Rolling-origin cross-validation. A way to tune a parameter using only pre-treatment data. Walk forward through the pre-period; at each stopping point, fit on everything before it, forecast the next step, and score the forecast.

Example

MASC uses it to pick the number of matched neighbours $m$ and the blend weight $\phi$. With 86 pre-treatment quarters it produces eighty rolling origins, each giving a one-quarter-ahead forecast error for every candidate pair. Here it lands on $m = 10$ and $\phi = 0.158$.

Analogy

You do not grade a weather forecaster on how vividly they describe yesterday. You replay the archive, ask them to predict tomorrow from each day’s vantage point, and total up the misses.

The in-sample placebo. Pretend the treatment happened earlier, when nothing did. Estimate the counterfactual anyway and compare it to what actually occurred. The error you get is pure false alarm, and it measures the estimator’s precision.

Example

We repeat the whole exercise for twenty artificial treatment dates, with last pre-treatment quarters running from 2010Q1 to 2014Q4. SDID’s root mean squared error over those twenty is 0.0067 log points; plain synthetic control’s is 0.0089.

Analogy

A fire drill. There is no fire, so any alarm is a false one, and the detector that stays quietest is the one you install in the server room.

The ridge leash. Augmented synthetic control drops the non-negativity constraint so the weights can leave the convex hull. A quadratic penalty pulls them back toward the ordinary SC weights, and the penalty parameter sets how far they may roam.

Example

The ASCM weights in this application include Switzerland at $-0.0090$ and Slovak Republic at $-0.0085$ — impossible under the simplex — and the resulting estimate is 3.04% at 2018Q4, very close to the 3.06% from plain SC.

Analogy

An elastic leash on a dog. Slack leash: the dog goes wherever the scent leads, including off the lawn. Taut leash: it stays where you started. The stiffness of the leash is the ridge parameter.

3. Setup

Five packages do the estimation and four more do the plotting. Three of the five are not on CRAN, which is worth knowing before you start.

library(quadprog) # exact simplex least squares -- our hand-coded solver
library(synthdid) # SC, DSC and SDID all come out of one function (GitHub)
library(Synth) # only needed for the nested V-optimisation with covariates
library(masc) # matching and synthetic control (GitHub)
library(augsynth) # ridge-augmented synthetic control (GitHub)
library(ggplot2); library(tidyr); library(readr)
library(patchwork); library(scales); library(jsonlite)
# The three GitHub packages:
# remotes::install_github("synth-inference/synthdid")
# remotes::install_github("ebenmichael/augsynth")
#
# masc declares a hard dependency on Gurobi, a commercial solver it never
# actually needs on the code path we use. Drop the dependency first:
# git clone --depth 1 https://github.com/maxkllgg/masc /tmp/masc_src
# sed -i '' 's/^ gurobi,$//' /tmp/masc_src/DESCRIPTION
# R CMD INSTALL /tmp/masc_src
set.seed(20260801)

Everything in this post runs on R 4.5.2 with synthdid 0.0.9, augsynth 0.2.0, masc 0.1.1 and Synth 1.1-10. The full script is analysis.R, which also needs patchwork, scales and jsonlite for its figures and exports.

If you want the estimates without the derivations, there are three cheat sheets — one per language — each of which calls the packages directly and ends with the same comparative table: cheatsheet_R.R, cheatsheet_stata.do and cheatsheet_python.py. Section 19 compares what the three languages produce, and why two of them disagree in the second decimal.

4. The data

The dataset is the one assembled by Born and coauthors [2] from the OECD Economic Outlook: quarterly national accounts for 36 OECD economies. Twelve are dropped for incomplete records, leaving 24 countries observed over 104 quarters with no missing values at all.

url <- paste0("https://raw.githubusercontent.com/cmg777/starter-academic-v501/",
"master/content/post/r_sc_dsc_sdid/brexit_analysis.csv")
panel <- if (file.exists("brexit_analysis.csv")) read.csv("brexit_analysis.csv") else read.csv(url)
COUNTRIES <- unique(panel$country[order(panel$unit_id)])
UK <- which(COUNTRIES == "United Kingdom")
DONORS <- setdiff(seq_along(COUNTRIES), UK)
# Y is [time x country]: 104 quarters by 24 countries.
Y <- matrix(panel$log_rgdp[order(panel$unit_id, panel$t)],
nrow = 104, dimnames = list(NULL, COUNTRIES))

 panel : 24 countries x 104 quarters (1995Q1 to 2020Q4)
treated : United Kingdom (unit_id 23)
donors (23) : Australia, Austria, Belgium, Canada, Finland, France, Germany,
Hungary, Iceland, Ireland, Italy, Japan, Korea, Luxembourg,
Netherlands, New Zealand, Norway, Portugal, Slovak Republic,
Spain, Sweden, Switzerland, United States
headline spec: treatment materialises 2016Q3, T0 = 86 pre-periods
evaluated at : 2018Q4 (t=96) and 2019Q4 (t=100)

Interpretation. The donor pool has 23 countries and the pre-treatment window has 86 quarters — a long panel by synthetic-control standards, which matters because the bias of these estimators shrinks with the number of pre-treatment periods. The outcome is the natural log of real GDP indexed so that each country’s 1995 average equals one, so all effects are in log points and a difference of 0.03 is about a 3% shortfall.

4.1 Two definitions of $T_0$, pinned down once

This is the single most common place to go wrong, so we fix it before writing any code. In the papers, $T_0$ is the treatment period. In the synthdid package, the T0 argument is the number of pre-treatment periods. Here they are 87 and 86.

Concept	In the papers	In `synthdid`	Quarter	`t`
First pre-treatment quarter	$t = 1$	—	1995Q1	1
Last pre-treatment quarter	$t = T_0 - 1$	period `T0`	2016Q2	86
Number of pre-treatment quarters	$T_0 - 1 = 86$	`T0 = 86`	—	—
Treatment quarter	$t = T_0$	period `T0 + 1`	2016Q3	87
First evaluation quarter	—	—	2018Q4	96
Second evaluation quarter	—	—	2019Q4	100

The referendum was held on 23 June 2016, right at the end of 2016Q2. Following the source paper we date the treatment by the quarter in which the effect materialises, which makes 2016Q3 the headline choice and 2016Q2 a robustness check. Section 17 shows the choice is not innocuous.

5. What the data look like before we assume anything

ggplot(donors, aes(date, y, group = country)) +
geom_line(colour = GREY_DONOR, linewidth = 0.35, alpha = 0.75) +
geom_line(data = uk, aes(date, y), colour = ORANGE, linewidth = 1.1) +
geom_vline(xintercept = 2016.50, linetype = "dashed", colour = TEAL)

Interpretation. The UK is one line in a crowd. No single donor tracks it, which is the whole reason we will be building a blend. But look at 2008–09: every line collapses at once. That synchronised movement is a common factor, and the estimators from stage three onward are built precisely to exploit it.

Now the same picture with the crowd averaged into a single line. This is the difference-in-differences counterfactual, drawn before we name it.

Interpretation. The two lines were already drifting apart from about 2013, three years before the referendum. That is parallel trends failing in plain sight. Difference-in-differences will happily produce a number anyway — we compute it in the next section — and the number will be nearly twice what every other method reports. This is not a subtle failure.

The six covariates that Born and coauthors match on tell their own story:

Interpretation. The six predictors live on scales that differ by orders of magnitude — a consumption share near 0.65, a quarterly productivity growth rate near 0.1. That is why Abadie’s method needs a predictor-importance matrix at all, and it is the first hint of why adding covariates might cost more than it buys.

6. One regression, seven sets of weights

Before the ladder, the frame. Everything below is the same regression run with different weights.

Write the observed outcome in potential-outcome form. Let $y_{j,t}$ be log real GDP in country $j$ and quarter $t$, and let $w_{j,t}$ be one when country $j$ is treated in quarter $t$:

$$y_{j,t} = w_{j,t}\, y^{1}_{j,t} + (1 - w_{j,t})\, y^{0}_{j,t}$$

In words, this says that for every country-quarter we observe exactly one of two numbers — the treated outcome if the referendum applies, the untreated outcome otherwise — and the other is permanently missing. In code, $y_{j,t}$ is Y[t, j] and $w_{j,t}$ is one only for the UK from t = 87 onward.

What we want is the gap between what the UK did and what it would have done:

$$\hat{\tau}_t = y_{1,t} - \hat{y}^{0}_{1,t}$$

In words, the effect in any post-referendum quarter is the observed UK outcome minus the estimated counterfactual. In code, $y_{1,t}$ is Y[96, UK] at 2018Q4 and $\hat{y}^{0}_{1,t}$ is Y[96, DONORS] %*% omega.

Now the frame. Every estimator in this post is a weighted two-way fixed-effects regression:

$$\left(\hat{\tau}, \hat{\alpha}, \hat{\beta}\right) = \underset{\tau, \alpha, \beta}{\arg\min} \sum_{j=1}^{J+1} \sum_{t=1}^{T_0} \left( y_{j,t} - \alpha_j - \beta_t - w_{j,t}\, \tau \right)^{2} \hat{\omega}_j\, \hat{\lambda}_t$$

In words, this says: regress the outcome on a country effect, a quarter effect and a treatment dummy, weighting each country by $\hat{\omega}_j$ and each quarter by $\hat{\lambda}_t$. In code, $\hat{\omega}_j$ is omega[j], $\hat{\lambda}_t$ is lambda[t], and $\alpha_j$ is the unit fixed effect that omega.intercept = TRUE switches on.

The entire ladder is a table of settings for that one expression:

Stage	Unit weights $\omega$	Time weights $\lambda$	Unit effect $\alpha$	Feasible set for $\omega$
DiD	fixed at $1/J$	fixed at $1/(T_0-1)$	yes	—
SC	optimised	none	no	simplex
SC(B)	optimised	none	no	simplex
DSC	optimised on demeaned data	fixed at $1/(T_0-1)$	yes	simplex
SDID	optimised on demeaned data	optimised	yes	simplex
MASC	$\phi \cdot$ matching $+\, (1-\phi) \cdot$ SC	none	no	simplex
ASCM	SC weights + ridge correction	none	no	sums to one, sign free

graph TD
OBJ["<b>One weighted two-way regression</b><br/>minimise the sum of<br/>(y - alpha - beta - w*tau)^2 * omega * lambda"]
OBJ --> A["<b>omega uniform</b><br/><b>lambda uniform</b><br/>alpha included"]
OBJ --> B["<b>omega optimised</b><br/><b>no lambda</b><br/>alpha SUPPRESSED"]
OBJ --> C["<b>omega optimised</b><br/><b>lambda uniform</b><br/>alpha included"]
OBJ --> E["<b>omega optimised</b><br/><b>lambda optimised</b><br/>alpha included"]
A --> A1["DiD"]
B --> B1["SC and SC(B)"]
C --> C1["DSC"]
E --> E1["SDID"]
B1 --> F["<b>Change the feasible set<br/>instead of the weights</b>"]
F --> F1["MASC<br/>cap omega at 1/m,<br/>then blend"]
F --> F2["ASCM<br/>drop non-negativity,<br/>add a ridge pull"]
style OBJ fill:#141413,stroke:#00d4c8,color:#fff
style A fill:#d97757,stroke:#141413,color:#fff
style B fill:#6a9bcc,stroke:#141413,color:#fff
style C fill:#6a9bcc,stroke:#141413,color:#fff
style E fill:#00d4c8,stroke:#141413,color:#141413
style A1 fill:#f5f5f5,stroke:#141413,color:#141413
style B1 fill:#f5f5f5,stroke:#141413,color:#141413
style C1 fill:#f5f5f5,stroke:#141413,color:#141413
style E1 fill:#f5f5f5,stroke:#141413,color:#141413
style F fill:#141413,stroke:#d97757,color:#fff
style F1 fill:#d97757,stroke:#141413,color:#fff
style F2 fill:#d97757,stroke:#141413,color:#fff

Two things are worth pausing on. First, synthetic control is the only stage that switches the unit fixed effect off, and that single omission is what forces it to match the UK’s level as well as its shape. Second, MASC and ASCM hang off a different branch: they do not re-weight the regression, they change what counts as an admissible weight vector.

6.1 One solver, used five times

Every stage reduces to the same problem — minimise a sum of squares over the simplex — with different inputs. So we write the solver once:

$$\underset{w}{\min} \, \lVert b - A w \rVert^{2} \quad \text{subject to} \quad w_k \geq 0 \, \text{for all } k, \quad \sum_k w_k = 1$$

In words, find the non-negative shares of the columns of $A$ that come closest to reproducing the target vector $b$. In code, $A$ is Z0 (the donors’ pre-treatment paths) when we want unit weights, and its transpose when we want time weights.

simplex_ls <- function(A, b, ridge = 1e-10) {
k <- ncol(A)
w <- solve.QP(Dmat = crossprod(A) + ridge * diag(k), # the quadratic term
dvec = crossprod(A, b), # the linear term
Amat = cbind(rep(1, k), diag(k)), # col 1: sum(w)=1; rest: w>=0
bvec = c(1, rep(0, k)),
meq = 1)$solution # meq=1: first constraint is =
w[w < 1e-10] <- 0
w / sum(w)
}

The 1e-10 on the diagonal is numerical hygiene so the Cholesky factorisation never fails. It is not the regularisation parameter of Arkhangelsky and coauthors, which is a modelling choice we look at in section 17.

7. Stage 1 — Difference-in-differences

DiD is the ladder’s ground floor: every donor counts the same, and a unit fixed effect absorbs whatever constant level gap remains.

$$\hat{\tau}^{did}_{t} = \left( y_{1,t} - \bar{y}_{1} \right) - \frac{1}{J} \sum_{j=2}^{J+1} \left( y_{j,t} - \bar{y}_{j} \right)$$

In words, the change for the UK from its own pre-referendum average, minus the average change across all 23 donors from theirs. In code, $\bar{y}_j$ is colMeans(Z0) and $1/J$ is rep(1/23, 23).

w_did <- rep(1 / 23, 23)
b_did <- mean(Z1 - Z0 %*% w_did) # the unit fixed effect
loss <- function(w, e, b = 0) -100 * (Y[e, UK] - drop(Y[e, DONORS] %*% w) - b)
c(loss(w_did, 96, b_did), loss(w_did, 100, b_did))

 DiD 2018Q4 4.981 2019Q4 6.182 uniform weights
pre-treatment RMSE vs the donor average : 0.02175

Interpretation. DiD says Brexit cost the UK 4.98% of GDP by the end of 2018 and 6.18% by the end of 2019 — roughly double what every other stage will report. It is not in the source paper’s tables, and it should not be trusted: its pre-treatment fit error is 0.0218 log points, four times what synthetic control will achieve. The estimator is fitting a trend divergence that began years before the referendum and calling it Brexit.

DiD’s error is that it gave Luxembourg and the United States the same vote. The next stage lets the data vote.

8. Stage 2 — Synthetic control

8.1 The optimisation problem

Instead of fixing the weights, choose them to make the blend track the treated unit as closely as possible over the pre-treatment window:

$$\hat{\boldsymbol{\omega}}^{sc} = \underset{\boldsymbol{\omega} \in \mathbb{W}}{\arg\min} \sum_{t=1}^{T_0-1} \left( y_{1,t} - \sum_{j=2}^{J+1} \omega_j\, y_{j,t} \right)^{2}$$

In words, pick the blend of donors whose path over the 86 pre-referendum quarters is as close as possible, in squared error, to the UK’s own. In code, the inner sum is Z0 %*% omega and the objective is sum((Z1 - Z0 %*% omega)^2).

The feasible set is the simplex:

$$\mathbb{W} = \left\{ \boldsymbol{\omega} \in \mathbb{R}^{J} : \omega_j \geq 0 \, \text{for all } j, \, \sum_{j=2}^{J+1} \omega_j = 1 \right\}$$

In words, the weights are shares — never negative, always adding to one. In code, that is the Amat/bvec/meq block of simplex_ls.

Once the weights are fixed, the counterfactual in any quarter is just the weighted sum of what the donors actually did:

$$\hat{y}^{0,sc}_{1,t} = \sum_{j=2}^{J+1} \hat{\omega}^{sc}_{j}\, y_{j,t}$$

8.2 What the simplex actually is

Two constraints, non-negativity and summing to one, sound innocuous. Geometrically they are not. They confine the synthetic UK to the convex hull of the donors — the region you can reach by averaging.

Interpretation. The shaded region is everything a non-negative blend can reach. The UK sits comfortably inside it, which is why synthetic control works well here. Had the orange point fallen outside the band — a very rich or very poor treated unit — no recipe of non-negative shares could have reached it, and we would need stage six.

That picture is in outcome space. The optimisation happens in weight space, which is a different object. Restrict attention to the three donors that end up carrying the most weight and you can draw the entire search:

Interpretation. Every point in the triangle is one set of weights summing to one; the corners are “put everything on one donor”. The minimum sits in the interior, meaning all three donors earn a positive share. The real problem is this same picture in 22 dimensions, and a weight of exactly zero means the optimum sat on an edge.

8.3 From scratch, then the package

Z1 <- Y[1:86, UK] # the UK's pre-treatment path
Z0 <- Y[1:86, DONORS] # the donors' pre-treatment paths
w_sc_qp <- simplex_ls(Z0, Z1) # exact, via quadprog
# The package. Note the layout: units x time, treated unit LAST, and T0 is the
# NUMBER of pre-treatment periods.
Y_sd <- t(cbind(Y[, DONORS], Y[, UK]))
sd_sc <- synthdid_estimate(Y_sd[, 1:87], N0 = 23, T0 = 86,
zeta.omega = 0, zeta.lambda = 0,
omega.intercept = FALSE, lambda.intercept = FALSE)
w_sc_pkg <- as.numeric(attr(sd_sc, "weights")$omega)

Why call synthdid rather than Synth here? Because with no covariates, the classic synthetic-control problem of the equation above is synthdid_estimate with both intercepts off and both penalties set to zero. Using one optimiser for the whole ladder means every difference we report between stages is the method, not the solver. Synth reappears in section 16, where its nested optimisation is genuinely needed.

There is a third way to solve the same problem, and we need it in a moment. synthdid does not call a quadratic programming solver at all — internally it runs Frank–Wolfe, an iterative method that walks toward the optimum one simplex vertex at a time. Porting it is twenty lines, and it is the only way to see what the package is actually doing:

# A line-for-line port of synthdid's internal optimiser (sc.weight.fw).
fw_step <- function(A, x, b, eta) {
Ax <- A %*% x
half <- t(Ax - b) %*% A + eta * x
i <- which.min(half) # the steepest simplex vertex
dx <- -x; dx[i] <- 1 - x[i] # the direction to move in
if (all(dx == 0)) return(x)
derr <- A[, i] - Ax
s <- -drop(half %*% dx) / (sum(derr^2) + eta * sum(dx^2))
x + min(1, max(0, s)) * dx # the optimal step length, clipped
}
simplex_fw <- function(A, b, intercept = FALSE, min.decrease = 1e-5,
max.iter = 10000) {
if (intercept) { A <- sweep(A, 2, colMeans(A)); b <- b - mean(b) }
run <- function(x, mi) {
vals <- rep(NA_real_, mi); it <- 0
# Stop on a small enough improvement -- OR on the iteration cap.
while (it < mi && (it < 2 || vals[it - 1] - vals[it] > min.decrease^2)) {
it <- it + 1
x <- fw_step(A, x, b, 0)
vals[it] <- sum((A %*% x - b)^2) / nrow(A)
}
x
}
x <- run(rep(1 / ncol(A), ncol(A)), 100) # short pre-round, then sparsify
x[x <= max(x) / 4] <- 0; x <- x / sum(x)
run(x, max.iter)
}
w_sc_fw <- simplex_fw(Z0, Z1, min.decrease = 1e-5 * sd(apply(t(Z0), 1, diff)))

Watch the while condition. It exits on either a small enough improvement or the iteration cap — and which of those fires turns out to matter.

 exact QP : SSR 2.686818e-03 nonzero 9 loss(2018Q4) 3.039
hand-coded FW: SSR 2.751662e-03 nonzero 13 loss(2018Q4) 3.056
synthdid : SSR 2.751662e-03 nonzero 13 loss(2018Q4) 3.056
|FW - package| max abs weight difference : 0.000e+00 <-- these agree
|QP - package| max abs weight difference : 1.564e-02 <-- these do NOT

Interpretation. Something has gone wrong — or rather, something instructive has gone right. Our hand-coded Frank–Wolfe loop matches the package exactly, to the last bit. But the exact quadratic-programming solution disagrees, giving 3.04% where the package gives 3.06%, and achieving a lower sum of squared residuals while doing it. The package is not finding the optimum.

8.4 Why the two solvers disagree

The reason is the shape of the objective. With 23 donors and 86 pre-treatment quarters and no regularisation, the problem is nearly degenerate:

 noise.level (sd of donor quarterly changes) : 0.012425
synthdid min.decrease : 1.242e-07
donor Gram: smallest eigenvalue 3.192e-04, condition number 7.487e+05

A condition number near a million means the objective has directions along which it is almost perfectly flat. Frank–Wolfe crawls along those directions and never triggers its stopping rule, so it halts on its iteration cap instead. Let it run longer and the answer keeps moving:

Frank–Wolfe iterations	Sum of squares	Loss at 2018Q4	Loss at 2019Q4
100	0.0034364	3.177	4.289
300	0.0031180	3.038	4.183
1,000	0.0029646	3.056	4.203
3,000	0.0028366	3.061	4.210
10,000 (the package default)	0.0027517	3.056	4.204
30,000	0.0027141	3.047	4.187
100,000	0.0026963	3.042	4.178
exact QP	0.0026868	3.039	4.172

Interpretation. The published synthetic-control estimate of 3.06% is the value Frank–Wolfe happens to be passing through at ten thousand iterations. The true minimiser of the stated objective gives 3.04%. The difference is 0.02 percentage points — economically nothing, and it does not change a single conclusion in this post. But it is worth knowing that it is there, because it tells you something general: when a synthetic-control objective is this flat, the donor weights are not identified to more than a couple of decimal places, even though the estimate they produce is stable. Two solvers disagreeing by 0.016 on individual weights agree to within 0.02 percentage points on the effect.

For the rest of the post we report the package answer, so the tables line up with the published ones.

8.5 The counterfactual and the gap

 country omega_fw omega_qp
Hungary 0.2186 0.2231
United States 0.1994 0.1926
Japan 0.1773 0.1826
Canada 0.1612 0.1751
Norway 0.1256 0.1350
Ireland 0.0543 0.0523
Italy 0.0353 0.0196
Portugal 0.0123 0.0124
pre-treatment RMSPE : 0.00566 (DiD: 0.02175)

Interpretation. Synthetic Britain is roughly one-fifth Hungary, one-fifth the United States, one-fifth Japan, one-sixth Canada and one-eighth Norway. That combination has no economic interpretation and is not supposed to have one — it is whatever reproduces the UK’s growth path. The pre-treatment fit error of 0.0057 log points is a quarter of what DiD managed, which is the entire argument for the method.

Interpretation. Twenty-one years of near-perfect tracking, then a gap that opens right at the referendum and keeps widening: 3.06% by 2018Q4 and 4.20% by 2019Q4. The flatness of the gap before 2016 is what licenses reading the gap after 2016 as an effect.

9. Stage 3 — Demeaned synthetic control

9.1 What SC gets wrong

Synthetic control has no intercept. Look again at what that means. Suppose some blend of donors moved in perfect parallel with the UK for twenty-one years but sat consistently 0.5% below it. SC’s objective would score that blend badly and reject it, even though it is exactly what we want for forecasting a counterfactual — a series with the right dynamics and a known, constant offset.

Ferman and Pinto [9] and Doudchenko and Imbens [8] both proposed the same fix: demean first, then add the offset back.

$$\hat{\boldsymbol{\omega}}^{dsc} = \underset{\boldsymbol{\omega} \in \mathbb{W}}{\arg\min} \sum_{t=1}^{T_0-1} \left( (y_{1,t} - \bar{y}_{1}) - \sum_{j=2}^{J+1} \omega_j\, (y_{j,t} - \bar{y}_{j}) \right)^{2}$$

In words, the same problem as before, but every country’s own pre-treatment average is stripped out first, so the fit is judged on shape rather than on level. In code, Z0_dm <- sweep(Z0, 2, colMeans(Z0)) and Z1_dm <- Z1 - mean(Z1).

The offset comes back as a constant:

$$b^{dsc} = \frac{1}{T_0-1} \sum_{t=1}^{T_0-1} \left( y_{1,t} - \sum_{j=2}^{J+1} \hat{\omega}^{dsc}_{j}\, y_{j,t} \right)$$

In words, the average distance over the 86 pre-referendum quarters between the UK and its blend. In code, b_dsc <- mean(Z1 - Z0 %*% w_dsc).

$$\hat{\tau}^{dsc}_{t} = y_{1,t} - \sum_{j=2}^{J+1} \hat{\omega}^{dsc}_{j}\, y_{j,t} - b^{dsc}$$

In words, the raw gap minus the offset we already knew about from before the referendum.

Adding $b^{dsc}$ is algebraically identical to putting the unit fixed effect $\alpha_j$ back into the master regression of section 6. And that gives us a tidy result: DiD is DSC with the weights frozen at $1/J$. The ground floor and the third stage are the same estimator with different $\omega$.

9.2 Two changed lines

Z0_dm <- sweep(Z0, 2, colMeans(Z0)) # <-- the only change, part 1
Z1_dm <- Z1 - mean(Z1) # <-- the only change, part 2
w_dsc_qp <- simplex_ls(Z0_dm, Z1_dm) # identical call to stage 2
# The package: one argument flips.
sd_dsc <- synthdid_estimate(Y_sd[, 1:87], N0 = 23, T0 = 86,
zeta.omega = 0, zeta.lambda = 0,
omega.intercept = TRUE, lambda.intercept = TRUE)
w_dsc <- as.numeric(attr(sd_dsc, "weights")$omega)
b_dsc <- mean(Z1 - Z0 %*% w_dsc)

 bias adjustment b_dsc : +0.00242 log points (0.242% of GDP)
|QP - package| max weight diff : 7.523e-03
correlation of SC and DSC weights: 0.9928
DSC 2018Q4 2.985 2019Q4 4.121

Interpretation. DSC is SC’s curve slid up by a single number, and here that number is +0.0024 log points — about a quarter of one percent. The estimate moves from 3.06% to 2.99%. This is an anticlimax, and the anticlimax is the finding: a small bias adjustment means the synthetic control was already level-balanced, so SC was not sacrificing shape to chase level. On a dataset where the treated unit sits awkwardly relative to the donor pool, this term would be doing real work. Notice also that the two weight vectors correlate at 0.993 — demeaning barely changed who gets picked, only how the result is read off.

10. Stage 4 — Synthetic difference-in-differences

10.1 What DSC gets wrong

DSC’s bias adjustment is a flat average over all 86 pre-treatment quarters. It gives 1995Q1 exactly as much say as 2016Q2 in deciding how far apart the UK and its blend sit. But 1995 resembles 2016 hardly at all, and if the gap between the UK and its blend has been drifting, a flat average is the wrong correction.

Arkhangelsky and coauthors [10] let the data choose which quarters to trust.

10.2 The time-weight problem is the unit-weight problem, transposed

This is the key structural insight of the whole post, so it gets its own sentence: the time-weight problem is the unit-weight problem run on the transpose.

$\omega$ asks: which countries, blended, reproduce the UK’s pre-treatment path?
$\lambda$ asks: which quarters, blended, reproduce the treatment quarter, judged across all the donors?

$$\hat{\boldsymbol{\lambda}}^{sdid} = \underset{\boldsymbol{\lambda} \in \mathbb{L}}{\arg\min} \sum_{j=2}^{J+1} \left( (y_{j,T_0} - \bar{y}_{T_0}) - \sum_{t=1}^{T_0-1} \lambda_t\, (y_{j,t} - \bar{y}_{t}) \right)^{2}$$

In words, find the blend of pre-referendum quarters that best predicts the treatment quarter, scored across all 23 donors, after removing the cross-country average at each date. In code, this is simplex_ls(t(Z0_centred), y_treatment_quarter_centred) — the same function, with a transposed argument.

$$\mathbb{L} = \left\{ \boldsymbol{\lambda} \in \mathbb{R}^{T_0-1} : \lambda_t \geq 0 \, \text{for all } t, \, \sum_{t=1}^{T_0-1} \lambda_t = 1 \right\}$$

A trap worth naming. The word “demeaned” means two different things in stage three and stage four. DSC removes each country’s own time-series mean: sweep(Z0, 2, colMeans(Z0)). The SDID time-weight problem removes each quarter’s cross-sectional mean: sweep(Z0, 1, rowMeans(Z0)). Same verb, orthogonal operations. If you take one thing away from this section, take that.

The bias adjustment then becomes a weighted average instead of a flat one:

$$b^{sdid} = \sum_{t=1}^{T_0-1} \hat{\lambda}^{sdid}_{t} \left( y_{1,t} - \sum_{j=2}^{J+1} \hat{\omega}^{dsc}_{j}\, y_{j,t} \right)$$

Compare that with $b^{dsc}$ two sections above. They are the same expression with $1/(T_0-1)$ replaced by $\hat{\lambda}_t$. That is the entire difference between stages three and four. SDID does not even estimate its own unit weights — it reuses DSC’s.

# A has donors as ROWS and quarters as COLUMNS -- the transpose of the omega
# problem. `intercept = TRUE` demeans each quarter across donors, rather than
# each country across quarters as stage 3 did.
#
# We use the Frank-Wolfe port here rather than simplex_ls, so that the result is
# comparable with the package to the last bit. The exact QP gives the same
# answer to six decimals and an identical treatment effect.
lambda <- simplex_fw(t(Z0), Y[87, DONORS], intercept = TRUE,
min.decrease = 1e-5 * sd(apply(t(Z0), 1, diff)))
b_dsc <- mean(Z1 - Z0 %*% w_dsc) # stage 3: flat average
b_sdid <- drop(lambda %*% (Z1 - Z0 %*% w_dsc)) # stage 4: weighted average

 |hand-coded lambda - synthdid lambda| : 0.000e+00
lambda(i): 3 nonzero weights; 0.958 on the last pre-period (2016Q2)
2016Q2 0.9585
2008Q4 0.0386
2014Q3 0.0029
DSC bias adjustment +0.00242 vs SDID(i) bias adjustment +0.00015
SDID (i) 2018Q4 2.758 2019Q4 3.894

Interpretation. Our hand-coded time weights match synthdid’s to zero — not to within a tolerance, exactly. And the answer is startling: 96% of the weight lands on a single quarter, 2016Q2, with a small 3.9% on 2008Q4, the trough of the financial crisis. Because the weighted average of pre-treatment gaps is so different from the flat one ($+0.00015$ against $+0.00242$), the estimate drops from 2.99% to 2.76%.

10.3 Where DiD and two-way fixed effects live inside SDID

$$\lambda^{did}_t = \mathbf{1}\{ t = T_0 - 1 \}, \qquad \lambda^{twfe}_t = \frac{1}{T_0 - 1}$$

In words, if all the time weight lands on the last pre-treatment quarter you have a difference-in-differences correction; if it is spread evenly you have the two-way fixed-effects correction that DSC uses. SDID is supposed to let the data pick a point between those two extremes.

Here it picks one of the extremes. That deserves an explanation, not a shrug.

Interpretation. Log GDP is very close to a random walk — the scatter of quarter $t$ against quarter $t-1$ sits almost exactly on the 45-degree line. If the best predictor of next quarter is simply this quarter, then the best weighted blend of past quarters for predicting the treatment quarter is the most recent quarter alone. The collapse is a property of the data, not a bug in the code. The source paper notes this too, and declines to switch to differenced data on the grounds that matching levels and trends is the entire point of a synthetic control. Exercise 4 asks you to try it anyway.

10.4 Three flavours of SDID

The time weights have to be fitted against something in the post-treatment period, and there are three natural choices:

Variant	$\lambda$ is fitted to predict	Evaluated at
(i)	the first treated quarter, 2016Q3	any horizon
(ii)	the average of all quarters from 2016Q3 to the evaluation date	that date
(iii)	the evaluation quarter alone	that date

 SDID (i) 2018Q4 2.758 2019Q4 3.894
SDID (ii) 2018Q4 2.787 2019Q4 3.923
SDID (iii) 2018Q4 2.787 2019Q4 3.923

Interpretation. The three variants land within 0.03 percentage points of each other, because all three put essentially all their time weight on the same last pre-treatment quarter. Section 15 asks whether the placebo evidence can tell them apart — and finds that the published answer to that question does not survive scrutiny.

11. The pivot: extrapolation bias and interpolation bias

We now have four stages and no principled reason to prefer any of them. This section supplies one, and it is the theoretical contribution of the source paper.

11.1 A warning about the names

Neither term means what you would guess.

Extrapolation bias is not about predicting outside the range of the data in time. It is about the blend having the wrong characteristics.
Interpolation bias is not about filling in missing quarters. It is about the response function being curved.

Hold those corrected definitions in mind, because the formal decomposition is short and it will go past quickly.

11.2 The decomposition

Write each country’s untreated outcome as a function of its characteristics, and write a generic weighted counterfactual:

$$\hat{y}^{0}_{1,T_0} = \sum_{j=2}^{J+1} w_j\, y^{0}_{j,T_0}\left[ \boldsymbol{x}_{j,T_0} \right]$$

In words, every estimator in this post is a choice of the weights $w_j$ in this one expression; the square brackets are a device for tracking where the response function is being evaluated.

The total bias is what we want and what we get:

$$\text{Bias}_{1,T_0} = y^{0}_{1,T_0}\left[ \boldsymbol{x}_{1,T_0} \right] - \sum_{j=2}^{J+1} w_j\, y^{0}_{j,T_0}\left[ \boldsymbol{x}_{j,T_0} \right]$$

Insert a middle term — the response function evaluated at the blend’s characteristics — and it splits in two. First piece:

$$B^{ext} = y^{0}_{1,T_0}\left[ \boldsymbol{x}_{1,T_0} \right] - y^{0}_{1,T_0}\left[ \sum_{j=2}^{J+1} w_j\, \boldsymbol{x}_{j,T_0} \right]$$

In words, the same function evaluated at two different places: the UK’s true characteristics, and the blend’s characteristics. If the blend matches the UK exactly, this term is zero.

Second piece:

$$B^{int} = y^{0}_{1,T_0}\left[ \sum_{j=2}^{J+1} w_j\, \boldsymbol{x}_{j,T_0} \right] - \sum_{j=2}^{J+1} w_j\, y^{0}_{j,T_0}\left[ \boldsymbol{x}_{j,T_0} \right]$$

In words, the outcome at the averaged characteristics minus the average of the outcomes. These coincide only if the function is a straight line, so this term is pure curvature.

$$\text{Bias}_{1,T_0} = B^{ext} + B^{int}$$

The middle term cancels, so the two pieces add up exactly. That is what makes it legitimate to ask, of any estimator, which of the two it is attacking.

11.3 A two-donor picture

With two donors and one characteristic, there are two obvious weighting rules and each kills exactly one term. Linear-interpolation weights place the blend’s characteristic exactly on the treated unit’s:

$$w^{li}_{3} = \frac{x_{1} - x_{2}}{x_{3} - x_{2}}, \qquad w^{li}_{2} = 1 - w^{li}_{3}, \qquad w^{nn}_{2} = 1, \qquad w^{nn}_{3} = 0$$

In words, one recipe puts the blend at the right place on the horizontal axis and reads off the chord; the other copies the nearest donor outright. Neither can do both.

Interpretation. On the left, the blend’s characteristic is exactly right, so there is no extrapolation bias — but the chord between the two donor outcomes sits below the curve, and that vertical distance is the interpolation bias. On the right, we copy the nearest donor, so there is no interpolation bias — but we are reading the curve at the wrong place. Two errors, two weighting rules, one killed each time. The question the ladder has been building toward is whether anything can kill both.

11.4 Where each estimator sits

graph LR
T["<b>Total bias</b><br/>true UK outcome minus<br/>weighted donor outcome"] --> X["<b>Extrapolation bias</b><br/>same function,<br/>WRONG PLACE"]
T --> I["<b>Interpolation bias</b><br/>right place,<br/>CURVED FUNCTION"]
X --> XA["<b>omega weights</b><br/>minimise pre-treatment<br/>prediction error"]
I --> IA["<b>lambda weights</b><br/>find pre-periods that<br/>resemble the treatment period"]
XA --> SC2["SC: yes"]
XA --> NN2["Matching: no"]
IA --> SC3["SC: only if y is linear in x"]
IA --> NN3["Matching: yes, by construction"]
XA --> SD["<b>SDID: yes</b>"]
IA --> SD
style T fill:#141413,stroke:#00d4c8,color:#fff
style X fill:#6a9bcc,stroke:#141413,color:#fff
style I fill:#d97757,stroke:#141413,color:#fff
style XA fill:#6a9bcc,stroke:#141413,color:#fff
style IA fill:#d97757,stroke:#141413,color:#fff
style SC2 fill:#f5f5f5,stroke:#141413,color:#141413
style NN2 fill:#f5f5f5,stroke:#141413,color:#141413
style SC3 fill:#f5f5f5,stroke:#141413,color:#141413
style NN3 fill:#f5f5f5,stroke:#141413,color:#141413
style SD fill:#00d4c8,stroke:#141413,color:#141413

Only one node has two arrows pointing into it. Synthetic control minimises extrapolation bias by construction, because it is the argmin of the pre-treatment fit. Matching minimises interpolation bias by construction, because it never blends. SDID’s unit weights do the first job and its time weights do the second, which is the source paper’s headline claim and the reason it recommends the method.

11.5 The honest caveat

Do not over-learn this. The decomposition assumes a common response function across countries and sets the idiosyncratic error aside entirely. SDID “targets” both biases, but it pays for the privilege by estimating 85 extra parameters, and the source paper’s own conclusion is that the gain over DSC is “marginal at best” for the kind of trend specification we have here. Section 20 returns to this.

12. Stage 5 — MASC

12.1 Buying the trade-off explicitly

If SC kills one bias and matching kills the other, why not buy some of each? Kellogg, Mogstad, Pouliot and Torgovitsky [11] do exactly that:

$$\hat{\boldsymbol{\omega}}^{masc}(m, \phi) = \phi\, \hat{\boldsymbol{\omega}}^{ma}(m) + (1 - \phi)\, \hat{\boldsymbol{\omega}}^{sc}$$

In words, a dial between pure matching and pure synthetic control, with the dial position chosen by out-of-sample forecast error rather than by taste. In code, phi * nn_weights(Z0, Z1, m) + (1 - phi) * w_sc.

The matching weights themselves are the solution to a linear program:

$$\hat{\boldsymbol{\omega}}^{ma}(m) = \underset{\boldsymbol{\omega} \in \mathbb{S}}{\arg\min} \sum_{j=2}^{J+1} \omega_j \lVert \boldsymbol{y}_j - \boldsymbol{y}_1 \rVert, \qquad \mathbb{S} = \left\{ \boldsymbol{\omega} : 0 \leq \omega_j \leq \tfrac{1}{m}, \, \sum_j \omega_j = 1 \right\}$$

In words, capping every weight at $1/m$ and minimising total distance forces the solution to spread $1/m$ across exactly the $m$ closest donors. Matching, written as an optimisation problem.

12.2 The cross-validation, written out

nn_weights <- function(Z0, Z1, m) {
d <- colSums((Z1 - Z0)^2)
sel <- d %in% sort(d)[1:m]
as.numeric(sel) / sum(sel)
}
# Rolling origin: for each stopping point k, refit BOTH estimators on quarters
# 1..k, forecast quarter k+1, and score. phi then has a closed form.
set_f <- 6:85
wt <- rep(1 / length(set_f), length(set_f))
ysc <- vapply(set_f, function(k) drop(Z0[k+1, ] %*% simplex_ls(Z0[1:k, ], Z1[1:k])), 0)
ytr <- Z1[set_f + 1]
cv <- do.call(rbind, lapply(1:10, function(m) {
ymt <- vapply(set_f, function(k) drop(Z0[k+1, ] %*% nn_weights(Z0[1:k, ], Z1[1:k], m)), 0)
phi <- min(1, max(0, drop((wt*(ytr-ysc)) %*% (ymt-ysc) / ((wt*(ymt-ysc)) %*% (ymt-ysc)))))
data.frame(m = m, phi = phi, cv = sum(wt * (ytr - phi*ymt - (1-phi)*ysc)^2))
}))

 hand-coded : m = 10, phi = 0.1577
masc package: m = 10, phi = 0.1577 |hand - package| = 0.000e+00
MASC 2018Q4 2.726 2019Q4 3.828 m = 10, phi = 0.158

Interpretation. The data buy 15.8% matching and 84.2% synthetic control, using the ten nearest donors. The estimate, 2.73%, is the lowest on the ladder. Note that $\phi$ is not a taste parameter: it is the minimiser of a forecast error computed entirely from pre-treatment data, and at $\phi = 0$ MASC collapses to plain SC, so on this criterion it can never do worse.

A trap in the package. The published replication code passes masc’s min_preperiods argument, but current package master reads that value as the fold start, producing only five folds. Five folds select $\phi = 1$ — pure matching — and an estimate of 2.36%, which does not match the published result. The authors’ numbers correspond to folds running from 6 to $T_0 - 1$, so we set set_f explicitly. Silent, plausible-looking, and wrong by a third of a percentage point.

13. Stage 6 — Augmented synthetic control

13.1 What every stage so far assumes

All six previous stages require the UK to lie inside the donors’ convex hull — inside the rubber band of section 8.2. When it does not, the pre-treatment fit is imperfect and Abadie’s own advice is to stop. Ben-Michael, Feller and Rothstein [12] instead de-bias:

$$\hat{\boldsymbol{\omega}}^{ascm} = \underset{\boldsymbol{\omega} : \sum_j \omega_j = 1}{\arg\min} \, \frac{1}{2 \lambda^{ridge}} \sum_{t=1}^{T_0-1} \left( y_{1,t} - \sum_{j=2}^{J+1} \omega_j\, y_{j,t} \right)^{2} + \frac{1}{2} \sum_{j=2}^{J+1} \left( \omega_j - \hat{\omega}^{sc}_{j} \right)^{2}$$

In words, keep improving the pre-treatment fit, but pay a quadratic price for every step away from the ordinary synthetic-control weights. Non-negativity is gone; only the sum-to-one constraint survives. In code, $\lambda^{ridge}$ is chosen by leave-one-period-out cross-validation inside augsynth.

The estimator has a closed form, so we can write it out: take the SC weights and add a ridge-predicted correction for whatever pre-treatment imbalance they left behind.

ascm_hand <- function(lambda_ridge) {
Xc <- sweep(t(Z0), 2, colMeans(t(Z0))) # donors x quarters, period-centred
x1c <- Z1 - colMeans(t(Z0)) # the treated path, same centring
# SC weights, plus a ridge regression of the residual imbalance on the donors
as.numeric(w_sc_qp + solve(tcrossprod(Xc) + lambda_ridge * diag(23),
Xc %*% (x1c - crossprod(Xc, w_sc_qp))))
}

One thing we cannot supply by hand is $\lambda^{ridge}$ itself — it is chosen by cross-validation, and inventing a value would be cheating. Fit the package, read its choice back out, and feed it in:

ad <- data.frame(unitnum = rep(1:24, each = 87), t = rep(1:87, 24),
value = as.vector(Y[1:87, c(UK, DONORS)]))
ad$treatment <- as.integer(ad$unitnum == 1 & ad$t == 87)
ascm <- augsynth(value ~ treatment, unitnum, t, ad, progfunc = "Ridge", scm = TRUE)
w_ascm_hand <- ascm_hand(ascm$lambda) # ascm$lambda is augsynth's CV choice
max(abs(w_ascm_hand - as.numeric(ascm$weights)))

 package : sum(w) = 1.0000, min(w) = -0.0090, 8 negative weights
hand : ridge lambda = 0.13858 (chosen by augsynth's own CV)
loss(2018Q4) = 3.045 |hand - package| max weight diff = 3.881e-06
negative weights (impossible under the simplex):
country omega
Switzerland -0.0090
Slovak Republic -0.0085
Belgium -0.0066
Spain -0.0056
Sweden -0.0030
Korea -0.0027
Austria -0.0013
Netherlands -0.0012
ASCM 2018Q4 3.045 2019Q4 4.187

Interpretation. Eight donors get negative weight — the synthetic UK now subtracts a little Switzerland and a little Belgium, which is flatly impossible under the simplex. And the estimate, 3.04%, is almost exactly plain SC’s 3.06%. That is the expected result when the pre-treatment fit was already excellent: the ridge correction has almost nothing to fix, so it barely moves. ASCM earns its keep on datasets where SC visibly fails, which this is not. For a case where it does matter, see the Kansas tax-cut tutorial.

14. The whole ladder, side by side

Everything is now in place. First the units. Because the outcome is in logs, the counterfactual-minus-actual difference is in log points:

$$L_t = 100 \times \left( \hat{y}^{0}_{1,t} - y_{1,t} \right)$$

In words, multiply the log gap by 100 to get an approximate percentage shortfall, positive when the UK underperformed its counterfactual. In code, the script stores tau = y1 - yhat0, so the reported loss is -100 * tau. At these magnitudes the log approximation is good to two decimal places.

Method	2018Q4	2019Q4	Published	Note
DiD	4.98	6.18	—	not in the paper; pre-trends fail
SC	3.06	4.20	3.06 / 4.20	Frank–Wolfe
SC (exact QP)	3.04	4.17	—	the true optimum
DSC	2.99	4.12	2.98 / 4.12
SDID (i)	2.76	3.89	2.76 / 3.89
SDID (ii)	2.79	3.92	2.79 / 3.92
SDID (iii)	2.79	3.92	2.79 / 3.92
MASC	2.73	3.83	2.73 / 3.83	$m = 10$, $\phi = 0.158$
ASCM	3.04	4.19	3.04 / 4.19	8 negative weights
Born et al. (2019)	2.40	3.60		with covariates

Interpretation. Every cell of that table reproduces the published one to within 0.01 percentage points. Three things stand out. First, the spread across methods at 2018Q4 is 2.73% to 3.06% — a range of a third of a percentage point, which is small next to the gap to Born et al.’s 2.40%. Second, the estimated damage grows over time, from roughly 2.9% to roughly 4.1%, which looks like a change in the growth rate rather than a one-off level shift. Third, and this is the paper’s empirical punchline: every stage of the ladder puts the cost of the referendum above the previously published figure, and the methods that adjust for level and timing put it lowest, not highest.

Interpretation. Before the referendum the six lines are on top of each other — they are all fitting the same 86 quarters, and all fitting them well. The disagreement is entirely a post-treatment phenomenon, which is a useful reminder that pre-treatment fit cannot arbitrate between methods that all achieve it.

Interpretation. Four of the five recipes are nearly the same blend, dominated by Hungary, the United States, Japan, Canada and Norway. The SDID and DSC panels are identical by construction, since SDID reuses DSC’s unit weights and changes only the bias adjustment. Only ASCM crosses zero.

15. Which stage should you choose?

15.1 The in-sample placebo tournament

Pre-treatment fit cannot choose between these methods, because they all fit. So the source paper does something better: it moves the treatment date back to a quarter when nothing happened, builds the counterfactual using only data up to that point, and compares it to what actually occurred. The true effect is zero, so every estimate is pure error.

$$\text{RMSE} = \sqrt{ \frac{1}{20} \sum_{k=1}^{20} \left( y_{1, T’_k + h} - \hat{y}^{0}_{1, T’_k + h} \right)^{2} }$$

In words, across twenty artificial treatment dates, how far off is the counterfactual from an outcome we can actually check. In code, the loop runs k over the last pre-treatment quarters 2010Q1 to 2014Q4, always starting the window at 1995Q1, with h the forecast horizon.

flowchart LR
A["<b>Pick a fake<br/>treatment date</b><br/>2010Q1 ... 2014Q4<br/>20 of them"] --> B["<b>Fit on 1995Q1<br/>up to that date</b><br/>all seven estimators"]
B --> C["<b>Predict h quarters<br/>ahead</b>"]
C --> D["<b>Compare to what<br/>actually happened</b><br/>true effect is zero"]
D --> E["<b>Score</b><br/>RMSE, mean and median<br/>absolute error"]
E --> A
style A fill:#6a9bcc,stroke:#141413,color:#fff
style B fill:#6a9bcc,stroke:#141413,color:#fff
style C fill:#d97757,stroke:#141413,color:#fff
style D fill:#d97757,stroke:#141413,color:#fff
style E fill:#00d4c8,stroke:#141413,color:#141413

15.2 The published table, reproduced

Method	RMSE	MAB	MedAB	Published RMSE
SC	0.0089	0.0072	0.0055	0.0089
DSC	0.0087	0.0070	0.0052	0.0087
SDID (i)	0.0067	0.0037	0.0016	0.0067
MASC	0.0080	0.0062	0.0045	0.0080
ASCM	0.0086	0.0068	0.0051	0.0086
SDID (ii)	0.0134	0.0111	0.0103	0.0134
SDID (iii)	0.0134	0.0111	0.0107	0.0134

Interpretation. Every number reproduces exactly. Read as published, the table says SDID (i) wins comfortably — its median absolute error of 0.0016 is a third of plain synthetic control’s 0.0055 — and that variants (ii) and (iii) are the worst of the lot, twice as bad as plain SC. That is the basis for the paper’s recommendation to use variant (i) in practice.

15.3 The table is not comparing like with like

Look closely at how those numbers are produced. In the replication code, SC, DSC, SDID (i), MASC and ASCM are all graded one quarter ahead. SDID (ii) and (iii) are graded four quarters ahead. Forecasting a year out is a strictly harder task than forecasting a quarter out, so part of the gap is the exam, not the student.

Running every estimator at both horizons settles it:

Method	RMSE, $h = 1$	RMSE, $h = 4$
SC	0.0089	0.0150
DSC	0.0087	0.0149
SDID (i)	0.0067	0.0134
SDID (ii)	0.0066	0.0134
SDID (iii)	0.0066	0.0134
MASC	0.0080	0.0140
ASCM	0.0086	0.0146

Interpretation. Graded on the same task, the three SDID variants are indistinguishable — 0.0067, 0.0066, 0.0066 at one quarter, and 0.0134 for all three at four quarters. The published conclusion that variants (ii) and (iii) “perform the worst” is an artefact of the horizon, not a property of the estimators.

What survives is the finding that matters more: at either horizon, the whole SDID family beats every other stage, and the ordering below it is stable — SDID, then MASC, then ASCM, then DSC, then SC. The time weights are doing real work. Which variant supplies them is not settled by this evidence, and variant (i) remains the sensible default simply because it is the cheapest and requires no choice of post-treatment window.

16. Do covariates help?

The Brexit dataset ships with six covariates, and Born and coauthors matched on them. The source paper’s most pointed conclusion is that you should not.

Matching on covariates requires Abadie’s nested optimisation: an inner problem that picks weights given a predictor-importance matrix $\boldsymbol{V}$, and an outer problem that picks $\boldsymbol{V}$.

$$\hat{\boldsymbol{\omega}}(\boldsymbol{V}) = \underset{\boldsymbol{\omega} \in \mathbb{W}}{\arg\min} \, (\boldsymbol{z}_1 - \boldsymbol{Z}\boldsymbol{\omega})’ \boldsymbol{V} (\boldsymbol{z}_1 - \boldsymbol{Z}\boldsymbol{\omega})$$

In words, for a fixed opinion about how important each predictor is, pick the blend that best balances the predictors. In code, Synth::synth(X1 = ..., X0 = ...).

$$\hat{\boldsymbol{V}}^{sc} = \underset{\boldsymbol{V} \in \mathbb{V}}{\arg\min} \sum_{t=1}^{T_0-1} \left( y_{1,t} - \sum_{j=2}^{J+1} \hat{\omega}_j(\boldsymbol{V})\, y_{j,t} \right)^{2}$$

In words, choose the predictor importances that make the resulting blend track the UK’s outcome best. The variant used by Born and coauthors — call it SC(B) — replaces the outcome in this outer objective with the full stacked predictor vector, so covariates are scored in both loops:

# SC with covariates: outer loop scores the OUTCOME
synth(X1 = X1, X0 = X0, Z1 = as.matrix(Z1), Z0 = Z0)
# SC(B): outer loop scores the stacked PREDICTORS
synth(X1 = X1, X0 = X0, Z1 = X1, Z0 = X0)

X1 <- as.matrix(c(Z1, cov_means[UK, ])) # 86 outcomes + 6 covariate means
X0 <- rbind(Z0, t(cov_means[DONORS, ]))
s_born <- synth(X1 = X1, X0 = X0, Z1 = X1, Z0 = X0) # SC(B)
s_sc <- synth(X1 = X1, X0 = X0, Z1 = as.matrix(Z1), Z0 = Z0) # SC cov.
s_dsc <- synth(X1 = X1d, X0 = X0d, Z1 = as.matrix(Z1_dm), Z0 = Z0_dm) # DSC cov.

 method loss_2018Q4 loss_2019Q4 published
SC(B) 2.428 3.606 2.43
SC cov. 3.028 4.170 3.11
DSC cov. 2.942 4.050 2.90
SDID cov. (i) 2.731 3.839 2.75
SC no cov. 3.056 4.204 3.06
DSC no cov. 2.985 4.121 2.98
SDID no cov. (i) 2.758 3.894 2.76

Interpretation. The first row is the headline. SC(B) with mean covariates gives 2.43% at 2018Q4 and 3.61% at 2019Q4 — that is Born et al.’s 2.4% and 3.6%, reproduced to the second decimal. So the gap between the earlier published figure and everything else in this post is not a data difference or a coding difference. It is entirely the choice of estimator, and specifically the choice to score covariates in both optimisation loops.

Note also that the four covariate rows sit below their no-covariate counterparts in every case, and that our Synth-based numbers drift from the published ones by up to 0.08 percentage points. That drift is honest and expected: the outer optimisation over the predictor-importance matrix is not convex, Synth runs a derivative-free search over 92 dimensions, and different starting values land in different local optima. When a specification’s answer depends on where the optimiser started, that is information about the specification.

But the decisive evidence is not in this table at all — it is in the placebo tournament. Rerunning section 15 with covariates in the matching set makes almost every estimator worse:

Method	RMSE without covariates	RMSE with mean covariates
SC	0.0089	0.0092
DSC	0.0087	0.0106
SDID (i)	0.0067	0.0063
MASC	0.0080	0.0048
ASCM	0.0086	0.0083

(the covariate column is Table 8 of the source paper; our no-covariate column reproduces its Table 7 exactly)

Interpretation. Adding six covariates degrades SC and DSC and barely helps SDID and ASCM. Only MASC clearly benefits. The source paper’s conclusion is blunt, and our replication supports it: if the object of interest is the GDP series, do not add covariates. The reason is not mysterious. Kaul and coauthors [13] showed that once all pre-treatment outcomes are in the matching set, covariates are redundant — the outcomes already encode whatever the covariates would have told you. With 86 pre-treatment outcomes in play, the six extra predictors add estimation noise and nothing else.

17. Robustness: the specification zoo

Four departures from the headline specification, each a short table and a single lesson.

(a) The other treatment date. The referendum fell at the very end of 2016Q2, so dating the treatment there rather than at 2016Q3 is entirely defensible.

 method 2016Q2_2018Q4 2016Q3_2018Q4
SC 3.123 3.056
DSC 3.036 2.985
SDID (i) 3.170 2.758
MASC 2.769 2.726

Interpretation. SC, DSC and MASC barely notice — they move by less than 0.07 percentage points. SDID moves by 0.41, from 2.76% to 3.17%, which is more than the entire spread across methods at a fixed date. The reason is visible in the time-weight figure: SDID fits its time weights against the treatment quarter, so changing which quarter that is changes the target of the fit. At 2016Q2 the weight splits 0.84/0.16 across the last two quarters instead of collapsing onto one. The choice of treatment date is not innocuous, and it bites hardest on precisely the estimator the tournament recommends.

(b) Dropping the United States. The US carries about a fifth of the weight, and if spillovers exist they should be concentrated in the highest-weighted donors [17].

 method with_US without_US
SC 3.056 3.067
DSC 2.985 3.032
SDID (i) 2.758 2.818

Interpretation. Every estimate moves by at most 0.06 percentage points and all three move up. Whatever the no-interference assumption is doing here, it is not driving the result.

(c) The ridge penalty. Arkhangelsky and coauthors propose an automatic regularisation of the unit weights, whose main theoretical benefit is that it makes the solution unique — which, given section 8.4, is not a trivial gain.

$$\zeta = \left( T_{post} \right)^{1/4} \sqrt{ \frac{1}{J (T_0 - 2)} \sum_{j=2}^{J+1} \sum_{t=1}^{T_0-2} \left( \Delta_{j,t} - \bar{\Delta} \right)^{2} }$$

In words, the penalty scale is the standard deviation of donors’ quarter-to-quarter GDP changes, inflated by the fourth root of the number of post-treatment quarters. In code, this is what synthdid uses when you omit zeta.omega = 0.

 method no_penalty with_penalty
SC 3.056 3.093
DSC 2.985 3.090
SDID (i) 2.758 2.642

Interpretation. Penalising moves SC up by 0.04, DSC up by 0.11 and SDID down by 0.12. All within the spread we have already seen. The penalty is worth switching on for the uniqueness it buys, not because it changes any conclusion.

(d) Mean covariates. Section 16 asks whether covariates help as a matter of estimator design; the same question belongs here as a specification cell, because it is the one departure that moves every stage in the same direction.

 method no_covariates with_covariates
SC 3.056 3.028
DSC 2.985 2.942
SDID (i) 2.758 2.731

Interpretation. All three fall, and none by more than 0.05 percentage points. The covariate route that genuinely moves the answer is SC(B) at 2.43%, and that is a different estimator — Synth’s nested optimisation over 92 predictors — rather than a different specification of the ones on this ladder. Section 16 has the detail.

Interpretation. Across every specification in this section, the estimated loss sits between roughly 2.6% and 3.2%. Nothing plotted here reaches Born et al.’s 2.4% — the closest is SDID under the ridge penalty, at 2.64. Only SC(B) gets there, and it sits in section 16 rather than in this figure. That is the whole robustness story in one picture, and it is the reason the source paper insists on reporting a cloud rather than a point.

18. Inference: what the paper does not do

This section goes beyond the paper. De Brabander, Juodis and Miyazato Szini state in their Remark 1 that they consider point estimates only and set inference aside entirely, on the grounds that inference for this class of problems is genuinely hard. That is a defensible position for a methods comparison, and a poor place for a first-time learner to stop. Everything that follows is our addition.

18.1 Placebo in space

The classic device, due to Abadie and coauthors: pretend each donor in turn was the treated country, run the whole procedure, and see whether the UK’s post-treatment gap is unusual against that reference distribution. The statistic is the ratio of post-treatment to pre-treatment fit error, which corrects for the fact that a country the method fits badly will show a large gap for uninteresting reasons.

$$R_j = \frac{ \sqrt{ \frac{1}{T - T_0 + 1} \sum_{t \geq T_0} \hat{\tau}_{j,t}^{2} } }{ \sqrt{ \frac{1}{T_0 - 1} \sum_{t < T_0} \hat{\tau}_{j,t}^{2} } }, \qquad p = \frac{1}{J+1} \sum_{j=1}^{J+1} \mathbf{1}\{ R_j \geq R_1 \}$$

In words, form the post-over-pre error ratio for every country and ask what fraction look at least as extreme as the UK. In code, ratio_of() applied to each placebo run, then the rank of the UK.

placebo_space <- lapply(DONORS, function(j) {
pool <- setdiff(DONORS, j) # the UK is excluded throughout
w <- simplex_fw(Y[1:86, pool], Y[1:86, j])
Y[, j] - as.vector(Y[, pool] %*% w) # this donor's placebo gap path
})

 UK post/pre RMSPE ratio : 5.82
rank among 24 countries : 1
permutation p-value : 0.042 (finest attainable: 0.042)
synthdid placebo standard error (SDID): 0.00948 log points

Interpretation. The UK’s post-treatment fit error is 5.8 times its pre-treatment fit error, and that ratio is the largest of all 24 countries. The permutation p-value is therefore 0.042, the smallest value this design can produce. Visually, the orange line leaves the grey band shortly after the referendum and never returns.

The synthdid placebo standard error tells a more sobering story: 0.0095 log points, which puts a conventional 95% interval around the SDID estimate at roughly 0.9% to 4.6%. The point estimate is much better determined than the interval, which is the normal state of affairs with one treated unit and 23 donors. Anyone quoting “Brexit cost 2.8% of GDP” without that interval is overstating what this design can deliver.

18.2 What this can and cannot tell you

Three caveats, all of which matter.

First, with 23 donors the finest attainable p-value is $1/24 \approx 0.042$. The test simply cannot reject at the 1% level no matter how extreme the UK looks. This is a property of the design, not of Brexit.

Second, this is randomisation inference: it asks how unusual is the United Kingdom among OECD countries, not what is the sampling error of this estimate. Those are different questions, and only the first one has a well-defined answer here.

Third, for a genuinely model-based alternative, the source paper itself points to the conformal-inference approach of Chernozhukov, Wüthrich and Zhu [16], which augsynth implements and which the Kansas tutorial works through in detail.

19. The same ladder in Stata and Python

Everything above is R. The ladder is not, and a reader who works in Stata or Python should not have to take the results on trust. This section ports the whole thing twice — cheatsheet_stata.do and cheatsheet_python.py, alongside cheatsheet_R.R — and the disagreements between the three turn out to be the most instructive part.

All three sheets share one device. Each of these packages reports an ATT averaged over all post-treatment periods, but we want the shortfall at two specific quarters. So keep the 86 pre-treatment quarters plus the single quarter of interest, renumber time, and the average over “all post periods” becomes an average over one period. The bare package call then returns exactly the number we want, and none of the three files contains any post-estimation arithmetic.

19.1 What maps onto what

Stage	R	Stata	Python (`mlsynth`)
DiD	`did_estimate()`	`sdid …, method(did)`	`FDID(…).fit().did`
SC	`sc_estimate()`	`sdid …, method(sc)`	`VanillaSC(…)`
DSC	`synthdid_estimate(lambda = uniform)`	`sdid …, method(sc)` on demeaned $y$	`TSSC(…, method = "MSCa")`
SDID	`synthdid_estimate()`	`sdid …, zeta_omega(0) zeta_lambda(0)`	`SDID(…, zeta = 0)`
MASC	`masc()`	— none —	`MASC(…, set_f = range(6, 87))`
ASCM	`augsynth(progfunc = "Ridge")`	`allsynth …, bcorrect(merge)`	`VanillaSC(…, augment = "ridge")`

Two entries need explaining before the numbers make sense.

DSC in Stata. There is no dsc command and none is needed. Demeaned SC is SC run on outcomes from which each country’s own pre-treatment mean has been subtracted: after demeaning, the pre-treatment gap averages to zero by construction, so the double difference collapses to the single one. Three lines of bysort and a method(sc) call reproduce it exactly.

ASCM in Stata is a different estimator. allsynth implements the bias-corrected synthetic control of Abadie and L’Hour and of Ben-Michael, Feller and Rothstein: fit SC, then regress the outcome on the predictors across the donor pool and subtract the predicted discrepancy. augsynth uses ridge-augmented SC. They are cousins, not the same estimator. Worse, the bias correction is an OLS fit across donors, so it needs more control units than predictors — with 23 donors we cannot hand it all 86 pre-treatment quarters the way a ridge penalty can. The path has to be summarised, and the summary matters enormously: with sparse individual lags the bias-corrected estimate swings between $-0.8$ and $5.1$ depending on which quarters you pick. Block means are far better conditioned, and the do-file fits a small grid of them and keeps the one with the lowest pre-treatment RMSPE — a rule fixed in advance that never looks at the post-treatment answer.

19.2 The three ports, side by side

Shortfall in UK real GDP (%), treatment dated 2016Q3, outcomes only.

Stage	R 2018Q4	Stata 2018Q4	Python 2018Q4	R 2019Q4	Stata 2019Q4	Python 2019Q4
DiD	4.98	4.98	4.98	6.18	6.18	6.18
SC	3.06	3.06	3.04	4.20	4.20	4.17
DSC	2.99	2.99	2.99	4.12	4.12	4.12
SDID (ii)	2.79	2.79	2.80	3.92	3.92	3.94
MASC	2.73	—	2.73	3.83	—	3.83
ASCM	3.04	3.10	3.04	4.19	4.22	4.19

The SDID row reports variant (ii) in all three languages, so that the comparison is like with like; section 14’s headline SDID is variant (i), at 2.76. Stata reproduces R to five decimal places at every stage it can fit. Python agrees on DiD, DSC, MASC and ASCM, and differs in the second decimal on SC and SDID.

19.3 The disagreement is the finding

The bolded cells are not a bug in any of the three libraries. They are section 8.4’s solver story arriving from a completely independent direction.

Recall the problem: the SC objective on this panel has a condition number around $7.5 \times 10^{5}$, which means a wide, nearly flat valley of near-optimal weight vectors. synthdid walks that valley with Frank–Wolfe on a capped iteration budget and stops at 3.06. mlsynth hands the identical problem to a convex solver, which runs it to optimality and returns 3.04 — which is precisely the “SC (exact QP)” row computed by hand in section 8.4, on both quarters. Stata’s sdid inherits synthdid’s Frank–Wolfe and stops in the same place; tighten its convergence with max_iter(100000) min_dec(1e-9) and the SDID estimate drifts from 2.79 to 2.80, which is where Python already is.

So three implementations, written independently in three languages, sort themselves into exactly two camps — and the split is by solver, not by language or by author. That is a much stronger piece of evidence for the section 8.4 claim than the iteration ladder in the original analysis, because nobody was trying to make this point when they wrote mlsynth.

The practical lesson is not that one library is right. It is that a synthetic control estimate carries its solver’s fingerprint, and that a second-decimal disagreement between implementations is the normal state of affairs rather than a cause for alarm.

19.4 Three traps the ports exposed

Each language has a default that quietly gives you the wrong estimator, and in all three cases it is the same default.

Every package penalises by default; the paper does not. R needs zeta.omega = 0, zeta.lambda = 0, Python needs zeta = 0, and Stata needs zeta_omega(0) zeta_lambda(0). Leave any of them alone and SDID reports 2.66–2.67 instead of 2.79. Stata’s version of this trap is the nastiest: the documented default is zeta_omega(1e-6), which looks like a value but is a magic sentinel — sdid.ado reads if (EOmega==1e-6) EtaOmega = (yNtr*yTpost)^(1/4), so passing the documented default explicitly still requests the full penalty. Only 0 switches it off.

MASC’s fold set has to be given explicitly in both languages that have MASC. R’s masc and Python’s mlsynth.MASC both cross-validate over a fold set that, left to its default, is not the one the paper uses. Pass set_f = 6:T0 in R and set_f=range(6, 87) in Python and the two agree to three decimals at 2.726. This is the same trap flagged in section 12, and it survives translation.

mlsynth.DSC is not this post’s DSC. mlsynth ships a class named DSC which implements Distributional Synthetic Control (Gunsilius) — matching whole outcome distributions. The DSC on this ladder is Demeaned Synthetic Control, which in mlsynth is TSSC(method = "MSCa"). Same three letters, different estimators, and importing the wrong one raises no error at all. It simply answers a different question. The mapping used here follows mlsynth issue #312, which is itself a reading of the paper this post replicates.

The Python column of these tables is only a cheat sheet. The Python edition of this post climbs the same ladder at full length with mlsynth alone — every config option, every result field, the three SDID flavours, the covariate routes this cheat sheet skips, and the wider catalogue of estimators the library ships. Read it if you work in Python; read this one for the derivations.

19.5 What each language cannot do

Reported plainly rather than papered over. MASC has no Stata implementation, so that row is empty rather than approximated. SC(B) and the other covariate specifications are in none of the three sheets, because they need Synth’s nested optimisation over 92 predictors and turn a thirty-second script into a coffee break — section 16 and analysis.R §14d cover them. And the standard errors each package reports are its own recommended method, not a common yardstick: R’s placebo SEs, Stata’s placebo SEs at a different replication count, augsynth’s jackknife and mlsynth’s analytic FDID error are not comparable digit for digit. Read them as orders of magnitude, and note that every one of them is wide enough to contain zero.

20. Discussion

What Brexit cost. Taking the ladder as a whole, the referendum had cost the UK somewhere between 2.7% and 3.1% of GDP by the end of 2018, and between 3.8% and 4.2% by the end of 2019. That is above the 2.4% previously published for this dataset, and the reason is not exotic: the earlier figure came from a specification that matched on covariates, and covariates make the counterfactual worse here rather than better.

Three caveats belong with that number. It is a net gap between the UK and a blend of OECD economies, not a Brexit-only effect — anything else distinctive that happened to the UK after mid-2016 is inside it. The no-interference assumption is strong over a four-year horizon when the United States carries a fifth of the weight in the counterfactual. And the estimate is a point on a specification cloud, not a parameter that has been pinned down.

What the ladder taught. The durable idea is the bias decomposition, not the leaderboard. Extrapolation bias and interpolation bias are separate failures with separate fixes, unit weights address the first, time weights address the second, and any weighted counterfactual you ever build can be interrogated on both counts. The ranking of estimators on this dataset is far more perishable — it depends on the outcome behaving like a random walk, on a long pre-period and on a treated unit that sits inside the convex hull.

It also cuts the other way, and the source paper says so plainly: SDID’s advantage over DSC is marginal once you count the 85 extra parameters it estimates, and neither MASC nor ASCM justifies its computational cost here. Our own placebo results agree — the whole SDID family clusters together, and the gap down to DSC is smaller than the gap between covariates and no covariates.

So what should you actually do? Fit the ladder, not a stage. Run an in-sample placebo tournament and check that the horizons are matched. Report the range. If one specification is going to be the headline, choose it before you see the estimates, and show the others anyway. Ferman, Pinto and Possebom [15] have documented how much room for cherry-picking this literature leaves; the honest response is to publish the cloud.

21. Summary and next steps

Every estimator here is one weighted two-way regression with a different choice of unit weights $\omega$, time weights $\lambda$, and whether the unit fixed effect is switched on. DiD, SC, DSC and SDID are four settings of the same expression; MASC and ASCM change the feasible set instead.
One solver does five jobs. The simplex least-squares problem solves the unit weights, the demeaned unit weights, the time weights (on the transpose) and both components of MASC’s cross-validation.
The Brexit cost is 2.7–3.1% at end-2018 and 3.8–4.2% at end-2019, above the previously published 2.4%, and the difference is driven mostly by the covariate specification.
The SDID family wins the placebo tournament at either forecast horizon, but the published ranking within that family does not survive matching the horizons.
Covariates hurt here. With 86 pre-treatment outcomes already in the matching set, six extra badly-scaled predictors add estimation noise without adding identification.
Two practical traps cost real accuracy: masc’s fold argument silently produces five folds instead of eighty, and a flat objective means synthdid’s optimiser stops on its iteration cap rather than at the optimum.
Both traps survive translation. Porting the ladder to Stata and Python (section 19) reproduces every estimate, and the places where it does not are the solver, not the language: mlsynth’s convex solver lands on the exact-QP answer while synthdid and Stata’s sdid stop where Frank–Wolfe stops. Three implementations, two camps, split by solver.

Where to go next: the three cheat sheets if you just want working code — cheatsheet_R.R, cheatsheet_stata.do, cheatsheet_python.py — then multi-country and staggered adoption with multisynth, the same SDID estimator in Stata on Proposition 99, or manual demeaning and the FWL theorem if the unit-fixed-effect story in stage three felt too quick. The Monte Carlo study in the source paper, which stress-tests this ranking on simulated data, is the subject of a future post.

22. Exercises

Move the treatment date. Re-run SC and SDID (i) dating the treatment at 2016Q1 rather than 2016Q2 or 2016Q3. Which estimator moves more, and can you explain why using the time-weight figure?
Read the simplex. Using simplex_ls, solve the SC problem restricted to just the United States, Hungary and Canada, and plot the objective over the triangle. Now add Japan as a fourth donor. By how much does the pre-treatment MSPE fall?
Break DSC on purpose. Add a constant of 0.05 log points to every UK observation, before and after the referendum. Which of SC, DSC and SDID change their estimated effect, and which do not? Explain the result using the unit fixed effect $\alpha_j$ in the master regression.
Force the time weights to spread out. Re-run SDID on first-differenced log GDP instead of levels. Does the spike on the final quarter survive? Then argue, using the source paper’s own reasoning, why the authors declined to make this switch in their headline results.
Finish the horizon audit. Section 15.3 matched the horizons for the placebo tournament. Extend it to $h = 2$ and $h = 8$. Is the SDID family’s advantage over SC stable in the horizon, or does it shrink?
Separate MASC’s two dials. Compute the 2018Q4 estimate for $\phi$ on a grid from 0 to 1 in steps of 0.05, holding $m = 10$. How much of the difference between MASC’s 2.73% and SC’s 3.06% is due to the chosen $\phi$ rather than to the choice of $m$?
Drop the biggest donor. The United States carries about a fifth of the weight in most specifications, and spillover risk is concentrated in the highest-weighted donors. Re-run the entire ladder without the United States. Does your conclusion about Brexit change? Then ask the harder question: does your conclusion about which estimator to use change?
Build your own stage. DSC and SDID differ only in how the bias adjustment is weighted across pre-periods — flat in one, optimised in the other. Propose a third weighting, for example exponentially decaying weights with a half-life you choose, implement it, and enter it in the placebo tournament. Does it beat SDID?

23. References

de Brabander, E., Juodis, A., & Miyazato Szini, G. (2025). On the use of synthetic difference-in-differences approach with (-out) covariates: The case study of Brexit referendum. Econometric Reviews, 44(10), 1617–1646.
Born, B., Müller, G. J., Schularick, M., & Sedláček, P. (2019). The costs of economic nationalism: Evidence from the Brexit experiment. The Economic Journal, 129(623), 2722–2744.
Abadie, A., & Gardeazabal, J. (2003). The economic costs of conflict: A case study of the Basque Country. American Economic Review, 93(1), 113–132.
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies. Journal of the American Statistical Association, 105(490), 493–505.
Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510.
Abadie, A. (2021). Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature, 59(2), 391–425.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
Doudchenko, N., & Imbens, G. W. (2016). Balancing, regression, difference-in-differences and synthetic control methods: A synthesis. NBER Working Paper 22791.
Ferman, B., & Pinto, C. (2021). Synthetic controls with imperfect pretreatment fit. Quantitative Economics, 12(4), 1197–1221.
Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2021). Synthetic difference-in-differences. American Economic Review, 111(12), 4088–4118.
Kellogg, M., Mogstad, M., Pouliot, G. A., & Torgovitsky, A. (2021). Combining matching and synthetic control to trade off biases from extrapolation and interpolation. Journal of the American Statistical Association, 116(536), 1804–1816.
Ben-Michael, E., Feller, A., & Rothstein, J. (2021). The augmented synthetic control method. Journal of the American Statistical Association, 116(536), 1789–1803.
Kaul, A., Klößner, S., Pfeifer, G., & Schieler, M. (2022). Standard synthetic control methods: The case of using all preintervention outcomes together with covariates. Journal of Business & Economic Statistics, 40(3), 1362–1376.
Botosaru, I., & Ferman, B. (2019). On the role of covariates in the synthetic control method. The Econometrics Journal, 22(2), 117–130.
Ferman, B., Pinto, C., & Possebom, V. (2020). Cherry picking with synthetic controls. Journal of Policy Analysis and Management, 39(2), 510–532.
Chernozhukov, V., Wüthrich, K., & Zhu, Y. (2021). An exact and robust conformal inference method for counterfactual and synthetic controls. Journal of the American Statistical Association, 116(536), 1849–1864.
Di Stefano, R., & Mellace, G. (2024). The inclusive synthetic control method. arXiv:2403.17624.
Tashman, L. J. (2000). Out-of-sample tests of forecasting accuracy: An analysis and review. International Journal of Forecasting, 16(4), 437–450.
Software, R: synthdid · Synth · masc · augsynth · quadprog
Software, Stata: sdid (Clarke, Pailañir, Athey & Imbens) · synth (Abadie, Diamond & Hainmueller) · allsynth (Wiltshire)
Software, Python: mlsynth (Greathouse). The estimator mapping used in section 19 follows issue #312, which reads the same source paper this post replicates.
Companion tutorials on this site: Synthetic control on the Basque Country · Augmented synthetic control and the Kansas tax cuts · Synthetic difference-in-differences on Proposition 99 · Manual demeaning and two-way fixed effects · Multi-country augmented synthetic control