When should you cluster standard errors? New wisdom from the oracle of econometrics (2023)

In ancient Greece, important decisions were never made without consulting the High Priestess in the Oracle at Delphi. She would bring wisdom from the gods, though that advice was sometimes vague or confusing, and often misinterpreted by mortals. Today I bring news that the High Priestess and the Priests (Athey, Abadie, Imbens and Wooldridge) have done itdeliverednew wisdom from the god of econometrics about the important decision ofWhen should you cluster standard errors?. This is definitely one of life's most important questions, as any avid seminar bingo player can surely attest. In case their paper is all Greek to you (half of it is literal), I'll try to summarize their recommendations so your standard errors are heaven.

1. You want to say something about the relationship between schooling and wages in a certain population group and use a random sample of workers from that population group. Then the standard errors for clustering do not need to be adjusted at all, even if clustering would change the standard errors.
2. The sample was chosen by randomly selecting 100 towns and villages within the country and then randomly selecting people in each. and your goal is to say something about the return to education in the general population. Here you should group the standard errors by village, since there are villages in the population of interest that are larger than those in the sample.
3. The same logic makes clear why you wouldn't group by age cohort in general (it seems unlikely that we would randomly select some age cohorts and not others and then try to say something about all age groups); and that we only want to group by industry if the sample would be drawn by randomly selecting a sample of industries and then sampling people from each industry.

Also in the second case, Abadie et al. Note that both the usual robust (Eicker-Huber-White or EHW) standard errors and the clustered standard errors (which they call Liang-Zeger or LZ standard errors) can do thisbothbe correct, it's just that they are correct for different estimates. That said, if you're content to just say something about the particular sample of people you have without trying to generalize to the population, the EHW standard errors are all you need; but if you want to say something about the wider population, the LZ standard errors are necessary.

Special case: Even when the sample is clustered, the EHW and LZ standard errors are equal when there is no heterogeneity in treatment effects.

Side note 1: This also reminds me of the Adjust Tilt Points commandnnmatchby Abadie (with another et al.) where you can get the narrower SATE standard errors for the sample or the broader PATE errors for the population.

Side note 2: This reason is rarely a justification for clustering in an impact evaluation. But Rosenzweig and UdryPapieron external validity indicates that we only observe treatment effects for certain points intime, and that if we want to say something more general about how our treatment behaves at other times, we need broader standard errors than we use to say just about our specific sample - which is very related to the point here about very clear what your estimate is.

The experimental design rationale for clustering
The second reason for clustering, which we are probably more familiar with, is when clusters of units are assigned to a treatment rather than individual units. Let's take the same equation as above but assume we have a binary treatment that assigns more schooling to people. So now we have:
Log(wages) = a +b*treatment + e

If treatment is then assigned at the individual level, there is no need for clustering (*). There has been much confusion as to how Chris Blattman wrote in two previous posts on this topic (the fabulous title "Cluster RuckandClusterjerk the sequel), and I still get occasional hints from reviewers that I should try clustering by industry or something similar in an individually randomized experiment. This Abadie et al. Paper is now finally a good reference to explain why this isn't necessary.
(*) unless you use multiple time periods and then want to group by person, since the randomization unit is individual, not individual.

What if your treatment is assigned at the village level? Then group by village. This is also why you would like tocluster-difference-in-differencesat the state level if you have a source of variation coming from differences between states and why "treating" like one side of a border versus the other is problematic (because you only have 2 clusters).

Add fixed effects
What if we sample at the city level, but then add fixed city effects to our mincer regression. Or we randomize at the city level but add fixed city effects. Do we still need to cluster at the city level?
The authors note that there is much confusion about the use of fixed effects clustering. As a general rule, if either sampling or allocation to treatment has been clustered, you still need to cluster. However, the authors show that cluster fits only make a fixed-effects fit when there is heterogeneity in treatment effects.

How is clustering done?
This is largely a paper aboutifclusters, nothowto group. There is, of course, a very different debate about when to rely on asymptotics, bootstrapping, or randomization inference approaches. They show, using asymptotic approximations, that the standard Liang-Zeger cluster fit is generally conservative, and they provide an alternative cluster-adjusted variance estimator that can be used when there are differences in treatment allocation within clusters and you know the fraction of clusters sampled. However, because of the concern that, with the sample sizes used in many experiments, asymptotic standard errors may not be conservative enough, you should be cautious about using such a fit for typical sample sizes.

subjects