In causal settings, such as instrumental variable settings, it is well known
that estimators based on ordinary least squares (OLS) can yield biased and
non-consistent estimates of the causal parameters. This is partially overcome
by two-stage least squares (TSLS) estimators. These are, under weak
assumptions, consistent but do not have desirable finite sample properties: in
many models, for example, they do not have finite moments. The set of K-class
estimators can be seen as a non-linear interpolation between OLS and TSLS and
are known to have improved finite sample properties. Recently, in causal
discovery, invariance properties such as the moment criterion which TSLS
estimators leverage have been exploited for causal structure learning: e.g., in
cases, where the causal parameter is not identifiable, some structure of the
non-zero components may be identified, and coverage guarantees are available.
Subsequently, anchor regression has been proposed to trade-off invariance and
predictability. The resulting estimator is shown to have optimal predictive
performance under bounded shift interventions. In this paper, we show that the
concepts of anchor regression and K-class estimators are closely related.
Establishing this connection comes with two benefits: (1) It enables us to
prove robustness properties for existing K-class estimators when considering
distributional shifts. And, (2), we propose a novel estimator in instrumental
variable settings by minimizing the mean squared prediction error subject to
the constraint that the estimator lies in an asymptotically valid confidence
region of the causal parameter. We call this estimator PULSE (p-uncorrelated
least squares estimator) and show that it can be computed efficiently, even
though the underlying optimization problem is non-convex. We further prove that
it is consistent.
