How can we handle a change in the [[statistics|data generating process]] of the [[covariate]]s? - eg from [[train loss|training distribution]] to [[generalization error|deployment distribution]] ([[transfer learning in reinforcement learning|sim2real]] / [[transfer learning in reinforcement learning|overfitting in imitation learning]] / [[multitask|zero shot]] in extreme case) - or shifting over time: if we have model 1, 2, 3, .... that are trained on different periods of time, we can use **stacking** to get a robust predictor if the covariate shift is known, could use [[importance sampling]] (reweight by $p_{*}^{\text{test}}/p_{*}^{\text{train}}$) to estimate test error (eg for [[hyperparameter]] tuning with [[cross validation]]) # [[AI safety]] perspective sometimes termed "inner alignment" making sure the policy we end up with is actually pursuing the objective we want to obtain eg a [[mesa optimizer]] might learn to pursue a different objective some possible approaches: - [[adversarial training]] - [[eliciting latent knowledge|ELK]] / [[interpretability]]