Markov Decision Processes with Unobserved Confounders: A Causal Approach

Standard MDP estimation assumes observed state captures the information needed for valid transition and reward learning; unobserved confounders break that assumption.

The paper formulates sequential decision problems in causal-model terms, clarifying which policy values are identifiable under hidden confounding.

Use as the foundational citation whenever an RL method claims robustness to hidden confounding in sequential decisions.

The formulation highlights identifiability limits; it does not make unobserved confounding disappear without additional structure or data.