Piecewise Constant Policy Approximations to Hamilton