This will give you more details on importance sampling.

But, in short:

The variance depends on how you sample data. That is why you cannot use the old policy for too long for the practical purpose.

I don’t know what do you mean by “shrink”.

See Dec 9, 2018 for a previous answer that may relate to your question.