It is updated in an alternative step. While I can think of many reasons why you don’t want to do both at the same time — including better generalization, less co-relationship between training data. But I will not discourage anyone to give it a try anyway. However, I believe there is a good chance that it will not work. The generator is changing from what last detected. Probably, it hurts the training stability pretty bad and too many memorizations instead of learning in general.