In general, the learning rate is decreasing. But an extra warming phase is added to gradually increase the rate at the beginning (which is not part of the cosine annealing).