mAP is measured independent of the threshold. So you should use all. But you can stop when the area under the curve becomes too small.