FPN addresses this multi-scale problem. https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c Take a look and it will explain some of the reason. BTW, SSD’s CONV4_3 is 38x38. So this resolution already triggers bad performance comparing with Faster R-CNN for small objects in COCO dataset.
