A Size-Aware Graph Embedding Approach to Remote Sensing Image Captioning With Object Relative Size Information

Remote sensing image captioning is the task of automatically generating descriptive texts for remotely sensed scenes and objects. A common shortcoming of existing methods is the inadequate consideration of object size, which often leads to captions that either omit size information or provide imprecise size descriptions. To overcome this deficiency, we develop a novel framework composed of three modules: (a) an object confirmation and relative size estimation module, (b) a graph construction and graph convolution module, and (c) a caption generation module. Our framework comprehensively characterizes object size to generate more quantitatively informative captions. Furthermore, we introduce a new evaluation metric, SizeNum-Meteor, designed to explicitly evaluate the correctness of object count and relative size information in generated captions. This provides a more comprehensive assessment, as standard metrics typically neglect the evaluation of object size. In addition, we construct extended benchmarks by enriching existing datasets with explicit annotations of object count and relative size. Extensive experiments on three existing benchmark datasets (i.e., UCM, Sydney, and RSICD) and the benchmarks we construct (i.e., UCM-N-S, Sydney-N-S, and RSICD-N-S) demonstrate that our framework achieves superior performance on both standard metrics and the proposed SizeNum-Meteor.