A method for estimating the contribution of an object to scene semantics

Zhiyuan Lin; Feng Zhu; Jianyu Wang; Qun Wang; Pengfei Zhao

doi:10.1117/12.3015832

29 January 2024 A method for estimating the contribution of an object to scene semantics

Zhiyuan Lin, Feng Zhu, Jianyu Wang, Qun Wang, Pengfei Zhao

Proceedings Volume 12984, Fourth International Conference on Computer Vision and Information Technology (CVIT 2023); 1298408 (2024) https://doi.org/10.1117/12.3015832
Event: 2023 4th International Conference on Computer Vision and Information Technology (CVIT 2023), 2023, Beijing, China

Abstract

The amount of semantic contribution of each object in an image to understand the scene is different. Understanding the relative importance of objects in a scene for scene semantics is critical for various computer vision applications, such as scene recognition and image captioning. In this paper, we refer to the contribution of an object to scene semantics as the degree of its gist and propose a method for Estimating the degree of the Gist of an Instance (EGoI). In the EGoI method, an object gist degree is estimated by the semantic features comparison strategy and the semantic distance comparison strategy. In the first strategy, the image is represented as a scene graph first, then the aggregation features and the node features of different graph node combinations are calculated to estimate the instance gist that is not in the combination of the nodes. In the second strategy, the captions of the complete and incomplete images are generated, then the semantic distance of these captions is used to estimate the instance gist deleted in the scene. Different strategies for estimating the gist degree of instances are tested in the experiments. The results show that the proposed method can effectively quantify the contribution of an instance to scene semantics. Among these strategies, the method that compares semantic features has better discriminative power for the gist degree of various instances in the scene.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Zhiyuan Lin, Feng Zhu, Jianyu Wang, Qun Wang, and Pengfei Zhao "A method for estimating the contribution of an object to scene semantics", Proc. SPIE 12984, Fourth International Conference on Computer Vision and Information Technology (CVIT 2023), 1298408 (29 January 2024); https://doi.org/10.1117/12.3015832

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available