Das Paper "A Fair Ranking and New Model for Panoptic Scene Graph Generation" von Julian Lorenz, Alexander Pest, Daniel Kienzle, Katja Ludwig und Rainer Lienhart wurde für die ECCV 2024 akzeptiert.

In der Veröffentlichung werden signifikante Fehler in der bisher verbreiteten Evaluierung von Panoptic Scene Graphs aufgezeigt. Die Autoren präsentieren eine Lösung für dieses Problem und werten existierende Veröffentlichungen auf den neuen Erkenntnissen aus.
Abschließend stellen die Autoren eine verbesserte neue Modellarchitektur für Panoptic Scene Graph Generation vor.

Weitere Informationen sind unter https://lorjul.github.io/fair-psgg/ zu finden.

 

In panoptic scene graph generation (PSGG), models retrieve interactions between objects in an image which are grounded by panoptic segmentation masks.
Previous evaluations on panoptic scene graphs have been subject to an erroneous evaluation protocol where multiple masks for the same object can lead to multiple relation distributions per mask-mask pair. This can be exploited to increase the final score. We correct this flaw and provide a fair ranking over a wide range of existing PSGG models.
The observed scores for existing methods increase by up to 7.4 mR@50 for all two-stage methods, while dropping by up to 19.3 mR@50 for all one-stage methods, highlighting the importance of a correct evaluation. Contrary to recent publications, we show that existing two-stage methods are competitive to one-stage methods. Building on this, we introduce the Decoupled SceneFormer (DSFormer), a novel two-stage model that outperforms all existing scene graph models by a large margin of +11 mR@50 and +10 mNgR@50 on the corrected evaluation, thus setting a new SOTA. As a core design principle, DSFormer encodes subject and object masks directly into feature space.