(Unseen) Event Recognition via Semantic Compositionality

Stöttinger, Julian and Uijlings, Jasper R. R. and Pandey, Anand K. and Sebe, Nicu and Giunchiglia, Fausto (2012) (Unseen) Event Recognition via Semantic Compositionality. Trento : Università degli Studi di Trento. ISBN E-ISBN 978-1-4673-1227-1; Print ISBN: 978-1-4673-1226-4

PDF - Full Text
Download (1090Kb) | Preview
    Official URL: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?...


    Since high-level events in images (e.g. "dinner", "motorcycle stunt", etc.) may not be directly correlated with their visual appearance, low-level visual features do not carry enough semantics to classify such events satisfactorily. This paper explores a fully compositional approach for event based image retrieval which is able to overcome this shortcoming. Furthermore, the approach is fully scalable in both adding new events and new primitives. Using the Pascal VOC 2007 dataset, our contributions are the following: (i) We apply the Faceted Analysis-Synthesis Theory (FAST) to build a hierarchy of 228 high-level events. (ii) We show that rule-based classifiers are better suited for compositional recognition of events than SVMs. In addition, rule-based classifiers provide semantically meaningful event descriptions which help bridging the semantic gap. (iii) We demonstrate that compositionality enables unseen event recognition: we can use rules learned from non-visual cues, together with object detectors to get reasonable performance on unseen event categories.

    Item Type: Departmental Technical Report
    FP7 Grant Agreement Number: info:eu-repo/grantAgreement/EC/FP7/287704, info:eu-repo/grantAgreement/EC/FP7/248984
    Publisher policy set phrase: "(c) 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works."
    Department or Research center: Information Engineering and Computer Science
    Subjects: Q Science > QA Mathematics > QA076 Computer software
    Additional Information: Also published in Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 16-21 June 2012.
    Report Number: DISI-12-020
    Repository staff approval on: 04 Jul 2013 10:00

    Actions (login required)

    View Item