Recognizing Manipulation Actions from State-Transformations (Technical Report)

Nachwa Aboubakr 1 James Crowley 2 Rémi Ronfard 3
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
3 IMAGINE - Intuitive Modeling and Animation for Interactive Graphics & Narrative Environments
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : Manipulation actions transform objects from an initial state into a final state. In this paper, we report on the use of object state transitions as a mean for recognizing manipulation actions. Our method is inspired by the intuition that object states are visually more apparent than actions thus provide information that is complementary to spatio-temporal action recognition. We start by defining a state transition matrix that maps action verbs into a pre-state and a post-state. We extract keyframes at regular intervals from the video sequence and use these to recognize objects and object states. Change in object state are then used to predict action verbs. We report results on the EPIC kitchen action recognition challenge.
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download
Contributor : Nachwa Aboubakr <>
Submitted on : Tuesday, July 30, 2019 - 2:21:57 PM
Last modification on : Wednesday, August 21, 2019 - 10:22:06 AM


Files produced by the author(s)


  • HAL Id : hal-02197595, version 1


Nachwa Aboubakr, James Crowley, Rémi Ronfard. Recognizing Manipulation Actions from State-Transformations (Technical Report). [Research Report] Univ. Grenoble Alps, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France. 2019. ⟨hal-02197595⟩



Record views


Files downloads