Abstract:

Humans are highly articulated, which leads to complex and idiosyncratic actions in space-time.  This complexity has challenged computational models of human action for some time now, and yet humans themselves are highly adept at parsing action.  In this talk, I will motivate the challenge of interpreting human action from spatial, temporal, and spatiotemporal points of view.  Then, I will present both computational and human perspectives on modeling action.  First, I will describe how video can be decomposed into a multilevel semantic scale-space using a Markov approximation framework.  Within this semantic scale-space, we have conducted a visual psychophysical study of how humans perceive action, and I will report our findings in that study.

Second, I will present a computational model for human action.  The method, called Action Bank, creates a high-level action space that is spanned by individual space-time actions.  Query videos are projected into this action space and non-linear classifiers are learned for recognition.  Experiments demonstrate how space-time, action-specific modeling can outperform conventional feature-based methods that do not leverage space-time continuity.  Third, I will bring these two perspectives together in a "full-circle" experiment leveraging ideas from both camps.  Time-permitting, I will relate these findings to other work in my group in computational neuroscience and cognitive robotics.

Bio:

Host:

Dr. Arun Ross and Dr. Xiaoming Liu