Research

EEG analysis for implicit tagging

it's me all strapped in for an EEG experiment
EEG signal differences for matching/nonmatching tags

I'm currently working on using EEG analysis for the implicit tagging of multimedia data. That is, we try to generate or validate tags for multimedia content, based on the users' brainwaves as they watch the content. I did an experiment showing users a set of videos, along with matching or non-matching tags. As it turns out, there are significant differences in EEG signals between the trials where matcing tags are displayed and those with non-matching tags. See also the slides of my presentation on this at the ABCI 2009 workshop. The dataset we collected is now available.

Spatiotemporal keypoint detection

An example showing detected keypoints in space and time
stability of region detection at different scales
comparison of stability of detectors at different scales

Spatio-temporal interest region detectors can be used in the analysis of video to determine sparse, informative regions as candidates for feature extraction. I introduced the new FAST-3D detector, loosely based on the FAST spatial interest region detector and compared it to existing interest region detectors by measuring the similarity between detected interest regions in original and transformed versions of videos. The FAST-3D detector performed on par with the other detectors in this test, while showing a significant increase in speed. See also my slides from WIAMIS 2009.

Facial Expression Recognition

an example of facial Action Units
a facial expression and its detected motion
The HMM used to model the timing of expressions

For my MSc. Project I did some research on automatic recognition of facial expressions. The idea is to automatically detect facial Action Units (AUs) and their temporal segments in frontal-view face videos. I used a non-rigid registration technique to determine the motion in the input videos. Each video was then segmented in to a set of regions, from which features were extracted. A combination of an HMM and a boosting algorithm then detects the presence of AUs. See also the slides from my talk at the Face and Gesture Recognition 2008 conference.