Tracking the Active Speaker Based on a Joint Audio-visual Observation Model
I. D. Gebru, S. Ba, G. Evangelidis, and R. Horaud (INRIA Grenoble)
ICCV Workshop on 3D Reconstruction and Understanding with Video and Sound, Santiago, Chile, Dec. 2015.
[showhide type=”Abstract”]Abstract: Any multi-party conversation system benefits from speaker diarization, that is, the assignment of speech signals among the participants. We here cast the diarization problem into a tracking formulation whereby the active speaker is detected and tracked over time. A probabilistic tracker exploits the on-image (spatial) coincidence of visual and auditory observations and infers a single latent variable which represents the identity of the active speaker. Both visual and auditory observations are explained by a recently proposed weighted-data mixture model, while several options for the speaking turns dynamics are fulfilled by a multi-case transition model. The modules that translate raw audio and visual data into on-image observations are also described in detail. The performance of the proposed tracker is tested on challenging data-sets that are available from recent contributions which are used as baselines for comparison. [/showhide]