Course Goals

The goal of the course is to prepare the students for industrial positions in the emerging field of multimedia and for pursuing further graduate studies in signal processing. Through a set of carefully designed machine problems, the student learns important tools in audio-visual signal processing, analysis, and synthesis, and their applications to biometrics, human-computer interaction, and multimedia indexing and search.

Instructional Objectives

1. After Machine Problem 1 (MP1), Week 3 of the semester, the students should be able to:
   • Understand speech features (esp. cepstrum coefficients) and nearest-neighbor pattern classifiers and their applications to speech recognition and speaker identification
2. After MP2, Week 5 of the semester, the students should be able to:
   • Understand principal component analysis and linear discriminant analysis, and their applications to face recognition
3. After MP3, Week 7 of the semester, the students should be able to:
   • Understand maximum likelihood (ML) classifies, Bayesian networks, and multimodal fusion, and their applications to audio-visual person identification
4. After MP4, Week 9 of the semester, the students should be able to:
   • Understand hidden Markov model (HMM), including algorithms for learning, inference, and decoding, and its application to audio-visual speech recognition
5. After MP5, Week 11 of the semester, the students should be able to:
   • Understand 3D face modeling and animation and applications to speech-driven lip movement in an audio-visual avatar (synthetic talking head)
6. After MP6, Week 13 of the semester, the students should be able to:
   • Understand and practice content-based image retrieval with relevance feedback
7. After MP7, Week 15 of the semester, the students should be able to:
   • Understand structuring and indexing video data, and algorithms for video shot segmentation based on audio and visual cues